Adabas Security
ADABAS SECURITYADABAS provides the following facilities to prevent unauthorized access to and/or updating of ADABAS database files: ADABAS Data Encryption (ciphering) which provides data security; ADABAS Multiclient Files to control access to records in a file; ADABAS SECURITY and the related security utility ADASCR, a selectable unit, which provides selective user access/update protection at a file, field, and field value level; and ADABAS EXTERNAL SECURITY INTERFACE (ADAESI), a selectable unit, whichprovides control of ADABAS resources at a database/utility, command, or file level through the widely used non-Software AG security packages RACF, CA–ACF2 and CA–TOP–SECRET.ADAESI is available for MVS/ESA and OS IV/F4 (FACOM) only.Security is accomplished by comparing passwords and authorization levels.Data EncryptionData encryption is an integral feature of ADABAS and requires no options or extra modules.Data may be enciphered before being placed in the database. The user must provide the cipher key at the time records are stored. This key is not stored and must be available to request or decipher the data. This minimizes the chances that data encryption can be compromised if unauthorized access to the system occurs.To retain maximum control over cipher codes, an ADABAS user exit program can be created to insert the currently valid cipher code into user applications; this removes the need to make the codes known to users, and protects the file from corruption that can occur by adding data that is encrypted with the wrong cipher code.Multiclient FilesAlso available as an integral feature of ADABAS that requires no options or special modules is the multiclient file.A single ADABAS physical file defined as “multiclient” can store records for multiple users or groups of users. The multiclient feature divides the physical file into multiple logical files by attaching an internal owner ID to each record.The owner ID is assigned to a user ID. A user ID can have only one owner ID, but an owner ID can belong to more than one user. Each user can access only the subset of records that is associated with the user’s owner ID.Note:For any installed external security package such as RACF or CA-TOP SECRET, a user is still identified by either NATURAL ETID or LOGON ID.All database requests to multiclient files are handled by the ADABAS nucleus.ADABAS SECURITY and ADASCRAccess/update control is available only with ADABAS SECURITY and the related security utility ADASCR that defines and controls ADABAS SECURITY functions.ADABAS SECURITY provides two types of access/update protection: “Access-/update-level” protection applies a basic level of security on a file-by-file basis.Access/update protection can be defined for some files and not for others. It restricts use of a file or field within the file to those having an appropriate access/update profile definition and a password specified by the user of the file.Access/update permission values ranging from 0 to 14 are defined for each user and attached to that user’s password, and each protected file (and selected field or fields, if desired) has equivalent access/update “threshold” protection values of the same range. Only a user whose permission value equals or is greater than the protection level of the specified file (and, when applicable, field) is permitted to perform that operation type (access or update) on the file or field. An access/update permission level of 0 only allows access/update of unprotected files or fields with protection level 0 or no defined protection password.ADABAS Security “Value-level” protection applies restrictions on the type and range of values that can be accessed or updated in specific fields. The restrictions are applied according to user password (files with fields using value-level protection must be password-protected), can be for specific values or for value ranges, and can be either “accept” or “reject” criteria.ADABAS EXTERNAL SECURITY INTERFACE(ADAESI)ADAESI allows the definition and protection of ADABAS resources using standard external (i.e., non-Software AG) security packages such as CA-ACF2, CA-TOP SECRET, and RACF installed on ADABAS systems running under MVS/ESA and OS IV/F4 (FACOM).Generally, a security package allows the system administrator to authorize a user’s access to system resources. The security package then monitors all users and their resource usage to ensure that no unauthorized access or change occurs. Attempts by unauthorized users to use either the system or specific system resources are recorded and reported.A user profile, which can be for a single user or a group of users, defines which system hardware and software resources a user is allowed to use. A resource profile defines access/update privileges for one or more devices, volumes, and/or programs (resources that must be used together to perform certain functions can be defined together in the same profile). When a user logs on to the system, the security package uses the user’s logon ID to identify that user’s profile. Each time the user attempts to perform a task or access information, the security package uses information in its resource profiles to allow or deny access. Using the profileconcept, the security package expands the single point of authorization—the logon ID—to provide extensive control over all system resources. ADAESI extends the ability of the related security packages to include the ADABAS database and users. Using ADAESI, the security packages can define and control access to the followingADABAS resources: database nucleus; database files; database commands; utilities(MODE=MULTI only); and ADABAS operator commands issued from an MVS console.Related Security OptionsADABAS ONLINE SYSTEM/Basic Services SecurityThe DBA facility ADABAS ONLINE SYSTEM/Basic Services also provides a security facility for restricting access to the ADABAS online facilities. Basic Services Security requires NATURAL SECURITY as a prerequisite.The NATURAL SECURITY system provides extensive security for ADABAS/NATURALusers. See the NATURAL SECURITY Manual for additional information.
Utilities
4ADABAS UTILITIESDatabase services such as loading or deleting files are handled by an integrated set of online andbatch-mode “utilities”. Most utilities can be run in parallel with normal database activity topreclude interruption of daily production. See the ADABAS Utilities Manual for moreinformation.ADABAS utilities address initial design and load operations, backup/restore/recovery routines,database modification routines, and audit/control/tuning procedures.Note:See the Optional Extensions chapter starting on page 71 for information about ADABASONLINE SYSTEM/Basic Services, a menu-driven, interactive DBA tool.Initial Design and Load OperationsADACMP : Compress / DecompressADACMP COMPRESS is used to edit and compress data records to be loaded into the databaseusing ADALOD; ADACMP DECOMPRESS is used to decompress individual files for datastructure or field definition changes, or for use as input to non-ADABAS programs.COMPRESSInputADACMP input data must be in a sequential dataset/file. Indexed sequential and VSAM inputcannot be used. The records may be fixed, variable, or of undefined length. The maximum inputrecord length permitted depends on the operating system. The maximum compressed recordlength is restricted by the Data Storage block size in use and the maximum compressed recordlength set for the file (see the MAXRECL parameter, ADALOD utility). The input records canbe in either blocked or unblocked format.It is possible to omit the input dataset if the parameter NUMREC=0 is supplied.ADABAS Concepts and FacilitiesThe logical structure and characteristics of the data for input to COMPRESS are described withfield definitions statements (FNDEF to define fields or groups of fields; SUBFN and SUPFNto define sub- or superfields, respectively; HYPDE, PHONDE, SUBDE, and SUPDE to definevarious types of descriptors). Field definitions are used to create the ADABAS Field DefinitionTable (FDT).By default, input data records are processed in the order of the field definition statements. TheFORMAT parameter allows you to change the order of field processing or skip fields.ProcessingADACMP COMPRESS edits and compresses the data records.Editing includes checking each field defined with a “packed” (P) or “unpacked” (U) format toensure that the field value is numeric and in the correct format. Any record that contains invaliddata is written to the ADACMP error dataset and is not written to the compressed dataset.ADABAS user exit 6 can be used to specify additional editing to be performed during ADACMPCOMPRESS processing. See the ADABAS DBA Reference Manual for information about userexits.Compression includes removing trailing blanks from alphanumeric fields; removing leadingzeros from numeric fields; removing trailing zeros in floating-point format fields; and packingnumeric unpacked fields. Fields with the fixed (FI) option are not compressed, and empty fieldslocated at the end of the record are neither stored nor compressed. Null value fields are processeddifferently depending on options being used. SQL null value processing is supported.OutputProcessed data records are written out together with the file definition information to asequential dataset with the “variable blocked” record format. This dataset, or several suchdatasets from multiple ADACMP executions, can be used as input to the ADALOD utility. Thedataset can be used as input to ADALOD even if it contains no records, meaning that no recordswere provided on the input dataset or all records were rejected during editing.The ADACMP processing report indicates the approximate amount of space required in DataStorage for the compressed records by device type (specified with the DEVICE parameter) andfor Data Storage padding factors between 5 and 30 percent. The compression rate is computedbased on the real amount of data used as input to the compression routine.Utilities 451DECOMPRESSADACMP DECOMPRESS accepts as input data records from existing ADABAS files, eitherdirectly without separate file unloading, or already unloaded with the ADAULD utility. If a fileis directly decompressed, it is unloaded without FDT information as part of the decomperssionprocess, which can save time when decompressing larger files.Direct decompression of multiclient files can be limited to records for a specific user only whena valid owner ID (ETID parameter) is specified.The FORMAT parameter may be used to decompress the record to a format other than thatspecified by the FDT. This is particularly useful when the FDT of an existing file is to bechanged.Processed data records are written to a sequential dataset with the “variable blocked” recordformat. Rejected data records are written to the error dataset.ADALOD : LoaderThe ADALOD LOAD function loads a file into the database. Compressed records produced bythe ADACMP or ADAULD utility may be used as input.ADALOD loads each compressed record into Data Storage, builds the Address Converter forthe file, and enters the field definitions for the file into the Field Definition Table (FDT).ADALOD also extracts the values for all descriptors in the file together with the ISNs of allrecords in which the value is present, to an intermediate dataset. This dataset is then sorted intovalue/ISN sequence and entered into the Associator inverted lists.The ADALOD UPDATE function is used to add or delete a large number of records to/from anADABAS file. The UPDATE function requires considerably less processing time than therepetitive execution of the ADABAS add/delete record commands. Records to be added maybe the compressed records produced by the ADACMP or ADAULD utility. The ISNs of recordsto be deleted can be provided either in an input dataset or by using control statements.Records may be added and other records deleted during a single execution of ADALOD.ADABAS Concepts and Facilities ADAULD : UnloadThe ADAULD utility unloads an ADABAS file. ADABAS files are unloaded to permit the data to be processed by a non-ADABAS program. In this case, the file must also bedecompressed after unloading using the DECOMPRESS function of the ADACMP utility. create one or more test files, all of which contain the same data. This procedure requires thata file be unloaded, and then reloaded as a test file with a different file number. change the Field Definition Table (FDT). This requires that the file be unloaded, decompressed,compressed using the modified field definitions, and reloaded. If the ADADBS utility is usedto add field definitions to a file, the file does not need to be unloaded first.The sequence in which the records are unloaded may bephysical the order in which they are physically positioned within Data Storage.logical a sequence controlled by the values of a user-specified descriptor.ISN ascending ISN sequence.The unloaded record output is in compressed format. The output records have the same formatas the records produced by the ADACMP utility.Utilities Backup / Restore / Recovery RoutinesADAPLP : Protection Log / Work PrintThe ADAPLP utility prints data protection records contained on the ADABAS Work dataset orthe ADABAS data protection log. You can specify whether to printALL all protection records—the defaultASSO just Associator protection recordsDATA just Data Storage protection recordsC1 records resulting from ADABAS C1 commandsC5 records resulting from ADABAS C5 commandsEEKZ records written at completion of a nucleus buffer flushET records resulting from ADABAS ET commandsREPR Work dataset records used by Autorestart to repair the indexSAVO records resulting from online SAVE database/file operationsVEKZ records written at completion of update commandsThe number of protection records printed can be reduced even more by specifying a file, ISN,or RABN.ADARAI : Recovery AidThe ADABAS Recovery Aid utility ADARAI can be used to automate and optimize databaserecovery. See also the restart/recovery information in the ADABAS Operations Manual.ADARAI supports all ADABAS-compatible tape management systems.The ADARAI utility prepares the recovery log files (RLOGs), lists the information containedin the RLOGs, creates the job control statements to recover the database, and disables ADARAIlogging.The RLOGs record the information about datasets, utility parameters, and protection logsneeded to build the recovery job control statements. There are two RLOGs: the primary file(DDRLOGR1 or RLOGR1) and a mirror file (DDRLOGM1 or RLOGM1). The mirror file mustalways exist; it can be used to recover the database if the primary file is not accessible.ADABAS Concepts and Facilities 454Information is stored on the RLOG by generations. A “generation” includes all activity betweenconsecutive ADASAV SAVE/RESTORE (database) and/or RESTORE GCB operations. Thefirst generation includes the first ADASAV SAVE/RESTORE (database) or RESTORE GCBoperation and extends to (but excludes) the second.Minimally, the RLOG retains the number of generations specified by the MINGENS parameterduring the ADARAI PREPARE step. However, a maximum of 32 generations will be stored onthe RLOG if there is enough space available.Systems using the Recovery Aid feature require a formatted recovery log (RLOG) dataset pair(normal and mirrored), which must first be formatted with the ADAFRM utility and thendefined using the ADARAI utility.ADARES : RestartThe ADARES utility performs functions related to database recovery: BACKOUT removes all the updates applied between two checkpoints. The checkpoints usedare normally the result of a non-synchronized checkpoint command (C1) but may also besynchronized checkpoints. The complete database may be included in the backout process, orbackout may be limited to selected files. CLCOPY copies a command log dataset from disk to a sequential dataset. This function isnecessary only if dual command logging is in effect for an ADABAS session. COPY copies a sequential ADABAS protection log dataset. This function should be executedif the ADABAS session in which the sequential protection log dataset was created wasterminated abnormally. PLCOPY copies a protection log dataset from disk to a sequential dataset. This function isnecessary only if dual protection logging is in effect for an ADABAS session. REGENERATE reapplies all the updates performed between two user-specified checkpoints.The checkpoints specified may be the result of a non-synchronized checkpoint command (C1)but may also be synchronized checkpoints. The REGENERATE function may process all filesor be limited to one or more files. It is most often used after the database (or one or more files)has been restored to a previous status with the RESTORE or RESTONL function of theADASAV utility. REPAIR repairs one or more blocks in Data Storage that, for any reason, have become unusable.The most recent save tape of the database and any protection log tapes created thereafter are usedas input to this function.Utilities ADASAV : Save / Restore Database or FilesThe ADASAV utility saves and restores the contents of the database, or one or more files, to orfrom a sequential dataset. ADASAV should be run as often as required for the number and sizeof the files contained in the database, and the amount and type of updating. For large databases,ADASAV functions may be run in parallel for the various disk packs on which the database iscontained.Special ADASAV functions are available for use with the ADABAS DELTA SAVE FACILITY.For more information, see the ADABAS DELTA SAVE FACILITY Manual.RESTONL functions restore from a SAVE dataset created while the ADABAS nucleus wasactive (that is, online); RESTORE functions restore from a SAVE dataset created while theADABAS nucleus was inactive (that is, offline).RESTONL and RESTORE have the subfunctions GCB, FILES, and FMOVE: Without a subfunction, RESTONL and RESTORE restore entire databases. With the GCB subfunction, they restore the General Control Block, Associator RABNs 2–30of the database, and specified files. With the FILES subfunction, they restore one or more files into an existing database to theiroriginal RABNs. With the FMOVE subfunction, they restore one or more files into an existing database to anyfree space, allowing changes to extent sizes.If changes occurred during the online SAVE, the RESTONL function is followed automaticallyby the RESTPLOG function. RESTPLOG applies the updates that occurred during, andtherefore were not included in, the online SAVE.RESTPLOG is also executed following a RESTONL or RESTONL FILES function that endedbefore the protection log (PLOG) updates were completely restored. RESTPLOG applies thedatabase updates not applied by the unsuccessful RESTONL function.The SAVE function to save a database or one or more files may be executed while the ADABASnucleus is active (online) or inactive (offline). If the Recovery Aid option is active, a SAVEdatabase operation begins a new RLOG generation.ADABAS Concepts and FacilitiesADASEL : Select Protection DataThe ADASEL utility selects information in the ADABAS sequential (SIBA) or dual (PLOG)protection log. ADASEL decompresses the information and writes it to a print dataset(DDDRUCK or DRUCK) or to a user-specified output dataset.The protection log contains information on all updates applied to the database during a givenADABAS session. Information selected by ADASEL can be used for auditing or as input to aNATURAL or non-ADABAS program.You can select before-images, after-images, or both for new, updated, and deleted records. Youcan also select data written to the protection log by an ADABAS C5 command.Database Modification RoutinesADADBS : Database ServicesAll ADADBS functions can also be performed using ADABAS ONLINE SYSTEM/BasicServices. When the ADABAS Recovery Aid is active, using Basic Services is preferable for filechange operations because it writes checkpoints that are necessary for recovery operation.ADADBS offers a variety of functions, any number of which may be performed during a singleexecution of the utility.Database FunctionsThe ADD function adds a new dataset to the Associator or Data Storage. The dataset to be addedmay be on a device type that is the same as that currently being used or different. A maximumof five datasets each may be assigned to the Associator and Data Storage.The DECREASE function decreases the size of the last dataset currently being used forAssociator or Data Storage. The space to be released must be available in the Free Space Table(FST).The DECREASE function does not deallocate any of the specified physical extent space. Todeallocate space, you must decrease the database with the DECREASE function; save it withADASAV SAVE; reformat the datasets with ADAFRM; and restore the database with ADASAV.Utilities The INCREASE function increases the size of the last dataset currently being used for theAssociator or Data Storage. This function may be executed any number of times for theAssociator. The maximum of five Data Storage Space Tables (DSSTs) limits Data Storageincreases to four before all five Data Storage extents must be combined into a single extent witheither the REORASSO or REORDB function of the ADAORD utility.The NEWALTS function assigns additional alternate RABN blocks for the Associator or DataStorage. Existing alternate RABN definitions cannot be changed with NEWALTS.Alternate RABN blocks can be used in place of RABN blocks that are physically damaged. IfADABAS detects that a write operation is not successful, ADABAS selects an alternate blockof the same type, and the data is written to it. This alternate RABN is then used in place of theoriginal block for all subsequent read and write operations.The RENAME function changes the name assigned to a (file or) database. If a file is notspecified or is specified with file number zero, the database is renamed.File FunctionsThe ALLOCATE function is used to allocate a logical extent (an Address Converter, DataStorage, Normal or Upper Index) of a specific size. The DEALLOCATE function is used todeallocate a logical extent. Only one extent may be allocated or deallocated per ADADBSexecution.The CHANGE function changes the standard length of an ADABAS field. No modifications torecords in Data Storage are made by this function. The user is, therefore, responsible forpreventing references to the field that would cause invalid results because of an inconsistencybetween the new standard length as defined to ADABAS and the actual number of bytescontained in the record.The DELETE function deletes an ADABAS file from the database. The file may not be coupled.If an ADABAS expanded file is specified, the complete expanded file (the anchor and allcomponent files) is deleted. This function results in the deallocation of all logical extentsassigned to the file. The released space may be used for a new file or for a new extent of anexisting file.The DSREUSE function determines, for a specified file, whether Data Storage blocks thatbecome free as a result of record deletion are reused. Block reuse is originally determined whena file is loaded into the database with the ADALOD FILE function, or when the system file isdefined with the ADADEF DEFINE function. In both cases, block reuse defaults to “YES” ifnot specified.ADABAS Concepts and Facilities The ISNREUSE function determines, for a specified file, whether ADABAS reuses the ISN ofa deleted record for a new record. If not, each new record is assigned the next higher unused ISN.For a specified ADABAS file that is not a system file, the MODFCB function modifiesparameters such as file padding factors for the Associator or Data Storage; maximum size ofsecondary logical extent allocations for Data Storage, Normal Index, and Upper Index;maximum compressed record length permitted; and whether a user program is allowed toperform a file refresh operation by issuing a special E1 command.The NEWFIELD function adds one or more fields to a specified ADABAS file that is not asystem file. The new field definition is added to the end of the Field Description Table (FDT).NEWFIELD cannot be used to specify actual Data Storage data for the new field; the data canbe specified later using ADABAS add or update commands, or NATURAL commands.The REFRESH function sets the file to “0” records loaded; sets the first extent for the AddressConverter, Data Storage, Normal Index, and Upper Index to “empty” status; and deallocatesother extents.The RELEASE function releases a descriptor from descriptor status. All space currentlyoccupied in the Associator inverted list for this descriptor is released. The space can then bereused for this file by reordering or by ADALOD UPDATE. No changes are made to DataStorage.The RENAME function changes the name assigned to a file or database. If a file is not specifiedor is specified with file number zero, the database is renamed.The RENUMBER function changes the number of an ADABAS file that is not a system file.If the new number specified is already assigned to another file, the RENUMBER function willnot execute.The UNCOUPLE function eliminates the coupling relationship between two files.Other FunctionsThe CVOLSER function prints the ADABAS file extents that are contained on a disk volumespecified by its volume serial number.The DELCP function deletes checkpoint information recorded up to and including a specifieddate; checkpoint information recorded after the date specified is not deleted. After runningADADBS DELCP, the remaining records are reassigned ISNs to include those ISNs madeavailable when the checkpoint records were deleted. The lower ISNs are assigned but thechronological order of checkpoints is maintained.Utilities The OPERCOM function issues operator commands to the ADABAS nucleus. ADABAS issuesa message to the operator, confirming command execution. Only the update ADABAS nucleus(NUC01) running in an ADASMP multiprocessing or an ADAPLEX+ nucleus cluster is allowedto run ADABAS utilities such as ADADBS. The optional SMPID or PLXID parameter,respectively, allows you to redirect the OPERCOM commands to another (read-only) nucleusin the cluster for execution, just as though the command had been issued by a locally runADADBS OPERCOM operation.The PRIORITY function sets or changes the ADABAS priority of a user. A user’s priority canrange from 0 (the lowest) to 255 (the highest, and the default). The priority value is added tothe operating system priority by the interregion communications mechanism. The user forwhich a priority is to be set or changed is identified by the same user ID provided in theADABAS Control Block (OP command, Additions 1 field).The RECOVER function recovers space allocated by rebuilding the Free Space Table (FST).RECOVER subtracts file, DSST, and alternate RABN extents from the total available space.The REFRESHSTATS function resets statistical values maintained by the ADABAS nucleus forits current session. Parameters may be used to restrict the function to particular groups ofstatistical values: ALL (the default) resets values for the combination of CMDUSAGE, COUNTERS,FILEUSAGE, POOLUSAGE, and THREADUSAGE. CMDUSAGE resets the counters for ADABAS direct call commands such as Lx, Sx, or A1. COUNTERS resets the counter fields for local or remote, physical or logical calls, ADABASSTAR calls, format translations, format overwrites, autorestarts, protection log switches, bufferflushes, and command throw-backs. FILEUSAGE resets the count of commands for each file. POOLUSAGE resets the high-water marks for the nucleus pools such as the Work pool, theCommand Queue, or the User Queue. THREADUSAGE resets the count of commands for each ADABAS thread.ADABAS maintains a list of the files used by each ADABAS utility in the Data Integrity Block(DIB). The DDIB operator command (or ADABAS ONLINE SYSTEM/Basic Services)displays this block to determine which jobs are using which files. A utility removes its entryfrom the DIB when it terminates normally. If a utility terminates abnormally (for example, thejob is cancelled by the operator), the files used by that utility remain “in use”. The RESETDIBfunction releases any such files and resets the DIB entries for a specified job and/or a particularutility execution.ADABAS Concepts and Facilities ADADEF : Define a DatabaseThe ADADEF utility is used to define a new database, including the Checkpoint file, or to definea new Work file (NEWWORK function) for an existing database.Databases are defined with name, ID, components (Associator, Data Storage, and Work) withdevice type and size.ADABAS uses certain files to store system information. The Checkpoint file is used to storecheckpoint data as well as user data provided with the ADABAS CL and ET commands. It isrequired and must be specified in the ADADEF DEFINE (database) function.Before database components (Associator, Data Storage, and Work) can be defined withADADEF, each must be formatted by the ADAFRM utility.ADAFRM : Format DatasetsThe ADAFRM utility formats the ADABAS direct access (DASD) datasets; that is, theAssociator, Data Storage, and Work datasets as well as the intermediate storage (Temp, Sort,and command/dual protection/recovery logging) datasets.Formatting with ADAFRM comprises two basic operations: creating blocks (that is, RABNs)on the specified tracks/cylinders; and filling the created blocks with binary zeros (nulls).Any new dataset must be formatted before it can be used by the ADABAS nucleus or anADABAS utility. After increasing a dataset with the ADADBS INCREASE or ADD function,new RABNs must also be formatted.ADAFRM also provides functions to “reset” existing Associator, Data Storage, or Work blocksto binary zeros (nulls).More than one ADAFRM function (ASSOFRM, DATAFRM, RLOGFRM, and so on) can beperformed in the same job. However, each function must be specified on separate statements.Utilities ADAINV : InvertThe ADAINV utility is used to create a descriptor (INVERT function); or couple two files (COUPLE function).The INVERT function modifies the Field Definition Table (FDT) to indicate that the specified field is a descriptor; and adds all values and corresponding ISN lists for the field to the inverted list.The newly defined descriptor may then be used in the same manner as any other descriptor. Thisfunction may also be used to create a subdescriptor, superdescriptor, phonetic descriptor, orhyperdescriptor.The COUPLE function adds a common descriptor to two files (updates their inverted lists). Anytwo files may be coupled provided that a common descriptor with identical format and lengthdefinitions is present in both files. A single file may be coupled with up to 18 other files, butonly one coupling relationship may exist between any two files at any one time. A file may notbe coupled to itself.Note:Only files with numbers 255 or lower can be coupled.Changes affecting a coupled file’s inverted lists are automatically made to the other file. TheDBA should consider the additional overhead required to update the coupling lists when thedescriptor used as the basis for coupling is updated, or when records are added to or deleted fromeither file. For example, if a field used as the basis for coupling contains a large number of nullvalues and is not defined with the NU (null suppression) option, the result may be a significantincrease in execution time and required disk space to store the coupling lists.An interrupted ADAINV operation can be restarted without first having to restore the file.ADABAS Concepts and Facilities ADAORD : ReorderThree types of functions are available within the ADAORD utility; only one function may beexecuted during a given execution of ADAORD.Reorder FunctionsThe REORASSO function physically reorders all Associator blocks for all files; REORFASSOreorders the Associator for a single file. This eliminates Associator space fragmentation, andcombines multiple Address Converter, Normal and Upper Index, and Data Storage Space Table(DSST) component extents into a single logical extent for each component.The REORDATA function reorders Data Storage for all files in the database; REORFDATAreorders Data Storage for a single file. This condenses extents containing only empty blocks,and also eliminates any Data Storage fragmentation caused by file deletion.The REORDB function performs both the REORASSO and REORDATA functions in a singleADAORD execution; the REORFILE function performs both the REORFASSO andREORFDATA functions in a single ADAORD execution. The records may be reordered in thelogical sequence by a descriptor, by ISN, or in the current sequence.Restructure FunctionsThe RESTURCTURE functions are used to relocate a database or specified files to a differentphysical device.The RESTRUCTUREDB function unloads an entire database to a sequential dataset;RESTRUCTUREF unloads one or more files to a sequential dataset. This dataset can be usedas input to the STORE function.Store FunctionThe STORE function loads one or more files into an existing database using the output producedby the RESTRUCTURE functions or the REORDB function.Utilities Audit / Control / Tuning ProceduresADAACK : Check Address ConverterADAACK should only be used for diagnostic purposes. It checks the Address Converter for a specified file(s) and ISN range. It is used in conjunction withADAICK. each Address Converter element to determine whether the Data Storage RABN is within theused portion of the Data Storage extents specified in the File Control Block. the ISN for each record in each Data Storage block within the specified ISN range to ensure thatthe Address Converter element for that ISN contains the correct Data Storage RABN.ADADCK : Check Data StorageADADCK should only be used for diagnostic purposes. It checks the Data Storage and the DataStorage Space Table (DSST) of a specific file (or files) in the database.ADADCK reads each used Data Storage block (according to the Data Storage extents in the FileControl Block) and checks whether: the block length is within the permitted range (4 block length physical block size). the sum of the lengths of all records in the Data Storage block plus 4 equals the block length. any record exists with a record length greater than the maximum compressed record length forthe file or with a length 0. any duplicate ISNs exist within one block. the associated DSST element contains the correct value. If not, the DSST must be repaired (seeREPAIR parameter).ADABAS Concepts and Facilities ADAICK : Check Index and Address ConverterADAICK should only be used for diagnostic purposes. It checks the physical structure of theAssociator. This includes validating the index based upon the descriptor value structures andthe Associator extents defined by the General Control Block (GCB) and File Control Block(FCB).ADAICK can check index and Address Converter for specific files; print/dump the contents of any Associator or Data Storage block in the database; or produce a formatted print/dump of the contents of the GCB, FCBs, and FDTs.ADAMER : ADAM EstimationThe ADAMER utility produces statistics that indicate the number of Data Storage accessesrequired to find and read a record when using an ADAM descriptor. This information is usedto determine
whether the number of accesses required to retrieve a record using an ADAM descriptor wouldbe less than the standard ADABAS accessing method; the amount of Data Storage space required to produce an optimum distribution of records basedon the randomization of the ADAM descriptor.The input data for ADAMER is a dataset containing the compressed records of a file producedby the ADACMP or ADAULD utility.The field to be used as the ADAM descriptor is specified with the ADAMDE parameter. Amultiple value field or a field contained within a periodic group may not be used. The ISNassigned to the record may be used instead of a descriptor as the basis for randomization(ADAMDE=ISN).The ADAM descriptor must contain a different value in each record, since the file cannot besuccessfully loaded with the ADAM option of the ADALOD utility if duplicate values arepresent for the ADAM descriptor. The ADAMER utility requires a descriptor field defined asunique (UQ), but does not check for unique values; checking for unique descriptor values is doneby the ADALOD utility when loading the file as an ADAM file.Utilities The BITRANGE parameter may be used to specify that a given number of bits are to betruncated from each ADAM descriptor value before the value is used as input to therandomization algorithm. This permits records containing ADAM descriptor values beginningwith the same value (for example, 40643210, 40643220, 40643344) to be loaded into the samephysical block in Data Storage. This technique can be used to optimize sequential reading ofthe file when using the ADAM descriptor to control the read sequence, or to removeinsignificant information such as a check digit.ADAREP : ReportThe ADAREP utility produces the database status report, which provides informationconcerning the current physical layout and logical contents of the database.The information provided in this report includes a database overview: the database name, number, creation date/time, file status, and current lognumber. current space resources for Associator, Data Storage, and Work: amount and locations ofcurrently used space, allocated but unused space, and alternate RABN blocks. summary and detailed file information: summary by file of ISN, extent, padding factor,used/unused Associator and Data Storage space, and file options; and detailed, optionally byfile, that includes all summary information plus MINISN/MAXISN settings, detailed spaceinformation, creation and last use date/time, Field Definition Table (FDT) contents, and generalor extended Checkpoint file information. checkpoint information: general and extended Checkpoint file information. physical structure: Associator/Data Storage RABN information including device type,VOLSER number, file number (if appropriate), and usage (AC, NI/UI, Data Storage, DSST,alternate, or unused).ADABAS Concepts and Facilities ADAVAL : Validate the DatabaseThe ADAVAL utility validates any or all files within an ADABAS database except theCheckpoint and Security files.ADAVAL compares the actual descriptor values contained in the records in Data Storage withthe corresponding values stored in the Associator to ensure that the Associator and Data Storageare synchronized, and that there are no values missing from the Associator.Before running ADAVAL, the consistency of the inverted lists should be checked with theADAICK utility.ADAPRI : Print Selected ADABAS BlocksThe ADAPRI utility prints the contents of a block (or range of blocks) contained in theAssociator, Data Storage, Work, Temp, Sort, dual command log (CLOG), dual data protectionlog (PLOG), the recovery log (RLOG), or the DELTA SAVE images (DSIM) dataset.
Using Adabas
335USING ADABASGenerally, ADABAS uses between 10% and 50% of the data processing resources (disk storage,CPU time, elapsed processing time) used by other database management systems. Since fewerhardware facilities are used, more can be accomplished with less. Very large online applicationsusing several gigabytes of data have been successfully implemented, with thousands of terminalworkstations, and with the response times and the cost of much smaller systems.Accessing a Database from ProgramsADABAS access is field-oriented: user programs access and retrieve only the fields they need.User program statements invoke ADABAS search and retrieval operations automatically.Direct Call InterfaceADABAS provides a powerful and flexible set of direct call commands for performing databaseoperations. ADABAS direct call commands provide a direct interface to the ADABAS databasewhen NATURAL or another fourth-generation database language is not being used.The commands can be categorized by function: Database query Read (Data Storage or Associator) Database modification Logical transaction processing Special commandsADABAS Concepts and Facilities 336Database Query Commands (Sx)Database query commands (S1/S4, S2, S5)) search for and return the ISNs of specified recordsor record groups according to specified search criteria. Other commands in this category (S8,S9) sort the resulting ISN lists in preparation for later operations.The ISN lists resulting from any Sx command may be saved on the ADABAS Work dataset forlater retrieval during the user session.In most cases, these commands do not actually read the database; ISNs are read directly fromthe Associator’s inverted lists. Options allow the ISN’s record to be placed in hold status toprevent it from being updated by other programs until the record is released; if desired,additional field values contained in the first ISN’s record can be read from Data Storage.Read Commands (Lx)The L1 through L6 commands are used to read actual records from Data Storage. Dependingon the specified command and its options, records are read individually in the sequence in which they are stored; in the order of an ISN list created by one of the database query commands; or in logical sequence according to a user-specified descriptor.See page 39 for more information about sequential access methods.A hold option allows the database records to be locked until released by a separate commandor at transaction end.The L9 and LF commands read information directly from the Associator inverted lists or FileDefinition Tables (FDTs), returning either the inverted list values for a specified descriptor orthe field definitions for a specified file in the database.Database Modification Commands (A1, E1, N1/N2)Database modification commands (A1, E1, and N1/N2) change, delete, or add database recordsand update the related Associator lists accordingly. ISNs can be assigned to new records eitherby the user or by ADABAS.The inverted lists and the Address Converter are automatically maintained by ADABAS. Whenthe user supplies a new value for a descriptor, either in a new record or as part of a record update,ADABAS performs all necessary maintenance of the inverted lists. When a new record is addedor a record is deleted, the Address Converter is appropriately updated. These operations arecompletely transparent to the user.Using ADABAS 337Logical Transaction Control Commands (ET/BT)An ADABAS logical transaction defines the logical start (BT) and end (ET) of the databaseoperation being performed. If the user operation or ADABAS itself terminates abnormally,these commands provide the capability for restarting a user, beginning with the lastunsuccessfully processed transaction. ET/BT commands define the transaction start and end; restore pretransaction conditions if a situation occurs that prevents successful completion of thetransaction; and read program-specified user data written during the transaction sequence.Programs that use these commands are called ET logic programs. Although not required,Software AG recommends that you use ET logic. See page 44 for more information abouttransaction logic.Special CommandsSpecial commands perform many of the “housekeeping” functions required for maintaining theADABAS database environment. Commands in this group open (OP) and close (CL) a user session (but do not control a transaction); write data protection information and checkpoints (C1, C5); and set (HI) and release (RI) record hold status.In addition, the RC command releases one or more command IDs currently assigned to a user,or deletes one or all global format IDs.The RE command reads user data previously stored in an ADABAS system file by CL or ETcommands.ADABAS Concepts and Facilities 338Complex SearchesIn many large database systems, the time required to process complex searches is oftenexcessive. ADABAS efficiently solves this problem. Many ADABAS applications are currentlyin use with up to 150 complex selection criteria created dynamically. Data is retrievedimmediately from files with more than 50 million records.Multifile SearchingIt is often necessary to perform a multifile search in order to resolve an inquiry. Multifilesearching can be accomplished using multiple search commands, physically coupled files, orsoft file-coupling. See page 21 for information about coupled files.Multiple search commands may be used in which a value retrieved in one command is used asthe search value for the next command. This process is not restricted to two files.Multi-index Searching“Fuzzy” matching (i.e., retrieving data based on similar rather than on exact search criteria)can be implemented using hyperdescriptors. Hyperdescriptors allow multiple virtual indexes,meaning that several different search index entries can be made for a single data field. See page34 for information about hyperdescriptors.Using ADABAS 339Access MethodsADABAS supports both sequential and random access methods. Different calls use differentADABAS access paths and components; the most efficient method depends on the kind ofinformation you want and the number of records you need to retrieve.Sequential AccessPhysical sequence retrieves every record in a file in the order the records are stored in DataStorage. You can limit the fields within each record for which values are to be returned. You canalso specify a “start” ISN: the sequential pass then begins at the first record physically locatedafter the record identified by the specified ISN.ADABAS bypasses the Associator and goes directly to Data Storage, reading the first data block(or the first record following a specified ISN) and continuing in consecutive sequence until thelast block is read. Physical sequence is the fastest way to process a large volume of records.UserProgram ADABASNucleusI/O BufferWork AreaInvertedListDataStorageReturns field valuesReadPhysicalL2/L5Address1 Converter423Figure 2-9: Read in Physical Sequence (L2/L5)ADABAS Concepts and Facilities 340ISN sequence retrieves records in ISN order. ADABAS uses database query commands (Sx) tobuild and sort ISN lists, which can then be read using L1/L4 commands with the GET NEXToption. When reading, ADABAS uses the Address Converter to find the RABNs of each ISNin the list and then reads and returns the records from Data Storage.UserProgram ADABASNucleusI/O BufferWork Area(Sx ISN List)Inverted ListDataStorageReturns field valuesRead by ISNSx, L1/L4AddressConverter162543Figure 2-10: Read in ISN Sequence (L1/L4)Logical sequence retrieves records by descriptor value. ADABAS finds the value(s) in theinverted list, uses the Address Converter to find the RABNs of the ISNs related to the value, andretrieves the records from Data Storage.UserProgram ADABASNucleusI/O BufferWork AreaInvertedListDataStorageReturns field valuesRead LogicalL3/L6Address1 Converter62543Figure 2-11: Read in Logical Sequence (L3/L6)Using ADABAS 341Reading in logical sequence retrieves all the records related to a single value or a range of valuesof the specified descriptor. It returns the records sorted in ascending/descending order bydescriptor value, and in ascending/descending ISN order within each value. You can specify astarting and ending value. You can also specify the field(s) for which values are returned. Readlogical is useful when you want the returned records sorted on a particular field.ADABAS provides a special read command (L9) to determine the range of values present fora descriptor, and the number of records that contain each value. Such a retrieval is called a“histogram”. The L9 command does not require any access to data records, only to the invertedlists stored in the Associator.UserProgram ADABASNucleusI/O BufferWork AreaInvertedListDataStorageReturns count of ISNs for each valueREADVALUES FORdescriptorAddressConverter1423Figure 2-12: Read Descriptor Values (L9)Random AccessADABAS uses the S1/S2/S4 commands to select records that satisfy a search criterion: thecount of the records found and a list of their ISNs is returned. The S1/S4 commands return theISNs in ascending sequence; the S2 command allows you to specify a sort sequence for thereturned ISNs.The search criterion may comprise one or more fields in a single file; fields contained in two or more physically coupled files; or search, read, and internal list matching based on the soft coupling feature.ADABAS Concepts and Facilities 342A search criterion may contain one or more fields that are not defined as descriptors. Ifnondescriptors are used, ADABAS performs read operations to determine which records toreturn to the user. If only descriptors are used within the search criterion, ADABAS resolves thequery by using the Associator inverted lists; no read operation is required.Random Access Using the ADABAS Direct Access Method (ADAM)Note:The ADAM feature is available on mainframe platforms only.The ADABAS Direct Access Method (ADAM) improves random access performance on aparticular descriptor field in a file. ADAM uses the field value to compute the relative blockaddress (RABN) for record storage. The ADAM descriptor may also be used like any otherdescriptor within a selection criterion, and for logical sequential processing as well. This optionmay be selected for any given file when the file is loaded into the database.Customer ID=17802Customer–ID=17802RandomizerADAMData Block 1Data Block 2Data Block 3Customer FileData StorageRABNFigure 2-13: ADABAS Direct Access Method (ADAM)Using ADABAS 343Using Triggers and Stored ProceduresThe ADABAS triggers and stored procedures facility is an integral part of ADABAS; however,NATURAL (see page 102) and ADABAS ONLINE SYSTEM (see page 71) are required to usethe facility.A procedure is a NATURAL subprogram that is written and tested using standard NATURALfacilities. A trigger is a procedure that is executed automatically by ADABAS when a specified set ofcriteria is met. The set of criteria is determined for each command sent to ADABAS and is basedon the target file number and optionally the command type and/or field. The command typerefers to the commands FIND, READ, STORE, UPDATE, and DELETE. The field must be inthe corresponding Format Buffer of the command. A stored procedure is executed by ADABAS, but is invoked directly by a special user call fromany of a number of applications that use it. Storing programs that are used by multiple clientsin an ADABAS file on the server reduces the amount of data traffic to and from the server.The same types of parameters are passed to the subprogram whether it is a trigger or a storedprocedure.The ADABAS facility for triggers and stored procedures allows you to implement and maintainboth types of procedures. It resides within ADABAS and provides an extension to anapplication. It can be used to implement various security and auditing features for an application; and provide a consistent, central environment where data can be verified or manipulated, eithermanually by the application or automatically by ADABAS when triggers are defined.ADABAS Concepts and Facilities 344Maintaining Database IntegrityADABAS provides facilities to ensure the logical consistency of data in a competitive updatingenvironment and when a user or ADABAS session is interrupted.Facilities are available for both online and traditional batch update. For online,transaction-oriented processing, ADABAS ensures that the database is free of incompletetransactions. For batch mode updating, ADABAS ensures restart in the event of failure byproviding checkpointing and backout/regeneration of updates.Transaction LogicADABAS data protection, recovery, and user restart are based on the concept of a “logicaltransaction”: the smallest unit of work (as defined by the user) that must be performed in itsentirety to ensure that the information contained in the database is logically consistent.A logical transaction may comprise one or more ADABAS commands that together perform thedatabase read/update required to complete a logical unit of work. A logical transaction beginswith the first command that places a record in hold status and ends when an ET (end transaction),BT (back out transaction), CL (close), or OP (open) command is issued for the same user.The OP (open) or RE (read ET data) commands can be used to retrieve user restart data storedby the ET or CL command. This data is also written to the ADABAS data protection log witheach checkpoint written by the transaction and can be read using the ADASEL utility.The ET command must be issued at the end of each logical transaction. Successful executionof an ET command ensures that all the updates performed during the transaction are physicallyapplied to the database, regardless of subsequent user or ADABAS session interruption.Updates performed during transactions for which ET commands are not successfully executedare backed out, either manually by issuing the BT command or automatically by theAutobackout routine (see page 48).Competitive UpdatingCompetitive updating is in effect when two or more users (in multiuser mode) are updating thesame ADABAS file(s). The ADABAS facilities used to ensure data integrity in a competitiveupdating environment include record hold/release, avoidance of resource deadlock, andexclusive control updating.Using ADABAS 345Record Hold and ReleaseThe ADABAS Record Hold facility ensures that a record will not be updated by more than oneuser at a time. A user can put the record in “hold” status (that is, the ISN of the record is put inthe Hold Queue) using the commands S4 (find with hold), L4/L5/L6 (read with hold), A1/E1(update/delete) with the hold option specified, N1/N2 (add record with hold), or HI (holdrecord).If a record requested for hold status is already being held by another user or utility, the userissuing the record hold command is put in “wait” status until the record becomes available, atwhich time ADABAS reactivates the command. You may request that a response code bereturned if you do not want to be put in “wait” status.Records in “hold” status can be accessed (found and read) by users who do not seek to hold therecord.Records in “hold” status are released by issuing the ET or CL commands. Options are availablewith ET and BT commands to release records selectively. The CL command releases all recordsin “hold” status for the issuing user.Avoiding Resource DeadlockResource deadlock occurs when two users are each waiting for a record held by the other user.ADABAS protects against such a user deadlock situation by detecting the potential deadlockand returning a response code to the second user after putting the first user in “wait” status.Exclusive Control UpdatingUsers who use logical transaction commands (ET/BT) are called ET logic users.Alternatively, a user can request exclusive control of one or more ADABAS files for the durationof the user session. If the file or files for which exclusive control is requested are not alreadyopened for update by another user or utility, exclusive control is granted and the user becomesan exclusive update (EXU) user. ADABAS treats EXU users as non-ET logic users.ADABAS does not place an ISN in “hold” status for EXU users. ADABAS disables hold logicprocessing for files being updated under exclusive file control.Instead of using ET commands, EXU users can request checkpoints to act as reference points;for example, updates applied after a checkpoint can be removed.ADABAS Concepts and Facilities 346Timeout ControlsADABAS times out transactions that exceed a specified limit; and users who are inactive for a specified amount of time.Transaction Time LimitADABAS provides a transaction duration time limit for ET logic users. The time limit is set withthe ADARUN TT parameter; an override for a specific user can be set using the OP command.If a transaction exceeds the prescribed limit, ADABAS generates a BT (back out transaction)command to remove all the updates performed during the transaction and release all heldrecords. The user can then either repeat the backed out transaction from the beginning or beginanother transaction.Non-Activity Time LimitAll users are subject to a non-activity time limit; different limits can be set for different usertypes and for specific users within each user type.If a user exceeds the prescribed limit and the user is an ET logic user, ADABAS backs out the current transaction, releases all held records andcommand IDs, and deletes the user’s file list. an EXU user, ADABAS deletes the user’s file list and releases all command IDs. The user loseshis EXU user status and becomes an access-only user. an access-only user, ADABAS deletes the user’s file list.Using ADABAS 347Backout, Recovery, and RestartBackout, recovery, and restart may be required when a user or ADABAS session is interrupteddue to a timeout (see page 46); a program error when ADABAS determines that a transactioncannot be completed successfully; an ADABAS, hardware, or operating system failure; or apower failure.A “user session” is a sequence of ADABAS calls optionally starting with an open (OP)command and ending with a close (CL) command. A “user” is either a batch mode program ora person using a terminal. The uniqueness of each user is assured by the user ID, a machine, anaddress space, and a terminal ID.An “ADABAS session” starts when ADABAS is activated and continues until ADABAS isterminated. During this time, the ADABAS nucleus creates a sequence of protection entries inexact historical sequence reflecting all modifications made in the database. The sequence ofprotection entries is written to the Work dataset (part 1) and to a protection log in blocks. Eachblock contains the nucleus session number, a unique block number, and a time stamp.User Program ErrorA user program that is in the middle of a transaction can detect that the transaction cannot becompleted successfully. In this case, a BT (back out transaction) command is used to removeor “back out” the incomplete transaction.If a user program error causes logical damage to the database, it may be necessary to recoverthe affected files using the ADASAV and ADARES utilities.ADABAS, Hardware, or Operating System FailureAfter any failure that causes the ADABAS nucleus to terminate abnormally, an automaticprocedure is executed when ADABAS is reactivated to bring the database to a physically andlogically valid status. All partially executed update commands are reset; all incompletetransactions are backed out.The automatic procedure comprises three steps: repair the database, Autorestart, andAutobackout. Database repair modifies the database to the status it would have had if a buffer flush had justbeen completed at the time of failure. That is, all blocks in the database are at a status thatenables the nucleus to perform normally.ADABAS Concepts and Facilities 348 Autorestart backs out updates of single update commands that were partially executed when thesystem failed. It resolves internal inconsistencies in the database and ensures physical integrity. Autobackout, which is performed only for ET logic users, backs out updates of user transactionsthat were partially executed when the system failed. ADABAS performs an internal BT (backout transaction) followed by Autorestart, and then informs the user that the last transaction hasbeen backed out.The Autobackout routine is executed at the end of an ET session that was terminated with HALT.It is also executed automatically at the beginning of the next ADABAS session to remove anyupdates performed within transactions that did not complete successfully.After Autobackout execution, the database contains updates only from logically completetransactions.Note:ET users can manually back out an incomplete transaction at any time by issuing the BT (backout transaction) command. See page 44.If an ADABAS, hardware, or operating system failure results in physical damage to thedatabase, it may be necessary to recreate the database using the ADASAV and ADARESutilities.Power FailureDepending on the hardware, a power failure during an I/O operation may damage the ADABASblocks that were being processed. This damage cannot be detected during Autorestart andtherefore can result in problems later such as unexpected response codes or lost databaseupdates.If the ADARUN IGNDIB=YES parameter is set, the Autorestart routine checks whether a bufferflush was active when the session interruption occurred. If a buffer flush was in process, theAutorestart shuts down and ADABAS alerts the user to the potential problem and includes a listof the files being updated when the buffer flush was in process. The DBA must then determinewhether a power failure occurred.If the cause of a session interruption is a power failure, Software AG strongly recommends recovering the affected files using theADASAV and ADARES utilities. is definitely not a power failure and the integrity of the information on the output hardware canbe guaranteed, the database can be reactivated immediately. Database recovery is not necessary.