Flex Inquiry Software Changes

                       MZIP Commands

                             +-- + --+
         +-----------<-------+-- , --+-<----+

These commands access a fast compression function, using the LZ77 algorithm. MZIP compresses files and their attributes in a format suited to the MCP system. GZIP compresses the data in byte stream files (and loses the file attributes), but the resulting file can be expanded using gunzip on almost any other system. UNZIP (or UZ) expands a file compressed by MZIP or GZIP. But, as yet (07/2010), it cannot expand most files created by gzip on other systems.

To get the output file name, add /MZ for MZIP, ".gz" for GZIP, and /UZ for UZIP. However, if the file already ends in ".gz" or /MZ, MZIP and GZIP will not process that file, while UNZIP will strip off the ending. If the output file title already belongs to a resident file, the action is skipped, unless the AutoRm option is set, when the resident file is replaced.

These commands can also be Actions in :ACT commands.

MZIP and GZIP have 2 specific options to control how they work. ZipMore or ZM controls how much effort goes into the compression in MZip and GZip. If it is OFF, the compression is done with as little CPU time as possible. If ON, about 50% more CPU is used to improve the compression, typically ending with 10% smaller compressed file size. ZM ON files normally UNZIP more quickly than ZM OFF ones.

ZM can also take an integer value, though this is not saved between sessions. The only current assigned meaning is for ZM 2 in MZip. This value will often add a further 50% CPU time over ZM 1, and drop the final compressed file size a further 7%.

All these % figures are data dependent and each file will behave somewhat differently. However, a file with ZM 2 compression will normally need about double the time to UNZIP, vs ZM 1.

The MemSz option is essentially a debug option to override the memory size used by MZIP and GZIP.

It is normally (Default), in which case MZIP/GZIP look at the system to determine how much memory to use, but can be set to (Big), (Med), and (Small) corresponding to 16MW, 1 MW, and 64kW respectively. Add at least 25% if ZipMore in set. MemSz is not kept between sessions.

The HEXLATE option has been re-named the XLATE option. It acts as before in LIST and WRITE, but in MZIP and GZIP, it does an EBCDIC to ASCII translation, and in UNZIP, an ASCII to EBCDIC one.

The output of the OPtion command has been re-arranged. Now the values of the NR, R, and REO options, are summarised in a new portmanteau option, SELRES.

The option NOZIP has been deimplemented. It had a small effect on certain jobs in the INSTALL module. These now act as if NOZIP is set. It was used in BACKUP to mean SYNTAX. Using :SYNTAX now has the same function. In LOAD, it was used as a debugging tool to save the LOADJOB generated. This can now be done with :SAVE.

The APPEND colon option now works correctly after being slightly broken by the change described by DNote 540.20.

This cleans up some inconsistencies in option handling. Flex will now allow options that have no meaning with a command, but will still restrict those that are misleading or harmful. SEARCH, as on the options report, can now be set explicitly. The DEBUG options, can no longer be saved between sessions.

The SECURE commmand can once again be used with lower case.

This patch reduces the lines in FLEX by about 500.

Adjust include from include/meta

Internal Change.

As far as we are aware flex will only be run on terminal emulators. The options used with old 'hard wired' terminals have been removed.

Flex is now more rigorous in checking for spurious input between a valid command and the : options.

Flex Help has been updated to include the WHERE clause.

The FInd,Append and Byte options now work with the FIND command.

The handling and reporting of errors for 'IN' files searches has been improved to provide more information about the failure. In particular, failed Supervisor PD searches did not always get an error returned in a timely manner.

The handling and reporting of errors for 'IN' files searches has been improved to provide more information about the failure. In particular, failed Supervisor PD searches did not always get an error returned in a timely manner.

FLEX wildcard searching did not work correctly for files whose top level directory or name was a quoted string e.g. (TEST)"example", (TEST)"example"/FILE.

A DO <file name> command is not permitted with a DO File. This restriction has been partly relaxed by a new feature allowing DO <opal string> within a DO file. The DO <opal string> is only permitted within a DO file. The expected use is to generate flex commands within a REPORT and then be able to execute them.

Directory searching using wildcard usercode patterns now works again correctly after being inadvertently disabled by FLEX 540.10.

The change described by DNote 540.10 caused some wild-carded FILES requests to fail with a '#Bad directory..' error. This problem has been fixed.

Previously, FLEX did not correctly handle all wild-card file patterns with quoted strings, discarding files that should have matched. Also, any patterns with a single quote or '.' would give a command error. FLEX now permits the use of these characters in patterns.

For example, the following FILES command will now work correctly:

  FILES ="DEV.HTM"=               -  Now finds matching files
  FILES ="DEV.=                   -  No command error now
  FILES =.=                       -  No command error now

Version 540.06 caused status line messages to be suppressed. Status line messages have been restored.

Version 540.06 introduced a bug which could cause flex to fault with an invalid operator error when run in Batch or called from Supervisor.

This problem has been corrected.

Fixes unsaved FLEX/DP/WHERE

The LISTVL HTML report would create a corrupted HTML file if a volume which was not in the volume library was encountered. This has been corrected.

The WHERE modifier can be used to invoke an 'inline selection. The text following WHERE should be valid SELECT code. A WHERE clause will be used in preference to an active SELECT. The where clause can be used after a directory list and before any colon options. Since currently the SECURE command does not accept a directory list the WHERE clause cannot be used with SECURE. This will be addressed in a subsequent release.

Ex FILES TEST WHERE SYMBOLFILE FILES = WHERE AGE=0:Flat REMOVE = WHERE CODEFILE and AGE >365 :FL

FLEX will no longer use the SAVEBACKUPFILE file attribute since it will be deimplemented in MCP 54.1/13.0. FLEX now uses the recommended replacement SAVEPRINTFILE.

INPUTPARAM Returns the text of the most recent command.

Ex REP HEAD INPUTPARAM Followed by FILES =BoB= ON DEV:FL Would return FILES =BoB= ON DEV:FL followed by the list of files

Previously, FLEX would fault with a SEG ARRAY error at 90444600 when processing a FILES command which had a long, incorrect file specification. This was usually caused by accidentally transmitting a large amount of screen data after the FILES command.

When called by Supervisor via the PD context, errors which would normally be reported by a Display and could be easily missed by a Supervisor user, are now correctly passed to Supervisor.

Ex.

/(pd:title) (*)= IN *METALOGIC/MAGUS

FLEX Error: Invalid Container or Image

previously the fooling would have been the result

/(pd:title) (*)= IN *METALOGIC/MAGUS

FLEX Error: #No files in directory (*)=

The FAMILY statement is now permitted again in batch DO files. Previously, the error '##Option PRINT incompatible with FAMILY statement' would have been returned causing the batch file to fail.

The LISTVL command has been improved. The original number of files copied to a volume are now reported in square brackets after the number of backup references.

A new modifier 'HTML' can now be used with a normal LISTVL or a LISTVL <familyname>.

Ex LISTVL HTML LISTVL DEV HTML

If the HTML modifier is used the report is generated as an HTML table which can be sorted by clicking on a column heading. Only volumes in use as backup are reported.

If the Metalogic mail library is available and the UserData entry for the user running flex has an email specification , the report will be emailed as an attachment.

Load x At <hostname> will no longer sometimes get the <kind> wrong for a Load At without a unit number.

Previously an expression using a double quote inside single quotes would give a syntax error in a rep foot report. This has been corrected.

Using CopyWrite, Flex has an undocumented extension to FILES for PUs to search on other hosts. The syntax requires "AT <hostname>" be placed after the normal title/pattern specifier. Most of the problems with the implementation are now resolved and it has been expanded beyond FILES and USAGE to allow LOAD, e.g.

F = CD 50 AT HISHOST USAGE (FRED)= AT SOURCES, (JIM)= ON BKMT(MT) AT TAPESYS:BR LOAD OLD/BONES IN (SHAFT)CDIMAGE ON IMAGEPK AT DRHOST, NEWR/=

If the CopyWrite option is used on a LOAD AT, a special form of COPY is used so that if there is a conflict, the file is only overwritten if the incoming one has a later timestamp. This makes it easy to keep a host up-to-date with file changes.

Note that ISO/JOLIET CDs, and non-CopyWrite containers, are currently not accessible using "AT", and that wildcards are disallowed on the family name.

DESTROY is now a preferred synonym for ZAP.

The RULES MANAGER command has been improved to allow better control of the setting of primary and alternate Rules Manager usercodes and accesscodes. Changing or setting a Rules Manager requires the usercode and accesscode to be valid in the USERDATAFILE and passwords must be supplied if applicable. All Rules Manager usercodes must be marked as privileged (PU).

The command 'RULES MANAGER ?' will show current Rules Manager details if run by a FLEX usercode that is PU or SECADMIN. A typical response might be:

    RULES MANAGER ?

    -- Rules Management details --
    Master Usercode:      FLEX
    Master Accesscode:    RULES

    Alternate Usercode:   Not Set
    Alternate Accesscode: Not Set
   #

Further details and syntax can now be seen using the HELP RULES MANAGER command in FLEX Inquiry.

Fixes a problem using BL/PACK tapes.

The FAST option now applies to searches on MT/CD or IN containers. This can be much faster than the non-FAST call, but it only looks at the Directory, not at the headers, which means only the title related attributes are available (and LicenseKey).

Note that the tape directory on a tape that has been Appended only shows the files of the first COPY.

Using :FAST on a container with CopyWrite allows access to more types of directory, including PKZIP and TAR containers.

Some directory list anomalies have been resolved.

In certain rare circumstances the MYSELF(PageSize) and MYSELF(Width) attributes would not work in a Report Foot. This has been corrected.

The maximum size of a parameter passed to a Flex DO file has been increased from 255 to 65535.

The Flex START command may now be used in an offline DO file. Parameters may now be supplied directly to the Start command.

A new variant on the Report command has been introduced: REPORT DO.

It is defined in the same way as the other reports but the code is executed immediately. Its main use is expected to be in DO files to set up variable to be used in a selection expression but can also be to interrogate system or PD attributes. Although flex file attributes may be used they will always return nul values since the code is not run against any file.

EX.

REP DO    
BEGIN    
   IF #TYPE# = "FULL" THEN
      #DT :=0
   ELSE    
      #DT:=#LASTDT.PERM ;
   #LastDt.Perm:=TODAY
END

SELECT ALTERDAY > #TS

REP DO "Last backup",DateToText(#LastDt.Perm,ddmmyy)   

LOAD can now load from wrapped containers, and, with CopyWrite present, it can also load from non-Library/Maintenance CDs or CD Image files.

e.g. LOAD (*)= IN WRAPPED/CONT

     LOAD =".txt" ON ISO_CD@#### (CD)
     LOAD (SS)S/= IN *PIFLEXBACKUP090122/DAILY
     LOAD (SS=)S= IN (XX)ANY/CONTAINER ON ANYCD(CD)

If a LOAD is not explicitly or implicitly recovering non-resident files, it will now ask whether resident files with the same name as files being loaded should be overwritten or not:

  #Loaded Files may OVERWRITE Resident Ones
  #Enter 'OVER' to OVERWRITE, anything else to preserve

If the response is OVER, any name conflicts will result in the resident file being removed. The response must be 'OVER' in a LOAD:FLAT without a 'TO' clause unless an ALT FLAT is active.

Because of this new feature, LOAD is now significantly faster for many large recoveries.

Until now, FLEX used CopyWrite only if there was no alternative. The COPYWrite option will now be applied to FILES and LOAD commands. This will allow access to BL/PACK tapes, containers on ISO CDs, etc.

An error in the FIND command has been corrected.

Flex will now return the volume information for MT, CD, or LIBMAINTDIR searches in the first Backup Reference. These can be seen using the :CAT report.

This also allows use of the LOAD command, e.g.

LOAD (BRI)B/T= CD 50 LOAD (BRI)OBJECT/= ON FLEX= (MT) LOAD (=)= IN *LIBMAINTDIR/USERSTUFF/20090122/G00123

(Change in OPAL/BINDLIB) For most usercode directories an = after a stand alone usercode is assumed.

Ex. FILES (ABC) is treated as FILES (ABC)= (*)

    FILES (*) is trated as FILES (*)=

If a usercode pattern other than (*) or (=) was used this was not the case.

A standalone usercode pattern will now assume an = after the usercode.

Ex.

  FILES (=B) will be treated as (=B)=

The Usage command will now allow :Resident and :REO options. The :-Brief report for :REO will not report Generation information. Also generations after the highest one found are not reported.

The full Usage report has also been enhanced to show the title of the largest file and to give totals of space wastage. The 3 categories of wastage are: "Mis-aligned Blocks" - caused by blocksize not being a

                                    multiple of 180 bytes.
             "End Block" - caused when the last block is not full.
             "Uncrunched" - the space released if all were crunched.

Also the :ATT and LF (but not LF:ATT), will now indicate mis-aligned block wastage by "+nn" after the "BlkSz".

LFILES also report total wastage.

Flex Inquiry was used for all recovery situations, except a lost family member when a flat directory was still available. That case used FamilyManager, which entailed multiple inconveniences. By allowing Load:Flat, the capability is now in Flex Inquiry.

Example:

To recover all recoverable files on a pack where index 2 has crashed:

Load (*)= on FAM:FL #2

Unlike other loads, a :Flat Load without a 'TO <dest>' part, will overwrite the resident file, so a confirmation is asked for.

Note that a Load:Flat or a Load:Fast will normally be slower than other Loads because Flex has to find backup info for each file one by one.

The FIND command was never finished and differed markedly from its documentation. It now works as one would expect, and in addition has new capabilities, as in the Find option, e.g. Hex, Whole Word, and case-independent searches.

The FIND Option can now be case insensitive (with single quotes), will respect the text field of symbol files (if whole word) and uses about 20% less CPU for the same search.

The :FIND option filters the files found depending on the presence of the target in the data content of the file.

Syntax:

FIND --+-------+--+--------+---+-- " -- <target> -- " --+--------|

       +-- ^ --|  +-- LIT -+   +-- ' -- <target> -- ' --+
                  +-- 8 ---+
                  +-- 7 ---+
                  +-- 48 --+

If "^" is present, the file returned do not have the target, otherwise the ones with the target are returned. "LIT" and "8" are equivalent, and indicate a literal FIND. "7" means search for the literal target in ASCII. "48" means search for the literal target in Hex. Note that the search is still a byte search, so there must be an even number of hexades. If no specifier is given, the Find is like a CANDE token FIND, with the important difference that in CANDE, the file is divided into records, whereas here, the file is considered to be a long string of characters between the beginning of the first segment to the end of the last one. For example if an identifier is at the beginning of a record and the end of the last record is an alphanumeric character, a token find will not signal it, whereas CANDE would.

The file search used must be on a disk and return a header. That excludes :FAST and :LIN, and searches in containers, tapes, and CDs. Use the CopyWrite FIND construct for these searches. Also no MAINT ALT FLAT can be used in a :FLAT search. The search used involves low-level IO and so is shockingly fast and does not update the file's timestamps. The IO time is charged to the directory search and not the FLEX task.

Version 530.19 broke the :byte and :redir options. The :FIle option would overwrite existing files even if the AUTORM option was not set.

The original functionality has been restored.

Fix confusing lack of errors in BACKUP.

Remove some over-large Defines. Approx 3% reduction in source, object, and working-set sizes, and compile times.

The internal queues were very restrictive in core usage. They will now use more. Large BACKUP commands will now run many times faster as a result.

Some option handling has been rationalised, saving about 100 cards.

A list of directories passed to a Flex command may now be separated by '+' as well as ','.

This makes it possible to pass a list of directories as a parameter to a Flex do file.

Ex. DO DO/FILECOPY(A/=+B/=,JOBID,TRue)

The BACK command now uses a direct call on Library/Maintenance and avoids WFL, which allows a much larger number of files (100x) and and reduces overheads and source lines (-20).

A new option, CopyWrite, forces a BACK command to use CopyWrite. The Compare default is now ignored.

The OP screen has been re-arranged.

The change in 530.15 introduced a bug when processing BEGIN END blocks in REPORTs REPORT FOOTs and REPORT HEADs. This has been corrected.

Version 530.08 introduced a bug when parsing a REPORT, REPORT HEAD, REPORT FOOT or SELECT command from a DO file. If the command was indented by more than one character and the following expression was on the same line then it would always fail syntax.

This problem has been corrected.

Previously, the LOAD of an archived file from a Disk Farm that did not have an associated LibMaintDir file would not have succeeded because FLEX did not use the hostname from the Archive record to identify the network location of the Farm. This problem has been fixed.

The required mechanism to re-enter a RULES SAVE file of FLEX rules is to use the DO command. However a DO of such a file would fail because of recent changes in the handling of general DO files that now do not need the continuation character '\'. This problem caused RULES INLINE commands to be incorrectly handled and is now resolved.

The DEFAULT TAPE command will now accept tape density mnemonic values greater than 20 (current MCP limit is 34).

Setting the Archive option using the OP command will now correctly use the archive instead of the catalog for subsequent Files commands.

Setting options which are mutually exclusive to other options will now ensure the conflicting options are reset.

Ex. NR with REO and RES. FAST with FLAT and LINEAR

The screen shown in response to the OP command has be redesigned in an attempt to make it clearer. Options which cannot be set using the OP command but can only be used after a : in a command are listed under Colon Options.

The NEXT command which allows stepping backwards and forwards through multiple pages of a Files command now works again. It is only valid if none of the options :FLAT,LIN or FAST are specified.

A command entered while in the middle of a multi page response is now correctly processed.

A bug where a multi page HELP ATT would not break has been fixed.

An MCP bug which could cause Flex to fault when directing output to a file on early versions of MCP 51 and 5.2 has been detoured.

A new output option REDIRECT has been added which allows output from a Files command to be written to a PC server.

Ex.

 FILES =:FL,Redirect:MYSERVER/PRIMARY/METALOGIC/"FILES.TXT"
 Or
 FILES=:FL,Redirect:"10.0.0.4"/PRIMARY/METALOGIC/"FILES.TXT"

The usercode and password used to access a server is controlled by the MAKECREDENTIALS Utility. It must be run from the usercode to be used to run Flex.

The MAKECREDENTIALS utility provides a method of encrypting credentials, eliminating the need to specify explicit credentials as part of the IOHSTRING attribute, and creates a credentials file for each system for which the user has remote access. The MAKECREDENTIALS utility can also be used to create a default credentials file.

MAKECREDENTIALS is a simple command mode utility that expects the following login credentials for a network host:

<host>
<username>
<password>
<user domain> (optional)

These four pieces of information correspond to the IOHSTRING keywords as follows:

SERVER = <host>
CREDENTIALS = <username>/ <password>
USERDOMAIN = <user domain>

Examples
The following example shows how the MAKECREDENTIALS utility is run from the CANDE terminal emulator for a host of MYSERVER (10.0.0.10), a username of OPS, and a password of frog:

U *SYSTEM/NXSERVICES/MAKECREDENTIALS MYSERVER OPS"frog"

or if the IP address is to be used:

U *SYSTEM/NXSERVICES/MAKECREDENTIALS 10.0.0.10 OPS "frog"

Note that if the password on the PC is not all uppercase then the password must be quoted.

Using a prefix of *UNC/ on an output filename has the same effect as Redirect.

So
FILES =:FL,Redirect:MYSERVER/PRIMARY/METALOGIC/"FILES.TXT"

is equivalent to

FILES =:FL,FILE *UNC/MYSERVER/PRIMARY/METALOGIC/"FILES.TXT"

If any of the directories specified in the file name do not exist, Flex will attempt to create them. The ability to create them will depend on the privileges of the user code specified in the Credentials file.

A word of warning. A credentials file created under one usercode will not work if copied to another usercode.

A new option AutoRM has been added for use with the Files,Byte,Append and Redirect options. Setting AutoRM allow a file specified in a Files option to be overwritten, whereas normal operation will return an error if the specified file already exists.

Ex.

FILES=:FL,Redirect:"10.0.0.4"/PRIMARY/METALOGIC/"FILES.TXT",AutoRM

or

FILES=:FL,AutoRM,File FLEX/OUTPUT

A change introduced in Flex 510.31 caused only the first report to be listed in the STD command. The full list is now reported.

Files in the directory METALOGIC/FLEX/DEFAULTREPORTS under the usecode used to run Flex are now also listed correctly.

In Report, Head, Or Foot code, all text not inside " " characters were being upcased. This could cause problems for strings delimited by ' instead of "".

Begin is now allowed after FOOT or HEAD. Previously it was only allowed after REP FOOT or REP HEAD.

REP HEAD(TITLE,NOT "/") is now correctly parsed.

The changes described in FLEX DNote 530.03 caused several problems with the handling of the optional '\' line terminator in Flex DO files. If a '\' appeared in line character position 80 or did not have a space in front of the terminator, compilation of that REPORT or SELECT would fail. These problems have now been fixed.

Several issues with the execution of DO files from remote sessions have been addressed. FLEX will now not wait at a page break when the SETL option is set and long REPORTs or SELECTS are being compiled; FLEX will also better page screen output when a FILES command is executed from a DO file; previously incorrect messages would have been displayed on the screen and paging would be erratic. This behaviour has been improved.

The maximum size of an individual REPORT or SELECT compiled from a DO file has been significantly increased from the previous limit of 8000 characters.

This change supports the new DFHIMAGE attribute for both FLEX and SUPERVISOR, as described in OPAL DNote 530.19.

The usage of the WFL function inside a REPORT BEGIN..END block has now been restricted when used from an interactive session. If a WFL function is triggered by a FILES command then, once the FILES request is complete, the REPORT will be disabled to prevent accidental invocation in further FILES commands in the same session. The REPORT can be easily re-enabled by using the REP + command.

This change does not affect the usage of WFL in REPORT HEAD or REPORT HEAD; normal non-remote batch DO files are also unaffected.

Printed lines longer than 132 characters generated by any REPORT statement used to be truncated. This behaviour has been changed; FLEX will now automatically split printed REPORT strings into 132-character chunks.

FLEX DO files will now automatically abort if syntax errors are detected in a SELECT or REPORT; previously FLEX would unconditionally continue to process the DO, despite the errors, with indeterminate results.

Further, the syntax error reporting for FLEX DO file compilation of SELECT or REPORT statements has been improved. The OPAL compiler will now write the failing line of source followed by the syntax error to the screen or print file. This is best shown by the example DO file below:

  00000100 REP BEGIN
  00000200        SHOW(TESTING); ILLEGAL;
  00000300        $A:="A";
  00000400        #Z:=214; ZZ;
  00000500     END;
  00000700 FILE ABC:PRINT
  00000800 QUIT

This DO will generate several syntax errors when compiled; on older FLEX versions, executing this DO will produce the following:

  #Compiling REPORT
              *Primary expected - TESTING
          *END expected - Z
           *Unrecognised construct - :=
  #three syntax errors found in REPORT compilation
  #
  #Scanning directory ABC

  ##Found 0 files [from 0 files]
  #Directory: ABC ON DEV
  #
  #ET=0.4 PT=0.1 IO=0.0

The syntax errors may not be easy to align with source lines in the DO file and, despite errors compiling the REPORT, the DO continues.

The above behaviour has now changed when compiled with the latest FLEX:

  #Compiling REPORT
         SHOW(TESTING);
              *Primary expected - TESTING
         #Z:=214; ZZ;
          *END expected - Z
           *Unrecognised construct - :=
  #three syntax errors found in REPORT compilation
  #
  #DO aborted by error at (00000700)

Source lines are now displayed and the DO is aborted because of syntax.

The \ character is optionally used to denote line termination in a FLEX DO file. However, using this character inside quoted strings within a DO script is valid, FLEX would incorrectly assume that the line was terminating inside the string causing syntax errors. This problem has been fixed.

In some cases, the Family Substitution was not being removed in a Usage command, causing puzzling results when a family pattern was used.

Object/Flex will again not be TADS capable.

Version 520.24 fixed a problem when scanning multiple directories when one or more directories had a Family but no usercode specified. Unfortunately it caused problems when scanning multiple directories with Usercodes and Families specified.

An unusual problem existed when scanning two directories where the usercode of the second directory matched the first level of the first directory. Ex FILES BOB/=,(BOB)DIR/= The second directory would be ignored.

The optimisation of multiple directories has been improved.

FILES A/= on DEV, B/= ON DISK,A/= ON DEV will now correctly scan DEV once. Previously a change of family in the middle of the list would break the optimisation causing the files returned from DEV to be duplicated.

Use new INCLUDE/META

Flex will no longer do family substitution if an explicit ON part is used with an IN qualifier or a MAINT ALT file.

For Example:

FAM DISK = DEV ONLY F (*)= IN WRAPFIL ON DISK

will no longer search in WRAPFIL ON DEV.

Fixes an error message for searches not compatible with :ACT

The FIND command will no longer change the UseTime of the files it searches.

If Accounting ODT command is set to other than the default value of Identified, and no other setting overrides it, Flex Inquiry will now create FILEOPEN and FILECLOSE Log records.

Flex will perform any search from a Supervisor PD request, irrespective of what attributes are requested.

If a FILES command used multiple directories with one or more 'ON <family>' specifications in a list, then a FLEX :FLAT run would not return the correct set of matching files. For example:

   FILE ABC/=, DEF/= ON DEV:FL

Would not return any files from the directories ABC or DEF.

   FILE GHI/= ON DEV, JKL/=:FL

Would only return files from the directory JKL. In both these cases, it has been observed that FLEX might randomly return one or more files if the directory was large enough. The :LIN and :FA modifiers were also affected by this problem.

This problem is now fixed; it should be noted that a non-FLAT request would always be correct.

This change support the new attribute LastDIr as described in Opal/GSTableGen note 520.52.

A files command on a directory which does not exist returns a message like '#No files in directory NODIR ON DEV'. If the output was directed to a Job file this message would cause a syntax error in the created job file. For output to file such messages are now prefixed by % which prevents the syntax error.

Example DO file using LastDir

WFL REMOVE WFL/FLEX/REMOVE

SELECT DAYS(TODAY,ACCESSDAY) > 365

REPORT HEAD "BEGIN JOB WFL/FLEX/REMOVE;",/,

   "REMOVE ",#Cnt.Store(0),#Segs.Store(0)

REPORT IF #Cnt.Sum(1) = 1 THEN " " ELSE ",",FILENAME,

   #Segs.Store(#Segs+Segments)

REPORT FOOT If Not LastDir Then Empty Else

   If #Cnt=0 Then
      #("BEGIN JOB WFL/FLEX/NOREMOVE;",/,
        'DISPLAY "No files selected to remove";',/,
        "END JOB")
   Else
   #(';DISPLAY "',#Cnt," files ",#Segs,' segments removed;"',/,
     "END JOB")

FILES TIDY/=,OBJECT/TIDY/= :FILE WFL/FLEX/REMOVE WFL STARTJOB WFL/FLEX/REMOVE

This change is to take advantage of the updated PD Object format.

:FLAT searches from Supervisor will now be sorted by default.

OBJECT/FLEX will no longer be TADS-capable, halving the code file size.

A Supervisor PD context OPAL with a list, which includes a file/dir that is not present, will no longer stop processing at that point.

The change applied by FLEX DNote 520.09 caused the PDEF destination check performed during a LISTVL run to be inadvertently removed. This caused all LISTVL reports ot be poorly formatted with report headers appearing every two lines. This behaviour has been corrected.

When using the SECURE command on a file or directory to change the files SECURITYTYPE to GUARDED, FLEX would fault with an INVALID INDEX @ 17455450 if the selected file was already GUARDED. This problem, which is now fixed, did not affect any other aspects of the SECURE command.

The changes decribed in DNotes 520.01 and 520.02 did not correctly force script termination after a SELECT fault. This oversight has been corrected.

Avoid an extra message if a PD context call hits a Supervisor limit.

Fix case where Select expression was not reloaded.

PD Context now allows AT HostName syntax.

Fixes a bug in PD context calls with FAMILYNAME and :FLAT

The change applied in SUPERVISOR DNote 520.99 for limited PD DO or DISPLAY requests did not fix the case for GETSTATUS file searches using a wild card pattern. Although the request worked correctly, FLEX would still generate many extraneous "Send_Ary Exceeded Do/Eval Limit" messages. This behaviour has been corrected.

For standalone SUPERVISOR PD DO requests that use a DO limit modifier, SUPERVISOR would not stop processing the DO when the limit was reached if the FLEX search type was :FLAT. This problem caused FLEX to generate multiple extraneous 'Bad Send_Ary Call' display messages only terminating when the list of files in the selected directory was exhausted. This problem is now fixed.

The DEFAULTS command screen has been updated to remove a superfluous question regarding lower case printing and to recognise new densities (such as FMTLTO3) for FLEX tape creation.

FLEX can now handle DO files without the need for the '\' continuation character for grouping multi-line REPORT or SELECT scripts. FLEX enables this feature by searching for command words at the beginning of each new line. FLEX will continue to handle existing DO files that use '\' terminators but may generally be regarded as noise.

Some ambiguities where FLEX keywords collide with OPAL functions and attributes have been addressed but other collisions are possible. In such cases, using the '\' terminator on a previous line in the DO file will always force FLEX to regard the following line to be part of the SELECT or REPORT.

Correction of problem caused by the PD context patch.

FLEX Inquiry will now act as a server for a PD context OPAL running in Supervisor. This allows Supervisor to access the full search abilities of FLEX.

300 lines of FLEX source, dedicated to handling SPO files and disabled since 1985, has been deleted. The SPO option is no longer recognised.

FLEX will now use the improved FILEKIND handling. This fixes many small problems with inappropriate handling of certain FILEKINDs and will make the handling of them more automatic in future. The change reduces start up time and core usage (by 300 words).

REMOVE can now be used with the :FLAT option.

USAGE will no longer family substitute if the target family is used explicitly. It now follows current practice for other commands.

Commands with intrinsic reports and no SELECT will save a little time by no longer inserting TERM information in the PD object.

With similarity to SUPERVISOR printing, FLEX will now automatically disable the printing of headings (and footers) from reports that have a PrintS destination that starts with 'MAIL'. This behaviour applies to LISTVL, LFILES, FILES and also OPAL reports. The change allows much tidier prints to be viewed by the email recipient.

In addition, a variety of minor report and formatting errors have addressed in the standard LISTVBL report.

Previously, if a disabled user RULE OP was saved into the symbolic file created by a RULES SAVE ALL command, a subsequent DO of the saved file would have faulted processing the disabled RULE OP. This problem has been fixed; all disabled user RULES OP settings are now preceded by an initial RULES OP ENABLE command which establishes the rules block and permits the disable to be applied.

Also, Flex rules compiled into the rules base using a DO command can generate a lot of rapidly scrolling screen output, making error resolution very difficult since no other diagnostics are produced. This is particularly true if the DO file had multiple rules such as that generated by a RULES SAVE ALL.

Now, when a DO file is executed that has RULES commands included, FLEX will automatically write all RULE action and compilation messages into a local print file for subsequent printing.

The RULES OP modifier LOCASE has now been deimplemented by FLEX change 520.03 but a RULES SAVE command was still including the options in decompiled RULES files. This caused such RULES OP commands to subsequently fail when a saved file was entered by a DO. This behaviour has been corrected; LOCASE will now be omitted from any future RULES SAVE files.

In addition, any RULES OP settings that use MANAGEREMAIL or USEREMAIL reporting will now be included in the reporting specification for RULES SAVE files. Previously, these settings were omitted.

Version 150.39 introduced a bug where a Remove command from within a files :act command would fail, if no family had been specified. If your default family was DEV then for FILES = ON DEV:ACT removes would be fine but would fail for FILES =:ACT. This problem has been corrected

Report the current Opal version and The Opal version used to compile the Rule in a RULE OP command.

FamilyManager reports can now be directed to email. For sites without Supervisor,the Metalogic Mail Library can be configured via Marc if the Mail directive is set up.

  The Marc command DIRECTIVE MAIL = *METALOGIC/MAILLIB/DIRECTIVE sets
  up the Directive.

  Once set up MAIL becomes a valid Marc command.

  Ex. MAIL STATUS
      MAIL HELP

MANAGEREMAIL  - Sends User reports in the form of HTML email                   
                 messages to the Rules Manager.  If the Rules Manager           
                 usercode has an email entry in the UserData file               
                 then that address is used otherwise the Usercode of            
                 the rules manager is used as the address.  If                  
                 set as a family option then the  FamilManager                  
                 summary report is sent as email instead of being               
                 printed.                                                       
 USEREMAIL     - Sends User reports as an HTML email to the User.  If           
                 the user has an email entry in the Userdata file               
                 then that address will be used otherwise the                   
                 Usercode is used as the address.                            

The RULES TEST command allows the user to simulate the effect that FLEX FAMILYMANAGER will have when the user's rules are applied to his files. A report is sent to the terminal describing where the results of the test can be found. Valid Options: RULES TEST RULES TEST USER <usercode> RULE TEST FAM <familyname> Use of the USER construct Or FAM contruct is restricted to the <Rules Manager>.

The ordinary user needs only enter RULES TEST to run a simulation for his files.

After a RULES TEST command, the user must use the <RULES CONFIRM> command to actually activate any User specified rules.

RULES TEST FAM <familyname> reports in different ways depending on the reporting option set for the Family. See RULE OP ? FAM <familyname> If MANAGEREMAIL or USEREMAIL is set then all user reports and the summary report is emailed to the Rules Manager. Otherwies all user reports are written to disk under the rules manager usercode on family being tested. The summary is printed.

The change described in DNote 520.01 caused erratic behaviour with file SELECT operations and non-FLAT FILE requests. These problems have been resolved.

FLEX scripts will now gracefully abort after any OPAL failure is detected whilst executing a REPORT or SELECT. Previously, FLEX would have continued to process the next selected file which would then caused the same fault to potentially appear many times. Such errors would include CONCATENATED STRING TOO LONG, DIVIDE BY ZERO etc.

When processing commands such as FILES = ON DISK:ACT where family substitution was in effect, FLEX would fail to perform requested actions, such as REMOVE or ZAP, for those files found on DISK. This problem was caused by FLEX not removing family substitution before applying the action and in effect only looked on the primary disk. This problem is now fixed.

This fixes the backup counts in LISTVL, altered by Dnote 510.31.

The UNSEQ option has been disabled for the FILES command; this allows an inconsistency with the UNSORTED modifier to be resolved.

The report generated by a FILES:n command, where n=directory level, will now always show file and directory entries correctly and no longer append '[NR]' where an entry is a directory only.

Due to recent changes, any ZAP command applied by a FILES:FLAT,ACT command would only remove the file and not purge any associated catalog entries. Previous FLEX behaviour has been restored to allow full ZAPs to occur when using :FLAT. A minor problem during :ACT handling caused the information message for the processing of a single file to be lost has been fixed.

Several problems have been corrected which could occur when recovering a complete disk family.

A seg array error could occur if more than 13 source volumes were required.

If the long filename option was set and files with more than 12 levels were to be restored the restore job would fail with syntax.

It was hard to identify Jobs which had failed syntax. Any failed Reload Jobs are now saved in the directory JOB/RELOAD of the user doing the load.

Loads started using the :linear option would not find CDImage containers.

Recent changes to FLEX caused report headings to be suppressed from a USAGE analysis when the :FILE or :APPEND modifiers were used. Both :PRINTER and the usual screen reports were unaffected. This problem is now fixed.

FLEX will no longer try to use the :ARC report id :FAST is set.

Archive Entries can have up to 4 backups. LOAD can now handle this. The ancient (1978) procedure that optimizes the volumes to use in the LOAD has been moved from FLEXLIB to FLEX. In some cases where files are backed up on one volume only, some unnecessary mounts are avoided. No extra memory is needed, but very large loads may use a little extra CPU time.

The ARChive option has been implemented. It gives the :CAT report, but forces the Archive to be used. It is valid for FIL,BACK,LOAD,ZAP,PURGE. For ZAP and PURGE, :ARC only purges the Archive, not the Catalog. :CAT purges only the Catalog, and if neither is given, BOTH are purged. Of course, the Catalog can only be purged if MCP Cataloging is set.

The :CAT reports have been updated to look better if data are missing.

In a :LIN search with no MAINT ALT CAT set, FLEX has until now looked for a *SYSTEM/CATALOG on the DL CATALOG family. Now, if the MCP Cataloging option is reset, (or, for testing, if a SEL or REP uses the ARCHIVED attribute as in SEL NOT ARCHIVED), FLEX will now search the *SYSTEM/ARCHIVE/= files on the DL CATALOG family. Even archives not INUSE are searched as this is effectively the same as searching the files of a non-mounted family in a Catalog.

PDT CD/MT <unitno>, broken for many releases, is now fixed.

The :CAT and :ATT,CAT reports have been re-written.

The reports are now suitable for use with the Archive, giving extra information and Backup entries. When CopyWrite Disk Farms are used they return the HostName. The :ATT,CAT report has all the abilities of both the latest :CAT and :ATT reports.

The :CAT report now works properly for :LINEAR and so the separate :LINEAR,CAT report has been suppressed.

The LOAD <fn> #<gen>, broken by Flex Dnote 510.18 has now been restored.

FLEX has a report for the LF command using no PD= attributes. This means it is much faster and works for all FLAT calls such as :FLAT, MT/CD <unitno>, IN <container> etc. Some new info has been added to reflect attributes that have become available since the first LFILES report was written. However, 3 atts (i.e. PPB, Cataloged, Archived) are no longer reported. An updated version of the old report is accessible from an LF:ATT, and is also used with :LINEAR or from the ACTion screen.

To support the new Flex Format and LF, FLEX uses the changed SameIds format and fixes a bug in USAGE.

If :ATT is set, FLEX will no longer check the attributes used are all compatible with the search method. This allows user reports to use SearchType and Valid to choose which attributes to display.

Flex will now request faster keyed searches on Cataloging systems and will give warnings for more invalid searches.

Maint Merge Files will now require a * if a merge of a system directory is required.

The MAINT MERGE VL process will now update the creation date, savefactor and site creation details of any new, merged volume with settings from the specified alternate catalog. Previously, MAINT MERGE did not assign these values, leaving them as UNKNOWN or zero.

The FAM modifier for the RULES command is now permitted as the minimum abbreviation for the FAMILY keyword. This was unintentionally blocked by the change described by DNote 510.23.

Version 510.15 introduced a bug in the "Maint Merge Files" command. This problem has now been corrected.

FLEX will now recognise the use of BEGIN blocks with REPORT from external files or the current worksource. FLEX will scan the file looking for the first valid keyword as 'BEGIN' and will compile the script accordingly. For example:

Worksource TEST/SCRIPT:

00000100 % This is a test 
00000200 BEGIN 
00000300 SHOW(TITLE); 
00000400 END;

This file may be compiled as a worksource with the REP command or, externally:

REP FROM TEST/SCRIPT

- BACKup -+---------------------------------------------------------+>>>
          |-- TO <familyname> --+---------------------------+-+-----|
                                |-- ( -- <att list > -- ) --| |- , -|
                                +---------<---------- , ----<-------+

>>> ------+-+-- <directory> ---------------|----------|
          |-- <pattern> -------------------|
          |-- <file name> -----------------|

BACKUP is used to make copies of files resident on PACK families to other destinations. After it has selected the files it generates a LIBRARY/MAINTENANCE ADD command to move the files. If no TO clause is specified, FLEXBACKYYDDD is assumed, where YYDDD is the current 5-digit Julian day. If the option TWObkup is set, two destinations are used, by adding Y and Z to the usual volume name. If the option CATalog is set, an ADD&BACKUP is done, unless the system option Cataloging is reset, in which case an ARCHIVE FULL is used.

Examples:

  BACKUP XYZ/=              - Copy the resident files in XYZ/= to a tape
                              named, say, FLEXBACK06123.
  BACK TO C(CD),Y/=:TWO,CAT - Copy Y/= with ADD&BACKUP to 2 CDs CY & CZ

Some cleanup of parsing and obsolete undocumented options has reduced the FLEX source by over 600 cards.

Usage of the SPLIT command after generating a printed report could produce an I/O ERROR:UNKNOWN ERROR fault when the next print action was effected. This problem is now fixed.

The change applied by DNote 510.23 caused the USER command to be inadvertently disabled (only 'USERCO' or greater was accepted). The USER synonym is now available again.

The PURGE command can be used wherever the ZAP command can be used, with the effect of doing a CATALOG PURGE. But unlike ZAP, it does not REMOVE the resident entry. The obscure PGONLY option will be removed in a future release.

This change considerable simplified the FLEX source, which is now over 200 lines shorter.

OPAL code REPORT BEGIN blocks now supports several new to control script execution. These statements are QUIT, EXIT and ABORT and their semantics are shown below:

    --+--- ABORT ------ ( <OPAL string> ) -------------+--------|
      |                                                |
      +--- EXIT ---------------------------------------+
      |                                                |
      +--- QUIT ---------------------------------------+

   #29066 FLEXLIB:ABORT:too many files
   #DO interrupted at line 00000620 of DO file ...

The QUIT statement is different in that the current scripted function, such as a FILES command, is gracefully terminated but later activities in the script will be processed normally.

Destination attributes in a LOAD are now checked to catch most errors before passing them to WFL. Some problems with long and unusual lists have been corrected.

Setting invalid mask bits in the OP DEBUG option would cause an INVALID OPERATOR @ 30381000. This option, intended for Metalogic usage only, is now protected from illegal mask values.

This minor change allows an attribute list after the Familyname in the LOAD TO case. e.g. LOAD TO A(KIND=CD) X/Y

The list has to be a valid WFL <destination volume attribute list> in a COPY statement. Because of the possibility of ambiguity (e.g. is "KIND=CD" as usercode pattern?), an optional comma can be placed after the ")", e.g. LOAD TO Y(TAPE), X

If there are multiple source volumes, the list will be added to the ADD destination in each job.

The LOAD and LFiles commands can now have the :ACT option. As with REL this pre-loads the corresponding command into the action field. In the case of LOAD, :ACT allows the LOAD TO <family> variant. FLEX will also now correctly display errors for the whole search. This change was implemented by forcing LOAD to use the same directory list handling as the other commands. This improves maintainability and reduces the source by 120 lines.

A new FLEX command, called SPLIT (or SPL), has been implemented. When used, this causes FLEX's current LINE file (used for report printing) to be closed and released to the Print System as a print request.

FLEX Utility now has a major functionality change in the way that REPORT Opal scripts can be defined. REPORTS were intended to generate simple or complex strings for file, print or screen output and are identical to SUPERVISOR DISPLay programs.

This behaviour has changed; FLEX will now permit REPORTS to be alternatively composed into OPAL blocks, similar to SUPERVISOR ODTSequences, so that they can use statements. The use of blocks is controlled by the key word BEGIN and is permitted for normal REPORTs, HEAD and FOOT.

    -- REPORT ---+----------+--+----------- <Opal string> --------+---|
                 |          |  |                                  |
                 +-- FOOT --+  +-- BEGIN -- <Opal block> -- END --+
                 |          |
                 +-- HEAD --+

The use of BEGIN to create OPAL blocks is currently mandatory. Due to the complexities of OPAL variable assignments, it is not yet possible to unconditionally create OPAL block code for the traditional OPAL string scripts.

As with ODTSEQUENCEs, statements in an <OPAL block> must be terminated by a semi-colon (;).

New OPAL statements, some familiar to SUPERVISOR users, are now permitted from a REPORT BEGIN block; these include the following:

   ----+--- DISPLAY --+------- (<OPAL string> -------------------|
       |              |
       +--- ODT ------+
       |              |
       +--- SHOW -----+
       |              |
       +--- WFL ------+

The ODT, DISPLAY and SHOW statements are identical to their SUPERVISOR equivalents. SHOW displays string contents back to the originating station; the DISPLAY command forces a display message of the string to be generated. ODT passes the string contents to the MCP using the DCKEYIN function; it is only available to Privileged or SecAdmin usercodes.

The WFL statement allows WFL-only commands to be locally processed (i.e using the current session attributes such as USERCODE and FAMILY) and, when associated with a REPORT, will act on each file returned by a SELECT. Because of this, the WFL statement should be used with care.

PRINT has not been implemented as printing behaviour is controlled with the usual FLEX options and command modifiers. If OP+PRINT is set or the :PR modifier is used, all SHOW statements will be printed.

In addition, several SUPERVISOR-only OPAL functions are now available for use in FLEX. These include the KEYIN and MAIL functions which function identically to the SUPERVISOR variants.

Some examples: the simple report 'REP TITLE' is identical to:

    REP BEGIN
           SHOW(TITLE);
        END;

To REMOVE files as they are reported using the WFL function:

    SELECT DAYS(TODAY,CREATEDAY) Gtr 7
    REPORT BEGIN
              WFL("REMOVE ",FILENAME);        % acts on current file
           END;

Variables are fully supported in REPORT BEGIN blocks; assignments are permitted as statements as in SUPERVISOR and string or real values stored by assignments are not passed back to FLEX for display.

To analyze the contents of the selected file, searching for a target string:

    SELECT SYMBOLFILE
    REPORT BEGIN
              $T:= TITLE;
              $TEXT:= $$T.READ(NOSEQ);       % Read file without seq num
              While $LINE:=$TEXT.SPLIT(/) Do
                 If $LINE Eqw "=SEARCHTEXT=" Then
                 Begin
                    SHOW("Text found in ",$T);
                    SHOW($LINE);
                 End;
              End;
    FILES MYFILES/= ON =:FL

The above script is a very simple example of a CANDE FIND, operating on the selected MYFILES directory on all pack families.

To email a selected file as an attachment from a FLEX DO:

    REPORT BEGIN
              $TEXT:="^ATTACH "&TITLE;
              If #RES:=MAIL("To:support@metalog.com;Subj:Req File",
                            $TEXT) Lss 0 And #RES Neq -999 Then
                 SHOW("Mail failure: ",#RES);
           END

Both REP HEAD and REP FOOT can use OPAL blocks in the same way. Existing FLEX DO files and FAMILTYMANAGER rules are unaffected by this change. Note that the FAMILYMANAGER rules mechanisms will not be enhanced to use this new feature.

If 2 directories in a list can be found in the same search, they will now be combined if that speeds up the search. For example: F X,Y:FL will now involve only 1 scan of the flat directory, not 2 as before.

510.11 made searches on DISK incorrectly following FAM substitution. This change returns to a search on the real DISK.

Family Lists caused :ACT to miscount lines and led to the first page continuation occurring before the footer was sent. This problem is now fixed.

The enhancements to USAGE broke its use with Wildcards. It now works, and has a better formatted :BR report.

Some incompatible options have better error messages, e.g. :LIN,ACT

If a directory has no <ON part>, its family defaults to DISK, and from there to the 'FAMILY DISK=' substitution. So a list like:

Files X, Y ON ZPK

meant Files X ON DISK, Y ON ZPK

This has been changed. An omitted family defaults first to the family of a subsequent member of the list. Only if there is no explicit family afterwards, will it default to DISK.

So the example now means

Files X, Y ON ZPK == Files X ON ZPK, Y ON ZPK

This also applies to containers, units, etc:

Files X, Y MT 27 == Files X MT 27, Y MT 27 Files X, Y IN ZZ == Files X IN ZZ, Y IN ZZ

But unchanged is:

Files X ON ZPK, Y == Files X ON ZPK, Y ON DISK

FLEX has long allowed some commands to process a list of directories:-

Files X,Y REM R/=,S/= LOAD L,M/=

In the LOAD case, this list is combined and processed, but for the others, the list was equivalent to multiple commands on single items:-

Files X,Y == Files X then Files Y

FLEX now combines the searches. In the Files command, the results are returned together. If :ACT is set, or for ZAP/REL/REM, there is only one authorisation step. USAGE also combines the results, however if :BRief is set, each item on the list is broken out.

FLEX will now report the number of catalog entries (if relevant), found during a :LINEAR search, that satisfied the SELECT criterion in the statistics footing at the end of a FILES report. Previously the number of generation zero files was always shown; this was confusing since it would often not match the number of SELECTS that returned TRUE.

When using the :FILE option flex attempts to split long filenames across multiple records. If a single level of a filename was longer than 80 characters the filename was corrupted.

This change corrects the problem.

If Cataloging is not set, Flex will now try to use the Archive to get the equivalent information. Only the first 2 Backup References will be returned. This change principally affects LOAD except when :LINEAR, and Files (such as in :CAT).

For sites not running Cataloging, LOAD :FAST allows for better recovery if a family is lost. First, do a COPYCAT of the SYSTEM/ACCESS, then RC a pack with the same name, run FLex, and set MAINT ALT CAT to the the old SYSTEM/ACCESS. Finally do LOAD:FAST, This is much better than an ARCHIVE RESTORE because only the files that were resident when the family was lost will be re-loaded, and Flex will better optimize the tape loads.

The FA(ST) option is available to allow users with the necessary privilege to search the File Access STructure. Use MAINT ALT CAT to specify a non-active Catalog/Access. This search is much faster for searching whole families or the entire system, but only SEARCHTYPE and the attributes of the file name are available.

If a FLEX session uses a print file DESTINATION identity beginning with the text 'MAIL', then FLEX will suppress the headings from various printed reports. This allows OPAL reports, FILES reports etc. to be generated in an email-friendly format to be without extraneous report headings.

The default output from the FILE command no longer uses an OPAL report. This saves about 5 ms per file reported and obviates the need for 2 files in the FLEX/LIBRARY/REPORT directory. The :BRIEF option is now valid for the FILE command and gives a report with only the TITLE. The FTV command will no longer ignore syntax errors in the input files.

Previously, if a ':' was used inside comment text following certain commands such as SELECT e.g.

SELECT FILEKIND=DCALGOLCODE % 10:00

Then the SELECT would fail with:

Option LEVEL incompatible with SELECT

This problem has been fixed; FLEX will now ensure that special characters such as ':' are ignored after the % comment delimiter.

If the :print or :File option was used with the PDT LD command, it was almost certain that Flex would fail with a 'SEG ARRAY' error. This problem has been corrected.

The ListVL LOWUSE command will now report the size of volumes in megabytes for statistics files converted by Mergetape.

In line with changes to FamilyManager and MergeTape, FLEX will now report MegaBytes and not Feet in the DEFAULT screens.

The Opal compiler *SYSTEM/OPAL can now compile Flex Do files but to be able to correctly compile Do files with parameters, it needs examples of valid parameters. These examples are provided in the Parameters heading.

Ex. PARAMETERS(FAM=A,MAXDAYS=10,MAXSEGS)

These assigned values will then be used in the compiled script to replace individual parameters. If no default is provided, SYSTEM/OPAL considers the parameter as numeric. Note that, in the current implementation, all default values must be shorter in length than the original parameter name to avoid complex substitution issues.

Flex now respects this syntax but note that the supplied values are not used as defaults when the DO file is run.

A problem which occurred when using a parameter in a quoted string which, after substitution, was split across two lines, has been corrected.

FLEX will now protect against I/O ERROR END OF FILE failures on the FLEXSOPTIONSFORME file, typically if the file was open through a H/L. If FLEX now detects a file in this state, a new FLEXSOPTIONSFORME file will be created from the old file which is then purged.

The DEFAULT command screens now allow for LIBMAINTDIR files to be made when MergeTape or FamilyManager make tapes. There are 3 settings:

0 (Auto) - Make LIBMAINTDIR files if the density can have a drive which is LOCATECAPABLE. This is the default.
1 - Always make a LIBMAINTDIR.
2 - Never make a LIBMAINTDIR.

If a tape has a LIBMAINTDIR, and is mounted on a LOCATECAPABLE drive, COPY can use direct positioning to speed up file access. A LIBMAINTDIR file is also needed to enable a subsequent LIBMAINTAPPEND=TOEND copy.

LOADs from Disk Farms will now use the "'#<file no.>'" syntax to resolve conflicts.

The response to a PV command will no longer indicate PK in the heading if it is not a real PACK. The device mnemonics are:

       CD = CD ROM
       CI = CD Image
       DF = Disk Farm

LOAD will now use a LIBMAINTDIR if available. The info in the LIBMAINTDIR will override the FLEX Verify option for the COPY. Also if the tape is not one made by FLEX, or if it has been appended to, FLEX will check where the correct files are, and, if necessary, use the '#<file no.>' syntax to access them.

A bug in handling tape RESOURCE specs has been fixed and obsolete TAPE7 and TAPE9 resources are no longer calculated.

The behaviour of the SECURE command has now been considerably enhanced. Previously, attempting to secure files with FILEKINDs of CATALOG, KEYSFILE or SYSTEMDIRFILE would give a SECURITY VIOLATION message without indicating which file had failed. Now FLEX will not attempt to change these files but give the warning messages:

#FLEX cannot SECURE SYSTEMDIRFILE or CATALOG: *SYSTEMDIRECTORY/001
#FLEX cannot SECURE KEYSFILE: *SYSTEM/KEYSFILE

Also, FLEX will now more accurately verify if a GUARDFILE name, provided by the GUARDED or CONTROLLED modifier, is already assigned to the file. As MCP, FLEX will automatically assign the usercode and/or family if they are missing. Previously, FLEX would always fail the check if either usercode or family was missing thus causing FLEX to reassign the GUARDFILE.

For example, if FLEX is running under the user META on family DEV:

SECURE TEST/= GUARDED MYGUARD/TESTFILES

This command would secure files in the TEST directory with the GUARDFILE name:

(META)MYGUARD/TESTFILES ON DEV

On older versions of FLEX, repeating the above SECURE command would have caused FLEX to re-assign the GUARDED attribute because the internal file title check would have not matched.

FLEX will now generate the usual MCP 'SECURITY CHANGED' message when changing a file security whereas before, FLEX used to open the file internally and change its security attributes directly. This allows the security changes messages to be easily tracked and reduce the amount of logging to the SUMLOG (no file open/close records).

The LISTVL command will now show the correct number of backup refs for any volume that has more than 4096 references.

When entering a Report command from a DO file it was possible to get unexpected syntax errors. The problem could happen in the rest of the line of the Report command was blank and the next line started with blanks. This problem has been corrected.

Previously FLEX would P-DS when trying to change the security of an invalid KEYEDIOII data or key file using the SECURE command. This was caused by the KEYEDIOII library rejecting the request by Flex to open the file. This problem has now been fixed.

PDT can now take the parameter LD to access the names in a LIBMAINTDIR.
The new syntax is:

-- PDT -+---------- <volume name> ---------------------------------+-|
    |---------- <device> --- <unit number> --------------------|
    |--- BU --- <device> --- <serial number list> -------------|
    |                                     |
    +--- LD -+-------------- <serial number> ------------------|
          |       |                       |
          |- <device> -+                         |
          |                               |
          +- IN --- <file title> ---------------------------+

<device> ::= MT | CD | PK

The 'IN' form takes the full title of the LIBMAINTDIR must be given. Note that 'F <pattern> in <file title>' is another way of looking at the contents of a LIBMAINTDIR. When <serial number> is used, FLEX tries to find the name using the Volume Library, Volume Directory, or TRIM, one or more of which must be present. If <device> is omitted, FLEX assumes first MT, and failing that PK/CD.

  PDT LD C00023
  PDT LD IN (PROD)LIBMAINTDIR/DBBKUP/20040312/C00023 ON DLPK

Some cleanup was done for other PDT forms and their reports all now have a heading at the terminal.

A small robustness patch was added to HELP GEN to avoid a potential infinite loop.

In the cause of standardisation, the MAINT ALT FLAT command will now allow a (CD) suffix to indicate that the file is on a Unisys format ISO CD. Also :ATT can now be used with :LIN.

'Enter HELP and XMT for Instructions' is displayed at the top right

Flex would hang requiring DISK if the UserData FAMILY was absent and there was no family DISK on the system. In this case Flex will use its task family, or if this is unspecified, the family of the Flex codefile.

This change removes the irritation of the first field of a :ACT screen being pre-filled with the word 'HELP'.

The first time in a session the :ACT option is used a message 'Enter HELP and XMT foor Instructions' is displayed at the top right of the screen.

Previously, during interactive FLEX sessions only, the DEFAULTS screen form would be unconditionally presented when a DEFAULT command was processed by a DO file or generated by the a Rules Base initialisation from the Flex INSTALL menu. These screens would require user confirmation for every DEFAULT command and would, in the case of the INSTALL variant, take significant time to complete.

This behaviour has now been disabled; the form will only appear for simple use of the DEFAULT command during an interactive FLEX session.

If a parameter passed to a DO was subsequently used to reference a real OPAL variable (likely to be the PERMANENT or CONFIG variety), such as:

	PARAMETER(VARNAME, DIRECTORY)
	REP FOOT ##VARNAME#.PERM:= #FILECOUNT

Then FLEX would give the following warning message when handling the ##VARNAME#.PERM expression:

	#Possible Unmatched Parameter name at line 00000300 

Also, FLEX would give the error 'Invalid DEVICE spec' if an 'ON' part was used with a parameterised DO file e.g.

 DO FLEX/DO ON DEV (TEST/=) 

This problem has now been fixed.

The FTAM protocol identifier is no longer used to select CopyWrite when reloading files from a Disk Farm.

Add message for LOAD from Disk Farm.

In order to be able to create files easily transferable to PCs, a new output option BYTE has been added. This is similar to the FILE option but instead of creating a job symbol file with a limit of 80 characters per line a bytestream file is created with effectively an unlimited line length. The Bytestream file has an extmode of Ascii and can be read by a PC via an NXServices

An expected use of BYTE would be to create comma separated files to be imported into an Excel spreadsheet.

Ex.
REP USER ",", FILENAMEID ",", FAMILYNAME ",", ALTERDATE
FILES (*)=:BYTE "EXTRACT.TXT"

Note that using a .TXT suffix makes it easier to open the file from the PC.

The LIST command has been extended to be able to list Bytestream files. By default each record is prefixed by a size digit record number and is truncated to the screen width. Resetting the Truncate option either using the OP command or as a colon option will list every record completely.

Ex.
L "EXTRACT.TXT":-TR

The Record numbers can be suppressed using the :UNsequenced option.

Ex.
L "EXTRACT.TXT":UN
or
L "EXTRACT.TXT":UN,-TR

The range option in the List command uses record numbers rather than sequence numbers for Bytestream files.

Ex.
L "EXTRACT.TXT" 9-19:-TR

would list records 9-19.

This change also corrects a bug in the List command where a range was ignored if and colon options were used.

Internal version change for MCP 49.1.

FLEX was not correctly handling certain long, quoted filenames when output was directed to a file for use as a WFL job. If the name was greater than 80 characters, it may not have been split in the correct place if any of the deeper levels had quoted names with spaces embedded. This problem is now fixed.

This change extends Flex to be able to reload files which have been backed up to a Copywrite disk farm.

The Default tape density now lists all valid Density mnemonics.

Flex DO files now accept spaces and parens i.e. () in parameters.

Previously, it was not possible to process a FLEX DO file which held any continuation characters embedded within strings in the file e.g.

REPORT HEAD "REPORT TITLE 40,,CREATIONDATE,, \"

Typically, the \ character would only be needed for generating a second DO file, but the above line in a DO file would fail with "Missing quote".

Now, FLEX will automatically check for the presence of any \ character with a quote immediately following it. If the string is found then FLEX will not consider this as a line continuation and will proceed to scan for the next occurrence.

If the USAGE command was used with a level option and a wildcard pattern, such as 'USAGE (*)=:1', the command would fail with the message 'The level option and wildcard patterns are incompatible'. This restriction has been removed.

Flex did not recover files to DISK from CD Images on a named Family.

The Title of a CD Container could be incorrect when building the Job to reload a file.

If a LOAD of a non-resident file from a CD image was initiated by Flex and the CD images were held on DISK, it was possible for the LOAD to fail with a NO FAMILY FOUND result, depending on the family substitution of the user. This problem is now fixed.

This change supports the change made in Familymanager Dnote 480.05.

Previously the FILES:ACT command did not return the Familyname of the file. This meant that FILES ...:ACT ON = was not practical. This deficiency has been corrected.

A related problem where the ZAP command would only work for files on the default family has also been corrected.

This change supports the introduction of variables in Opal, as described in Opal Dnote 480.14.

There was a problem when using *... directories in a list. Any wild card used in the usercode of a directory which appeared after a *.. directory in a list would only match non usercoded files.
Ex.
FILES *=/BOB,(=)=/BOB
would not give the expected results but
FILES (=)=/BOB,*=/BOB
would work correctly.

This change also corrects a problem when using the # wildcard pattern. The # is also used to delimit parameters and the use of ## in a wild card pattern was incorrectly reported as parameter error. Now, if no parameters are specified in a DO file the use of ## wildcards is the same as before the implementation of Parameters. If parameters are specified and a ## is encountered, a warning is given of a possible missnamed parameter.

Errors reported in DO files, where the input was from as card deck, always reported the sequence number of the error as 00000000. The sequence number is now reported as a relative card number.

i.e. 00000001,00000002,00000003 etc.

Previously, when processing a FILES:PRINT command without a:FLAT modifier, the printed report would have a page skip after each switch of directory. A similar report with :FLAT did not show the same behaviour. This problem is now resolved.

This patch only changes the version to 48. Version 470.17 is still valid for MCP 48.

The LIST and WRITE commands have undocumented sequence range modifiers. e.g. LIST 100-500 or LIST 800-2000,2300-END. Up to 14 range pairs can be specified. However, only the first sequence range would work and plus a LIST of any file whose name started with a number, followed by letters or numbers would not be recognised. Both these problems have been fixed and the on-line help now includes sequence range info.

The changes implemented by patch 470.15 inadvertently highlighted an existing problem where Flex was not correctly handling any SElECT or REPORT that used the FROM modifier to load an external file. This might have caused any SAVE command to incorrectly write information to the saved file. When encountering this situation, with the changes applied by 470.15, FLEX may have faulted with a SEG ARRAY error.

This fault has now been fixed.

This change allows parameters to be passed to FLEX 'DO' files. The implementation is similar to that provided by CANDE and is illustrated by the following example DO file:-

     PARAMETER (DIRNAM,SOURCEFAM,TAPENAME)                                      
     REPORT FOOT "FROM #SOURCEFAM#(PACK) TO #TAPENAME#",/,"END JOB"             
     REPORT HEAD "BEGIN JOB SAVE/#TAPENAME#;",/,"COPY"                          
     REPORT IF MYSELF(ENTRIES)=1 THEN EMPTY ELSE GETSTR("OLD"),\                
            STORE("OLD",(IF SUM("FCNT",1) > 1 THEN "," ELSE EMPTY)),\           
            STORE("OLD",&FILENAME)                                              
     SELECT "#DIRNAM#" ISIN TITLE                                               
     FILE ON #SOURCEFAM#:FLAT,FILE WFL/#TAPENAME#

The parameter names are defined within parentheses on the first line and in this example are DIRNAM, SOURCEFAM and TAPENAME. The parameters are referenced by enclosing the name within #'s eg. #SOURCEFAM#. When the DO file is invoked references to parameter names will be replaced by the parameter values supplied.

A DO file that expects parameters is invoked by following the DO file name by a list of parameter values enclosed in parentheses. For example if the above example was called MY/DO/FILE it could be invoked as follows:-

     DO MY/DO/FILE (METALOGIC,PACK,METADUMP)
 

This would result in METALOGIC being assigned to the parameter DIRNAM, PACK to SOURCEFAM and METADUMP to TAPENAME.

This initial implementation does not support the assignment of DEFAULT values to parameters.

If the output from a FILE command was directed to a file using the <file option> the next message displayed by FLEX could contain an erroneous character. This has now been corrected.

Previously, FLEX Inquiry could fault with a SEG ARRAY error if certain command input (esp. SELECT or REPORT) was exactly 132 characters in length, including any trailing spaces. The problem could be readily detoured by adjusting the input length by one character. This fault is now fixed.

Implement Files AT <hostname> and Usage AT <hostname> by calling the CopyWrite entrypoint FetchForeignHeaders. This command provides Flex access to Disk Farms on remote hosts.

The directory *METALOGIC/FLEX/DEFAULTREPORTS is used to hold both the reports used by flex internally and reports listed in the STD command. The internal reports should not be listed in the STD command. ACTFLAT used for the command FILES ...:ACT,FLAT and ATTFLAT used for the command FILES ....:ATT,FLAT were listed in the STD list or reports. This is no longer the case.

With this release the STD reports have been refined. Some of the original reports are now obsolete. A new STD report CLEANSTD will remove the obsolete reports (Enter STD CLEANSTD). Another new STD to report HELP give a brief description of the other STD reports (enter STD HELP).

When running on screens wider than 80 characters, it was possible for Flex to terminate with a seg aray error when executing a SAVE command.

FLEX will now correctly handle long file names on system with CATLOGING and the SYSOPS LONGFILENAMES set if encountered during a MAINT MERGE operation.

FLEX INQUIRY now supports the LISTVL and INSTALL CATALOG commands on cataloging systems with SYSOPS LONGFILENAMES set, as discussed in FLEX LIBRARY Dnote 470.08.

The SAVE command has been enhanced to allow an optional 'AS' before the file namme; previously the 'AS' would have been assigned to the title. In addition, saved state files are now crunched.

On 80-column terminals, depending on the emulator, the status line shown at the end of a REP or SEL interrogation was not appearing on a new line. This problem is also fixed

A previous change (see 440.02) to protect Flex from slow Tape management systems resulted in Flex being unable to detect WFL DATA decks - Flex would report that the Batch Input file was not available. This problem has now been corrected.

In addition, Flex will now terminate after generating a warning message if it is running in Batch mode and is unable to open a specified DATA file.

The interactive interrogation of active SELECT and REPORT expressions using SEL ? and REP ? have been much improved. Both these commands will now display the original content of the SELECT or REPORT from the home position of the terminal, allowing easy modifications.

e.g. a SEL ? might return

  SEL FILEKIND NEQ {BACKUPPRINTER,LOG,SYSTEMDIRFILE,CATALOG} AND 
     NOT TITLE INCL "DIAGNOSTICS" AND 
     FILEID(FILENAME,1) NEQ {"PDUMP","SUMLOG","GOODDISK","METALOGIC",
                             "METANOTES","PATCHNOTES","PATCHESFOR", 
                             "PUBLIC","ENVOY","NJEDB","SNAPS", 
                             "ATLAS","SYMBOL", 
                             "TRACE","SUPERVISOR","FLEXREPORTSFORME"} 
  #Current SELECT is enabled

FLEX will also better use the attributes of larger screens lengths and widths; using the whole screen to return the expression, using truncation if necessary.

This patch also fixes the corruption and loss of data when using the SAVE command from terminals running with large term specifications. Where appropriate, line wrapping will be used to protect expressions with individual lines greater than 80 characters. Also, if a DO file failed syntax when run from an online Flex session, the internal TD830 was reset.

This problem, which then caused all user input to be treated as TTY input, is now fixed.

This change allows Flex to reload files backed up to CopyWrite Image files.

The LISTVL report for volumes which were Copywrite CD images, or CDs reported each file backed up twice. This change corrects the problem.

In order to be able to pass parameters to Flex do files a limited File Equation facility has been provided. In Batch mode any single level filename used in a Files like command (FILES,REMOVE,RELEASE,ZAP, FORGET) may be file equated. The file equation is only used if there is no Usercode or * prefix.

Ex.

With a Flex DO file Of TEST/DO: 
 REP FROM TEST/REP 
 SEL FROM TEST/SEL
 FILES X :FILE WFL/TEST 

In a job you could then use:

 RUN OBJECT/FLEX("DO TEST/DO");FILE X = BOB ON MERGE                    
 FILE X (TITLE=#("*BD/" & STRING(MYJOB(JOBNUMBER),7) & "ON DISK"));

This would scan all *bd files created by this job. If the DO file had REMOVE X/= all *BD files created by the job and matching the selecion in TEST/SEL would be removed. Note that in order to remove a directory the /= is required.

You can test the behaviour of file equations when running Flex online by setting SW2.

Ex.

 U FLEX;FILE X= MY/DIRECTORY ON TEST;SW2 

A FILES X command would then scan MY/DIRECTORY ON TEST.
FILES X/= would also scan MY/DIRECTORY ON TEST.
FILES =X= would ignore the file equation and scan for any file with X in the title.

Various formatting improvements have been made to the Flex Help file, *METALOGIC/FLEX/TEXTBOOK. Several internal amendments have been applied to procedures used to maintain the Help file.

Implements the more disciplined masking needed by GSTABLEGEN 460.09. New :ATT and :ATT,CAT reports have been introduced which give more info in less space. The :ATT report is usable with :FLAT or MT/CD searches. The report for :LIN,CAT now gives the SECURITYTYPE.

HELP ATT now gives info about the ATTribute option.

This change allows more selective access to the INUSE attribute, which is expensive to process during :FLAT. Candidates for REM, REL, or ZAP are no longer retained if a BRK is done during the "Now Scanning .." phase.

There appears to be a problem with the Unisys RESTORE command on some MCPs where trying to copy from the second or subsequent reel of a multi family multi reel backup. Setting the config variable FLEX_NORESREEL2=TRUE cause COPY to be used instead of RESTORE. This means that if two files with the same title exist on the tape then it is possible that the wrong one will be loaded. This option should only be used until the MCP bug is fixed.

To set the option enter U META/INSTALL FLEX_NORESREEL2=TRUE

To Reset the option enter U META/INSTALL -FLEX_NORESREEL2

The PV command now correctly returns STATS info for numeric serials.

FLEX will now permit input greater than 1920 characters from a terminal that has TERM settings up to 47 lines, 96 characters wide. Previously, FLEX would have truncated the input.

FLEX will now remember the CANDE TERM values after a USER command.

The use of ODT and KEY commands from any privileged usercode can now be disabled by setting the config option, FLEXU_NOKEYIN to TRUE. This can be enforced by the INSTALL utility:

     U META/INSTALL FLEXU_NOKEYIN=TRUE

If set, any KEY or ODT command by a PU will return the error message:

     #ODT commands not permitted

The ACTION screens will now use the full page size. The authorisation screen now presents (PAGE_SZ - 11) files, rather than the 10 it used to present.

The LF action item is now implemented.

Default actions are now displayed for 1 or 2 character FILENAMEs and error lines will no longer contain end of line detritus.

FLEX will no longer do an automatic MA APL of its code file.

FLEX will now handle screen sizes other than 24x80 up to 8192 chars. The dimensions are read from the CANDE TERM values.

Reports that have multiple lines per file will now repeat the first line for any file that is split across a page boundary.

The LFILES report has been replaced. It now contains much more info, particularly about code files. LFILES is now compatible with :FLAT. See HELP LF for details of the contents of the new report.

F and LF are now the minimum abbreviations of FILES and LFILES.

FLEX now supports the code form of the standard reports in FLEXLIB. The FLEX/LIBRARY/REPORT files used in FTV can now be 80 character records and can be split between the usercode and * directories.

This change allows zero to be specified as the length of an output tape. If zero is specified an infinite length of tape is assumed. This means that multi reel volumes may be created.

When a LOAD command was used from the Action screen and the tape containing the file to be loaded also contained files from another family, the load would fail. The generated job failed with the syntax error 'ORIGIN' CLAUSE REQUIRED.

The LOAD command now works correctly.

The USAGE report, in particular with :BRIEF, was reporting segments values greater than 10,000,000 as exponent values e.g. 1.18E+8. The report layout has now been changed to handle much 12-digit values for segment totals and 7-digit for file counts.

The syntax of the FILE and USAGE commands has been enhanced to allow searches on Library Maintenance tapes, CopyWrite Images, and both ISO and Library Maintenance CDs. Tapes and CDs can be also specified by unit number. The syntax is:

Using the (MT) modifier, or MT <unitno> FLEX looks for a Library Maintenance tape and then reads down the tape to extract the file information. The attributes available are the same as in a FLAT search. This is quite different from the TAPE family, which is available on Cataloging systems and gives the catalog info for non-Library Maintenance tapes.

Using the (CD) modifier or CD <unitno>, FLEX looks for a CD. If it is a Library Maintenance CD, it is treated in the same way as a tape. If it is an ISO CD, FLEX uses the MCP GETSTATUS procedure. The valid attributes on ISO CDs are limited.

The IN modifier locates a container for the file search. Currently the container must be a CopyWrite Image file on disk or ISO CD. The attributes available are the same as in a FLAT search.

An <input title> has the format:

FILES returns the info unsorted for containers or Library Maintenance formats. On ISO CDs it returns files in ISO directory order.

Two new Directory attributes have been added to facilitate the new searches. SEARCHTYPE, returns a mnemonic of the search method being used. ORIGIN is only valid on a Library Maintenance tape and returns the disk family that the file originally came from.

LIST has been enhanced to allow access to files on ISO CDs. The syntax now uses an <input title>