Distributed Disk Farm

Overview

The Distributed Disk Farm feature allows native A-Series files to be copied to and from disk storage devices attached to Windows NT or Windows 2000 systems in a TCP/IP network.

When the CopyWrite services are installed on a Windows 2000 system, the system becomes part of the Library/Maintenance CopyWrite Native File Transfer Network.

This feature allows an enterprise to harness the immense disk resources which are often present on the network of PCs.

The destination volume name of an A-Series WFL Copy statement is combined with the Disk Farm Root Directory on the Windows 2000 system, to create or locate a directory on the destination hostname. CopyWrite calls this directory the Windows Volume.

The A-Series files in the Windows Volume are stored in a similar way to those in a Library/Maintenance CD-ROM volume. The directory for the Windows Volume is in a file called FILE000.dir. Each A-Series file is stored in a compressed Windows file called FILEnnnnnnn.mcp. CopyWrite calls these files "Capsules".

Each capsule is formatted as an A-Series Library/Maintenance CD-ROM volume.

This means that any of the individual files can be burnt directly onto CD-ROM media. Thus, they may be mounted and read by the MCP on an A-Series computer, or read by CopyWrite for NT, or CopyWrite on the A-Series can copy the file directly from the capsule.

Files are copied to and from the Disk Farm using the WFL COPY, ADD and ADD&COMPARE statements. A COPY statement copies files to a new Windows Volume. An ADD statement adds files which are not already present to a Windows Volume. An ADD&COMPARE adds a file if it does not already exist, otherwise it replaces the Disk Farm file only if the file being copied is a better version, as determined by the Cycle, Version, Creation Timestamp and TimeStamp attributes.

The &BACKUP modifier may be added to these statements, on a Cataloging system, to add a backup reference to the Disk Farm Volume into the catalog entry for each file copied.

The &CATALOG modifier may be added to these statements, on either a Cataloging or a Non Cataloging system, to add a backup reference to the Disk Farm Volume into the archive entry for each file copied.

Installation on the A-Series

The Distributed Disk Farm uses the CopyWrite Native File Transfer services. There are no additional components on the A-Series

Installation on Windows 2000

Execute Setup.exe.

Setup extracts the files to a directory, and then updates the Registry with default settings.

Note: There should be no need to reboot the computer, even if Setup displays the option to restart the computer.

After SetUp has completed, CopyWrite should have been added to the Start menu.

Start CopyWrite, and click on NFT | NFT | Installation Check and press the Check button. This will complete the installation and check that CopyWrite has been properly installed.

The most important thing to check is that the CopyWrite Bridge is running. The CopyWrite Bridge is listening for connection requests from the A-Series. Use the NFT | Bridge | Status menu to check that "CopyWriteBridge is Service, Running".

The Windows Volumes are stored by default in the directory <WINDISK>:\ASeriesBackup. See Disk Farm Root Directory for further information.

The default security is set to *ANYUSER at *ANYHOST. See Security for further information.

Compression is set by default. See Compression for more information.

Now, an A-Series WFL Copy statement, such as, COPY *METALOGIC/COPYWRITE TO COPYWRITE(DISK,HOSTNAME=NX4200NT), will create these files, where the <WINDISK> is G:,

G:\ASeriesBackup\COPYWRITE\FILE000.dir
G:\ASeriesBackup\COPYWRITE\FILE0000001.mcp

The A-Series COPY statement, ADD statement and ADD&COMPARE statement have a particular meaning when used with a Disk Farm.

Files may be copied from the Disk Farm, in the same way as an NFT Copy, to any destination KIND supported by Library/Maintenance. For example, COPY *= AS (META)= FROM COPYWRITE(DISK,HOSTNAME=NX4200NT) TO COPYWRITE(CD,CDCOPIES=1).

Disk Farm Root Directory

The Disk Farm Root Directory is maintained in the Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\Services\NFTkey propertyDLRoot.

During installation it is set to <WINDISK>:\ASeriesBackup, where <WINDISK> is typically Drive C:.

The destination volume name specified in the WFL COPY statement is appended to the DLRoot, to establish the name of the Windows Volume.

The DLRoot property can be changed using the CopyWrite NFT | NFT | Configuration menu;

If there are multiple A-Series in the network, they can all access the Windows Volume, and store and retrieve files from it. This is called a Shared Volume Disk Farm, and is set by default when CopyWrite is installed. The Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\Services\NFT key property SharedByAll is set to 1.

A Private Volume Disk Farm is one which is not shared between multiple A-Series hosts. In this option, the source hostname is appended to the DLRoot, and then the destination volume name is appended. This gives a name, for example, like this, C:\ASeriesBackup\NX4200MCP\COPYWRITE, where NX4200MCP is the source hostname, and COPYWRITE is the destination volume name.

The SharedByAll property can be changed using the CopyWrite NFT | NFT | Configuration menu. However, it may only be changed when the CopyWrite Bridge Service is not running. See Starting and Stopping CopyWrite Bridge.

t is possible for one host to retrieve the files from the Private Disk Farm of another host, by specifying the FAMILYOWNER attribute on the Library/Maintenance COPY statement. When the FAMILYOWNER attribute is specified, it is appended to the DLRoot, and then the destination volume name is appended. For example, COPY *= FROM COPYWRITE(PACK,HOSTNAME=FARM,FAMILYOWNER=CASTOR) , would search for files under C:\ASeriesBackup\CASTOR\COPYWRITE.

Security

The Disk Farm uses a similar security mechanism to that used by the A-Series NFT. An incoming connection from an A-Series provides the UserCode and HostName of the initiating host.

The Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\RemoteUsers Key is checked for a *ANYUSER key. If the key exists, then the ValidHostList property is checked for a HostName called *ANYHOST . If the *ANYHOST HostName exists the connection is accepted.

If there is no HostName called *ANYHOST, the ValidHostList property is checked for membership by the HostName. If the HostName is a member, the connection is accepted.

If the UserCode is not allowed by the *ANYUSER key, the UserCode is used to find a matching key under the RemoteUsers key. If it does not exist, the connection is refused.

If there is a key for the UserCode, a check is made for a HostName called *ANYHOST in the ValidHostList. If present, the connection is accepted, otherwise the ValidHostList is checked for membership by the HostName. If present, the connection is accepted, otherwise it is refused.

RemoteUser entries can be added using the CopyWrite NFT | NFT | Remote Users menu.

Compression

The Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\Services\NFT key property Compression is set by default to 1 when CopyWrite is installed.

If the Compression property is set, when a new Windows Volume is created, the directory is marked as Compressed. Any files which are created in the directory are compressed by Windows 2000. By default, the Windows Explorer shows the compressed directories and files in a different colour.

If the Compression property is not set, then no compression occurs, unless the COMPRESSIONREQUESTED attribute is set on the Library/Maintenance COPY statement.

If the COMPRESSIONREQUESTED attribute is set to TRUE, then the Windows Volume directory is marked as Compressed, and any files copied into the directory are compressed.

The compression under Windows 2000 NTFS is transparent to programs.

Compression may be set or reset using the CopyWrite NFT NFT | Configuration menu.

Mount Points

A Mount Point is a folder in one file system (the source folder), to which is attached an NT logical volume(the destination volume). Any reference to the source folder is a reference to the root directory of the destination volume.

For example, when using DirectCD, the CD Drive can be mounted onto the Disk Farm, so that any files copied from the A-Series to that Volume are copied directly to the root directory of the CD or DVD.

Any volume which supports memory-mapped files may be mounted at a Mount Point, not only NTFS volumes.

A Mount Point may be created or deleted using the MountPoint.exe program provided with the release.

Junction Points

A Junction Point is a folder in one file system (the source folder), to which is attached, the folder of another file system(the destination folder). Any reference to the source folder is a reference to the destination folder.

In the Disk Farm, the DLRoot and a volume name describe a folder, such as C:\ASeriesBackup\TEST, which may have files added to it using an A-Series ADD *= TO TEST(PACK,HOSTNAME=...) command.

If a Junction Point were created with the name C:\ASeriesBackup\TEST, before any files were copied, and then a folder, such as D:\EricTheHalfABee were joined to it, then whenever files were added to to TEST, they would be physically stored at D:\EricTheHalfABee.

A source folder must reside on an NTFS volume.

A destination folder may reside on any directory or device.

A Junction Point can be created using the LinkD.exe program provided with the Windows 2000 Resource Kit.

COPY Statement

The COPY statement is used to copy files to a Windows Volume, when the volume does not already exist.

If the Windows Volume does exist, the COPY statement fails, and no files are copied.

ADD Statement

The ADD statement is used to add files to a Windows Volume. If the Windows Volume does not already exist, then it is created, and then the files are added.

If a file with the same title already exists in the Disk Farm, the file is not added.

ADD & COMPARE Statement

The ADD&COMPARE statement is similar to the ADD statement.

When the COMPARE option is set, a file will only be copied if it is a "better version" than the one in the Disk Farm.

The file being copied is a 'better version' if the file does not exist in the Disk Farm, or if it has a higherCYCLE and VERSION, a later CREATIONTIMESTAMP and a later TIMESTAMP, in that order. It is not a better version if it is identical to the one in the Disk Farm.

&BACKUP Modifier

The &BACKUP modifier may be used on Cataloging systems.

When a file is successfully added to the <destination volume name> on the Disk Farm, a backup reference to the volume is added into the catalog entry for the file.

The ...&BACKUP statement must specify a SERIALNO. If the <destination volume name> is not present in the Volume Library, it is added to the Volume Library with the specified SERIALNO.

If the <destination volume name> is present in the Volume Library, it must have a matching SERIALNO and KIND.

A Windows Volume which is created using &BACKUP is created with the SERIALNO appended to the <destination volume name>. For example, COPY&BACKUP *= TO TEST(PACK,SERIALNO="TEST01"), would on a default system be created as C:\ASeriesBackup\TEST[TEST01].

Files created using &BACKUP can only be restored by specifying a SERIALNO in the COPY statement. For example, COPY&BACKUP *= FROM TEST(PACK,SERIALNO="TEST01").

&CATALOG Modifier

The &CATALOG modifier may be used on both Cataloging and Non Cataloging systems. (Note: The semantics for COPY&CATALOG on a Cataloging system have been changed for Disk Farms.)

When a file is successfully added to the Windows Volume on the Disk Farm, a backup reference to the SERIALNO, VOLUME and HOSTNAME is added into the archive entry for the file.

The SERIALNO must either be specified as a <destination volume> attribute, or the automatic SERIALNO generation option must be switched on.

If the MCP archive structure does not exist for the family on which the file resides, it is created by the MCP.

The backup references for the file may be displayed using the LFILES <filename> : ARC command, or using a PD command on the ODT, or by using the Metalogic FLEX utility.

A file with a backup reference may be removed from the Disk, but the backup reference is retained. The backup reference can be purged from the archive using the ARCHIVE PURGE statement of WFL.

A file with a backup reference may be released from the Disk using the ARCHIVE RELEASE statement of WFL. A file which has been released is a candidate for an AUTORESTORE, whereas a file which has been removed is not. The file retains the backup reference in any case. An LFILES : ARC shows whether a file was released or removed. The AUTORESTORE feature is not currently implemented for Disk Farms.

Copy from NULL

The Copy from NULL syntax is used to release files from a Disk Farm.

The syntax is,

   ADD <file list> FROM NULL TO <disk farm volume>

The <file list> may only consist of filenames, and may not include directories.

CopyWrite on NT finds the Capsule for each file in the <file list>, and if it has not already been released, it replaces the Capsule with a 'Released Capsule'.

A 'Released Capsule' is a valid Library/Maintenance Capsule, with a Tape Directory, but the Disk File Header is replaced by a 'Missing Header'.

A 'Missing Header' is a Library/Maintenance Block Control Word with the LMBMissF Flag set, and when copied, Library/Maintenance reports the file as 'Not On' the Volume.

Additional information about the released file is stored in the 'Released Capsule'.

A 'Released Capsule' is marked by this Icon

At the completion of the Copy to NULL, CopyWrite checks all Capsules in the Disk Farm, and if they are all 'Released Capsules', then the Disk Farm Volume is deleted.

A special syntax is available for releasing all the files in a Disk Farm, and deleting the Disk Farm itself.

The syntax is,

 ADD *= FROM NULL TO <disk farm volume>

In this example, the Disk Farm F is released,

Add *= From Null to F(Pack,Hostname=WARATAH)
 #RUNNING 9170
 #BOT 9171  (NFT)WFLCODE
 #BOT 9172  (NFT)FILE/TRANSFER
 #BOT 9173  (NFT)FILE/TRANSFER/SERVICES
 #9173 Metalogic CopyWrite Version 50.500.01 was compiled at 18:42:3 on the 6th
 May 2004 (MCP 48.189.9445)
 #8260\9175 BOT  (NFT)SESSION/"Mix# 9173"/"192.168.20.2"/WARATAH/NFT/COPYWRITE
 #9173 Release ALL 12 FILES in F
 #9173 File Release Completed. Volume Release Check Pending
 #9175 [INSPIRON] C:\ASeriesBackup\F\*.* Deleted
 #9175 [INSPIRON] C:\ASeriesBackup\F Deleted
 #8260\9175 EOT  (NFT) (NFT)SESSION/Mix# 9173/192.168.20.2/WARATAH/NFT/COPYWRITE
 #EOT 9173  (NFT) (NFT)FILE/TRANSFER/SERVICES
 #EOT 9172  (NFT) (NFT)FILE/TRANSFER
 #EOT 9171  (NFT) (NFT)WFLCODE

Copy to NULL

The Copy to NULL feature can be used to verify the files stored on a Disk Farm.

The syntax is,

COPY <file list> FROM <disk farm volume>(PACK,HOSTNAME=<hostname>) TO NULL(CD)

The files are copied from the Disk Farm and then discarded.

Recreating a LIBMAINTDIRECTORY

A LibMaintDirectory for a Disk Farm can be recreated on an MCP System, using the Copy To NULL feature.

If a Copy to NULL is performed, and LIBMAINTDIR is specified, a SERIALNO must be assigned to the <destination volume>.

If the SERIALNO is found in the Volume Library, the LIBMAINTDIR is created using the Volume Name, SerialNo and Creation Date of the Volume Library entry.

If not found, a message is displayed and the LIBMAINTDIR is created for a Volume called NULL.

Merging Disk Farms

Disk Farms which reside on a Windows NT/2000 Server can be merged using a COPY Statement which is issued from an MCP System.

The syntax is,

   COPY <file list> FROM <source volume>(PACK,HOSTNAME=<farm hostname>),
 <file list> FROM <source volume>(PACK,HOSTNAME=<farm hostname>
 TO <destination volume>(PACK,HOSTNAME=<farm hostname>

The Files are copied from each of the <source volumes> at the remote host into the <destination volume> at the remote host. No changes are made to the <source volumes>.

The <farm hostname> must be the same for all volumes.

After each file has been successfully transferred into the <destination volume>, the Disk File Header is transferred to the MCP System, and a message like this is displayed,

  		#5166 <1> (NFT)A recorded

If the LIBMAINTDIR Modifier is specified, then the <destination volume> must have a SERIALNO specified. When the Disk File Header is received, it is used to create an entry in a LibMaintDir.

The LibMaintDir has a title in this form,

LIBMAINTDIR/<destination volume>/<date>/<serialno> ON <catalog family>

If the &BACKUP Modifier is specified, then the <destination volume> must have a SERIALNO specified. When the Disk File Header is received, it is used to create a Backup Reference for the Copied File, which must reside on the same Family as the Primary Family of the Copy Task.

When the file is catalogued, a message like this is displayed,

			#5179 <1> (NFT)A cataloged on DEV

If &BACKUP is specified, then the SINGLEUNIT attribute may be set on the <destination volume>. If SINGLEUNIT is specified, then both Backup References for each file are set to the same volume.

If an ADD&BACKUP is performed, then there must be a matching Catalog entry, or else a Backup Reference will not be created. MERGETAPE uses this option as there must be a matching Catalog entry.

A COPY&BACKUP will create a Catalog entry that matches the file, if necessary by creating a file, entering it into the Catalog, making a Backup Reference, and then removing the File.

If the &CATALOG Modifier is specified, then the <destination volume> must have a SERIALNO specified. When the Disk File Header is received, it is used to create an Archive Reference for the Copied File, which must reside on the same Family as the Primary Family of the Copy Task.

When the file is archived, a message like this is displayed,

			#5179 <1> (NFT)A archived on DEV

If &CATALOG is specified, then the SINGLEUNIT attribute may be set on the <destination volume>. If SINGLEUNIT is specified, then both Archive References for each file are set to the same volume.

Structured Disk Farms

In a common Disk Farm, files are copied into Capsules which reside under the Windows Volume directory.

In a Structured Disk Farm, files are copied into Capsules which reside within a File Heirarchy that corresponds to the structure of the Titles of the files copied. The root of the File Heirarchy is the Windows Volume directory.

In this example, the files (U)A and (U)A/B were copied to a volume called STRUCT

The directory remains under the Windows Volume and the Capsules retain their ordinal file number even though they are distributed through the folder heirarchy.

When a Structured Disk Farm is created the Directory in FILE000.dir is marked, so that any subsequent copies to or from the Volume can find the Capsules.

A Structured Disk Farm is created using a COPY command and specifying SCRATCHPOOL=STRUCTURED on the destination volume.

These are some of the advantages of a Structured Disk Farm.

If multiple files exist with the same Title, then all the capsules are stored within the same Folder.

If a Capsule Script is used to process the contents of each Capsule as it is created, then generated files can be stored along with the Capsule.

Scripting with Capsules

When a Capsule is created in a Disk Farm, an associated VBScript can be executed.

If the FILE SCRIPT(NOTE="<windows title>") is Label Equated to the COPY command and the <windows title> references a resident file, then the <windows title> is stored in the Farm.ini configuration file section [Capsule] Script = <windows title>.

When a Capsule has been created and the Title added to the Directory, the Farm.ini file is checked for the presence of the [Capsule] Script=<windows title>, and if specified, the Script is executed.

The Script is provided with two predefined objects.

The NFTScript Object contains information about the Script and its environment (see NFTScript). In this case, the ScriptFileNo attribute is the FileNo of the Capsule.

The Selector Object contains information about the Capsule (see SelectorX).

In this example, a Script called C:\Temp\Test.vbs is being associated with a Disk Farm being created with a COPY statement under CANDE.

  WFL Task T;T(File Script(Note="C:\Temp\Test.vbs"));
	Copy A/= To AVOL(Pack,Hostname=Cpnta,Scratchpool=Structured)[T]

This is the VBScript.

'  ***** The Selector references the Capsule *****
   Spout("There are " & Selector.FileCount & " Files in the Capsule")
   Spout("Capsule Path is " & NFTScript.VolumePath)
 '  ***** Select the Files *****
 For Each File In Selector 
 Spout(File.FileName)
 FileName=NFTScript.VolumePath & "\" & File.FlatName & ".txt"
 Spout(FileName)
 If Not Selector.ConvertToText(File.FileNo,FileName,TRUE,Reason) then
 Spout(Reason)
 End If
 Next

Since the title of the Script is stored in Farm.ini, subsequent COPY and ADD commands to the Disk Farm will use the same Script.

If a later COPY or ADD command specifies a different Script, then the Farm.ini is changed to refelect the new Script.

Scripting with Capsules can be used with both common and Structured Disk Farms.

Automatic SERIALNO Generation

A COPY&CATALOG requires a SERIALNO to be specified as a <destination volume> attribute. The SERIALNO attribute is stored in the archive entry for the file, along with the Volume and HostName.

A SERIALNO can be automatically generated by setting the MAGUS Configuration variable called COPYWRITE_COPYCATSN. MAGUS Configuration Variables may be set by U META/INSTALL <variable name>=<value>.

The value of this variable must be in the form <prefix><number>, in which the <number> is incremented to form the next SERIALNO. It must be 6 characters long.

For example, U META/INSTALL COPYWRITE_COPYCATSN=DF0001. The first generated SERIALNO is DF0001, the next DF0002 and so on.

Summary of Registry Keys

The Disk Farm Root Directory is maintained in the Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\Services\NFT key propertyDLRoot.

A Shared Volume Disk Farm has the Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\Services\NFT key property SharedByAll set to 1.

The Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\Services\NFT key property Compression is set to 1 to enable compression.

The Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\RemoteUsers contains a key for each valid user, and may also have a key for *ANYUSER . The ValidHostList property of each valid user contains a list of hostnames, and may also have a hostname called *ANYHOST .

Windows Explorer Extensions

File Extensions

The following file extensions are associated with CopyWrite during installation. When a file with one of these extensions is selected, Explorer executes the Volume Directory Utility.

.dir	Disk Farm Directory
.mcp	Disk File Capsule
.con	MCP Wrapped File or Container
.asd	A-Series Pseudo Disk

Tips

Tips are available with Windows 2000 and later Explorer. When the mouse hovers over a file with a CopyWrite File Extension, a Tip (also known as a Hint), is displayed.
If a file has a File Extension of .mcp, and it contains one file, the Tip displays the MCP TITLE, FILEKIND and some other file attributes. If the file contains more than one file, then as much of the file directory as possible is displayed.
If a file has a File Extension of .con, as much of the file directory as possible is displayed.
If a file has a File Extension of .asd, then details of the Disk label are displayed.

File Details

In Windows 2000 Explorer, the Details view of a file folder provides additional columns for Disk File Capsules which have a file extension of '.mcp'. The View | Choose Columns... menu selection lists A-Series file attributes such as TITLE and FILEKIND, which can be selected and displayed.

Icons

In Windows NT and 2000 Explorer, an icon is associated with the class of file which the capsule contains. These classes are defined,

Icon	File Class	Definition
	Objects	Object files. (FileKind >= ALGOLCODE) AND (FileKind <= CODEFILE), except for self extracting archives.
	Symbols	Symbol files. (FileKind >= ALGOLSYMBOL) AND (FileKind <= FILEKIND_127)
	DMS	DMSII Files. FileKinds DBRESTARTSET, DBDATA.
	Wrapped	FileKinds WRAPPEDDATA and CONTAINERDATA.
	CD	FileKind PROMBURNERDATA.
	System	System Files. (FileKind >= NULLFILE) AND (FileKind <= MCPCODEFILE), except for print files.
	Archive	FileKind ESPOLCODE.
	Stream	Files with a FileStructure of STREAM, and a FileKind as for Data.
	Data	Record Data Files. FileKinds DATA, SEQDATAV, CDATA, TEXTDATA.
	Missing	A Valid capsule, but Library/Maintenance marked the header as missing, for example, a file which was OF'd.
	Bad	Not a capsule, but has an .mcp suffix.
	Printer	FileKinds BACKUPPRINTER and BACKUPPUNCH.

Context Menus

A context menu is displayed by clicking the right mouse button, after having selected a file in Explorer. The menu is constructed depending on the context, which is given by the file extension or some other information. The Directory commands are displayed in the menu when the file has a '.dir' extension, while the Capsule Commands are displayed when the file has an extension of '.mcp'.

Directory Commands

These commands are available in the Directory context.

List Directory	This command lists the files in the Disk Farm Directory. The Directory is maintained in the file called FILE000.dir. The codefile is ListDirectory.exe.
Verify Disk Farm	This command may be used to verify the Disk Farm. It checks that the files in the Directory exist in their corresponding capsules, and it verifies each capsule, checking the volume structure and checking the CheckSums for each Library/Maintenance block. The codefile is VerifyDiskFarm.exe.

Capsule Commands

These commands are available in the Capsule context.

Extract File	This command extracts the file in the capsule and converts it to Windows format. The codefile is CapsuleContents.exe.
File Attributes	This command displays the Disk File Header attributes for the file in the capsule. If an attribute is 0 and has no associated comment, it is not displayed. The codefile is CapsuleAttributes.exe.
Verify Capsule	This command verifies a capsule, checking the volume structures and the CheckSum for each Library/Maintenance block. The codefile is CapsuleVerify.exe.
Annotate Capsule	This command may be used to attach text to a capsule. The text is stored as a Windows NT alternate stream, and is therefore only available on the Windows NT File System (NTFS). The annotation is displayed in the Tip on Windows 2000. The codefile is CapsuleAnnotate.exe.

Diagnostic Options

Warning: Use of the diagnostic options can severely affect the stability of the Windows 2000 system.

Recovering a Disk Farm

The integrity of a Windows Volume can be checked by using the CopyWrite NFT | NFT | Verify Disk Farm function. This function verifies the Tape Directory and all the Disk File Capsules, and produces a report on the state of the Windows Volume.

It is possible for a Disk Farm to be left in a damaged state when either the Windows 2000 or A-Series system fails during a Copy operation.

This section hopes to provide sufficient information for the state of the Disk Farm to be determined, and to suggest some remedial action.

If a Windows Volume does not have a FILE000.dir, then the Windows Volume did not get created properly. The Windows Volume should be removed, and the A-Series Copy restarted.

Otherwise, use CopyWrite to open the Disk Farm directory, and check what files should be in the Windows Volume, and what is the highest file number.

Check that a FILEnnnnnnn.mcp Capsule with the highest file number exists, and that there are capsules for all file numbers from 1 up to that highest file number. If for some reason there is a file number missing, it is possible to substitute that capsule with a 'Missing File Capsule'.

A 'Missing File Capsule' is a capsule which has a Disk File Header with the LMBMissF flag set in the Library/Maintenance block word. When this Disk File Header is passed to Library/Maintenance it skips the file. This happens on the A-Series, for example, during a Copy when an I/O error occurs on a file.

To create a 'Missing File Capsule' use the CopyWrite NFT | NFT | Create Missing Capsule menu, which will require the A-Series File Title of the missing file. Give the Capsule the filename derived from it's ordinal position within the Tape Directory.

If there are any files called .tmp, they can be removed.

Starting and Stopping CopyWrite Bridge

The CopyWrite Bridge is a Windows 2000 Service.

As such, it can be started and stopped from the Control Panel.

It may also be started and stopped using the CopyWrite NFT | Bridge | Start and Stop menu. The NFT | Bridge | Status menu gives the current status of the service.

When the CopyWrite Bridge is stopped, there are no listening TCP/IP socket connections, and therefore any Copy statement started on the A-Series will fail with an OPENABORTBYCORRESPONDENTRSLT message.

Architecture

This brief description of the architecture is provided as an aid to the investigation and resolution of problems with the Disk Farm software.

The CopyWrite TCP/IP Bridge is a Windows NT Service. As such, it is under the control of the Service Control Manager (SCM), which provides a user interface for controlling services, in the Control Panel. The CopyWrite utility may also be used to start and stop the CopyWrite Bridge.

The CopyWrite Bridge runs as a separate process. When it starts it listens on Port 22256. When an incoming connection is received, the Bridge initiates a separate process called TCPIPDialog, passes it the socket, and then, once again, starts to listen for incoming connections.

TCPIPDialog handles all communication for a socket. It awaits a 'Connect Request' packet from the initiating host. When the 'Connect Request' is received, it checks that the user at the originating host is an authorized user of the Disk Farm, and then it locates the service, by searching the Registry keys.

When the service has been located, it is initiated as a separate process, and TCPIPDialog offers a named pipe. The service, such as NFTCopyWrite, then opens a corresponding named pipe and two way communication is established.

A packet arrives from the A-Series through a socket to TCPIPDialog, which then forwards the packet over the named pipe to the NFTCopyWrite service. The service responds, sending a packet over the named pipe to TCPIPDialog, which then sends the packet through the socket back to the A-Series.

All packets sent to and received from the A-Series have an A-Series CheckSum to verify the integrity of the packet during transmission.

The CopyWrite Bridge, the TCPIPDialog and the NFTCopyWrite service are all separate processes with their own address space. This means that when they terminate, all resources are returned to the system.

Only the CopyWrite Bridge remains permanently in memory, listening for incoming connections.

When files are copied to a Windows Volume, the path to the volume is used as the name of an inter process lock called a Mutex. If another copy to the same volume is attempted, it will wait for the lock to become available. Since there is a 60 second timeout on packets being acknowledge across the network, this means that normally subsequent copies will fail to acquire the lock, and will need to be retried. Multiple users may copy from the same Windows Volume, however, an attempt to copy the same file will fail because the capsules are currently opened for exclusive use. This may be remedied in a future release.

Trouble Shooting Guide

Symptom	Possible Cause	Action
The A-Series COPY statement fails with the message "Open failed because OPENABORTBYCORRESPONDENTRSLT" or "Open failed because UNAVAILABLEHOSTRSLT"	The CopyWrite Bridge is not installed or is not running on the Windows 2000 remote host.	Use the CopyWrite NFT \| Bridge \| Start menu to initiate the service. If CopyWrite Bridge fails to start, check the log using the Start Programs \| Administrative Tools \| Event Viewer menu.
The A-Series COPY statement fails with the message "Open failed because NAMESERVICENOTAVAILABLERSLT"	The HOSTNAME specified in the COPY statement is misspelled, or does not have a TCPIP MAPPING defined.	Add a mapping NW TCPIP MAPPING + <hostname> <IP address> For example, NW TCPIP MAPPING + CPNTA 192.168.1.1
NFT is not Available at <hostname>: All pipe instances are busy (Erc 231 4" E7000000")	The pipe between the Dialog process and NFT is being used, but there are no instances on offer.	This should only occur if the Testing property is set to 1 in the Registry key (HKEY_LOCAL_MACHINE\ Software\ Metalogic\ CopyWrite\ Services\ NFT) . If this has not been set, then submit a fault report.
The A-Series COPY statement fails with the message "Files may be added to <volume name>, but not copied, as a Volume called <volume name> already exists"	A Windows Volume with the same name as the <destination volume name> in the COPY statement already exists.	Use the ADD or ADD&COMPARE statement to add files to an existing Windows Volume. Remove the Windows Volume.
CreateFileMapping: Insufficient system resources exist to complete the requested service (Erc 1450 4"AA050000")	The Windows Volume may have run out of disk space.
Unable to install a new version of NFTCopyWrite.exe because there is already an instance of it running, but there are no active COPY statements.	Software problem	A program called Discontinue.exe is provided in the release directory. This is a console program which can terminate a process running under the local system account, provided the 'Debug Program' privilege is set for the logged in user. This privilege is set by default for the Administrator. Use the Windows NT Task Manager to find the PID (a numeric Process ID) for NFTCopyWrite, then execute C:>Discontinue <PID>. This program can terminate any process, including those essential to Windows NT, and so should be used carefully.

Glossary

Capsule	Each A-Series file copied to the Disk Farm is enclosed in a Capsule. The Capsule is in the same format as a Library/Maintenance CD Image. Each Capsule contains one file. It contains a Tape Directory which references only that file, and it contains the Disk File Header and the Data for the file.
Container	A container of A-Series native files, retaining both the Disk File Header information and the Data. There are several formats of container, for example Wrapped Files and Library/Maintenance CD Images.
<destination volume name>	Part of the syntax of a Library/Maintenance WFL COPY statement. For Example, COPY *= TO <destination volume name>(PACK). See WFL Reference Manual for more information.
Disk Farm	A Disk Farm is a disk storage system which allows files to be stored and retrieved from logical volumes. A Private Disk Farm isolates the logical volumes of one host from another. A Shared Disk Farm allows logical volumes to be shared amongst hosts.
DLRoot	An attribute of the Disk Farm on the Windows NT system, which specifies the path of the Disk Farm, for example, C\ASeriesBackup. C is the drive, and \ASeriesBackup is the directory under which VOLUMES are created.
Junction Point	The name of a directory under the DLRoot which has an NT directory mounted on it. This provides redirection of the volume to another location.
Missing File Capsule	A 'Missing File Capsule' is a capsule which has a Disk File Header with the LMBMissF flag set in the Library/Maintenance block word. When this Disk File Header is passed to Library/Maintenance it skips the file.
Mount Point	The name of a directory under the DLRoot, which has an NT logical volume mounted on it. For example, when using DirectCD, the CD Drive is mounted onto the Disk Farm, so that any files copied from the A-Series to that Volume are copied directly to the root directory of the CD or DVD. Any volume which supports memory-mapped files may be mounted at a Mount Point, not just NTFS volumes.
NTFS	Windows NT File System. This file system is provided on Windows NT and Windows 2000.
Volume	The <destination volume name> specified for a COPY statement. COPY X TO FRED(DISK). FRED is the <destination volume name> and it is used to create a directory on the Disk Farm.
Windows Volume	The name of the directory on the Windows system which contains the Farm Directory (FILE000.dir) and the Files (FILExxxxxx.mcp) comprising the volume. For example C:\ASeriesBackup\ERICTHEHALFABEE.

Remote ODT

The Remote ODT process interprets a command which was entered at an A-Series through the ODT or MARC 'AT' Directive, and returns a response.

The ODT Command text is translated from EBCDIC to ASCII.

The first word of the ODT Command text is used as the command key.

The parameters which modify how the Remote ODT process behaves are maintained in the Registry HKEY_LOCAL_MACHINE\Software\Metalogic\CopyWrite\Services\ODT\Scriptskey.

The Path property defines the volume and directory in which the ODT scripts are stored. It is set by default to the directory where CopyWrite was installed.

The Language property defines a list of the Scripting Languages which may to be used. The default Scripting Language is Visual Basic Script, with the extension '.vbs'.

The command key is used to search for a file called <scripts path>\<command key>.<language>.

For example, given an installation path of C:\Program Files\Metalogic\CopyWrite and the ODT Command PDT and using Visual Basic Script, a search is made for C:\Program Files \Metalogic\CopyWrite\PDT.vbs.

If the script is found, the Scripting Engine and the Environmental Objects are created. The script is loaded into the engine and executed.

The Environmental Objects give the script access to the incoming Controller message, the Windows NT system and provide the mechanism for building the ODT Command response.