Dynamic Code Compilation

A program can generate Source Declaration and Statements, Compile them using the MCP entrypoint called DYNAMIC_CODE_HANDLER, and then Execute the Code referenced by the generated PCW.

MCP references are for MCP Version 5.0.

1 DYNAMIC CODE HANDLER

The caller passes Source Declarations and Statements, a reference to a PCW, and the name of an SL Library which exports a DYN_COMPILER function, to DYNAMIC_CODE_HANDLER.

DYNAMIC_CODE_HANDLER calls the DYN_COMPILER entrypoint in the SL Library, passing it the source Declarations and Statements, the Segment Address of the next code segment in the Dynamic Codefile, and the SDI of the next entry in the Segment Dictionary.

The DYN_COMPILER compiles the source Statements and Declarations, and returns AREAS of Code, additions to the Segment Dictionary and a PCW.

When the PCW is referenced, the code is fetched from the Dynamic Codefile and executed.

The entrypoint definition in the MCP is at 17922560,

INTEGER PROCEDURE DYNAMIC_CODE_HANDLER(SOURCE_DECS,SOURCE_STMTS,          
                                       ERRMSGS,PCWREF,                    
                                       LIBFUNCTION);                      
VALUE        PCWREF;         % REALLY BY NAME BUT ALLOWS MATCHING         
ARRAY        ERRMSGS,                                                     
             SOURCE_DECS,                                                 
             SOURCE_STMTS[*];                                             
EBCDIC ARRAY LIBFUNCTION[*];                                              
WORD         PCWREF;

DESCRIPTION

THIS PROCEDURE IS EXPORTED TO ALLOW CODE TO BE GENERATED DYNAMICALLY BY A RUNNING STACK. THE CODE IS GENERATED BY CALLING A COMPILER LIBRARY SPECIFIED IN THE PARAMETER LIBFUNCTION. THE GENERATED CODE IS STORED IN A TEMPORARY FILE AND WORDS (SEGMENT DESCRIPTORS,ETC.) WHICH REFER TO THIS CODE ARE ADDED TO THE D1 STACK. THE PCW SPECIFIED BY THE PARAMETER PCWREF IS UPDATED TO POINT AT THE NEW CODE.

PARAMETERS

SOURCE_DECS THE SOURCE DECLARATIONS TO BE COMPILED SOURCE_STMTS THE SOURCE STATEMENTS TO BE COMPILED ERRMSGS ERROR MESSAGES FROM DYN_COMPILER PCWREF SIRW TO THE PCW TO BE UPDATED LIBFUNCTION FUNCTION NAME OF THE COMPILER LIBRARY

RESULTS

THE PROCEDURE WILL RETURN A NEGATIVE VALUE FOR ANY ERROR OTHERWISE THE RESULT OF THE CALL ON THE DYN_COMPILER.

The DYNAMIC_CODE_HANDLER is exported from the MCP without restriction and can be called using either NEWP or Algol.

If importing the entrypoint in Algol, change WORD PCWREF to ANYTYPE PCWREF, and delete VALUE PCWREF, and pass the PROCEDURE to be compiled as the actual parameter.

The scheme of this function is:

Check the PCWREF is a reference to a PCW (IRW to a word with a PCW Tag). If it is not a valid reference, the program is terminated with the error message "DYNAMIC CODE PARAMETER MISMATCH";
Sets the SYNTAX_ONLY flag from BOOLEAN(SOURCE_DECS[0].[31:1]). This is the only MCP dependency on SOURCE_DECS.
SetLibAttributes copying LIBFUNCTION for SIZE(LIBFUNCTION[*]).
Link to the DYN_COMPILER entrypoint of the SL LIBFUNCTION Library
Check the linked Library Stack is marked as a Compiler. If it is not marked as a Compiler, the program is terminated with the error message "ILLEGAL COMPILER".
Lock the DYN_CODE_EVENT of the D[1] Stack SPIB to restrict the process family to a single compilation.
Checks if there is already a Dynamic Codefile associated with the D[1] Stack (SEGDYNCODEHEADERINDEX(D1_SNR) ^= 0) and retrieves the HDR.
If there is no Dynamic CodeFile associated with the D[1] Stack, one is created on the H/L Unit and the USERINFO attribute of the HDR is set to the BASERECORDS.
The BASERECORDS is the number of sectors in the running codefile (DSKEOFV) rounded up to the next Row boundary.
The running CodeFile, rounded up to the next Row boundary, combined with the Rows of the Dynamic Codefile create a Virtual CodeFile, the rows of which are physically disjoint but logically a single codefile.
Calls the DYN_COMPILER(SOURCE_DECS,SOURCE_STMTS,
                       DSKEOFV(HDR)+USERINFO(HDR),
                       D1_NEXT_SEGMENT(D1_SNR),
                       CODE,
                       D1_ADDITION,
                       PCW_EBC_ARRAY,
                        ERRMSGS)
Loops through the Rows of Code. The CODE array is declared CODE[0:NUMROWS*(SECTORSTOWORDS(CFROWSIZE)+1)-1], where NUMROWS=5 and CFROWSIZE=504.
Copies the code to the Rows and updates the Dynamic CodeFile Header. If the maximum number of Rows is exceeded, the program is terminated with the error message "EOF ON DYNAMIC CODEFILE".
Copies the D[1] Stack entries to the D[1] Stack, expanding the D[1] Stack if necessary. If the D[1] Stack cannot be expanded, the program is terminated with the error message "D1 STACK STRETCH FAILED".
Copies the PCW from PCW_EBC_ARRAY, and converts it to a GAMMA format PCW, then updates the original reference.
Calls INTRINSICHANDLER to fix up any intrinsics which were added to the D1 Stack.

The D[1] Stack.

The D[1] Stack being used is the one associated with the PCW passed in PCWREF.

TEMP_WORD:=REF_TO_SD_OF_AR(PCWREF);

The parameter to REF_TO_SD_OF_AR is a reference to the MSCW of the activation record, or any item within the activation record. In this case it is the reference to the PCW which will point to the dynamically generated code.

The result of REF_TO_SD_OF_AR is an SIRW to the MSCW of the Segment Dictionary.

D1_SNR:=SIRWSTKNO(TEMP_WORD);

Executing the Dynamic Code.

When the PCW is referenced, it touches the entry in the Segment Dictionary specified by the SDI. This entry was created by the DYN_COMPILER and added to the Segment Dictionary by the DYNAMIC_CODE_HANDLER.

The touch causes a TBIT interrupt, and the MCP attempts to find the code.

 
	REC := CFREC(HDR,ABSENTADDRESS.CODEINDEXF);                
 IF DYNCODEHDR:= (DYNAMIC_CODE_CAPABLE(SDSNR) CAND          
                  REC GEQ DSKEOFV(HDR))   THEN              
      BEGIN                                                 
      HDR:=HEADERDESC(SPIB[SDSNR,DYNCODEINFO].DYNCODEINXF); 
      REC:=*-USERINFO(HDR);                                 
      END;

The Dynamic CodeFile is a virtual extension to the end (DSKEOFV(HDR)) of the running codefile.

The MCP detects that the record being referenced by the Segment Descriptor is beyond the EOF of the running codefile, and that there is an associated Dynamic CodeFile, and loads the actual Code from the Dynamic CodeFile.

2. DYN_COMPILER.

The DYN_COMPILER is the name of the entrypoint in the SL Library given by the LIBFUNCTION parameter to the DYNAMIC_CODE_HANDLER.

This is the function which compiles the source Declarations and Statements, and returns either ERRMSGS, or AREAS of Code, additions to the Segment Dictionary and a PCW binding token.

It is declared in the MCP at 17926960,

INTEGER PROCEDURE DYN_COMPILER(SOURCE_DECS,SOURCE_STMTS,              
                               RECORD_INDEX,SEGMENT_INDEX,            
                               CODE,D1_ADDITION,PCW,ERRMSGS);         
VALUE   RECORD_INDEX,                                                 
        SEGMENT_INDEX;                                                
ARRAY   CODE[*],                                                      
        D1_ADDITION[*],                                               
        ERRMSGS[*],                                                   
        SOURCE_DECS[*],                                               
        SOURCE_STMTS[*];                                              
EBCDIC ARRAY PCW[*];                                                  
INTEGER RECORD_INDEX,                                                 
        SEGMENT_INDEX;                                                
LIBRARY DYN_COMPILER_LIB;

 PARAMETERS                                                         
       SOURCE_DECS      THE SOURCE DECLARATIONS TO BE COMPILED      
       SOURCE_STMTS     THE SOURCE STATEMENTS TO BE COMPILED        
       RECORD_INDEX     THE STARTING CODE FILE RECORD NUMBER        
       SEGMENT_INDEX    THE FIRST D1 LOCATION                       
       CODE             THE GENERATED CODE                          
       D1_ADDITION      THE WORDS TO BE ADDED TO D1 STACK          
       PCW              THE GENERATED PCW - NO TAG                  
       ERRMSGS          ERROR MESSAGES RETURNED                     
 RESULTS                                                            
      THE DYN_COMPILER WILL RETURN A VALUE                         
           < 0  COMPILER ERROR                                      
             0  COMPILE OK                                          
           > 0  SYNTAX ERROR COUNT

                                                             
DEFINE D1_ADD(N)           = REAL(PTR_D1_ADDITION+(6+N*8+2),6) #,     
       D1_ADD_TAG(N)       = REAL(PTR_D1_ADDITION+(6+N*8),1) #,

The MCP interprets BOOLEAN(SOURCE_DECS[0].[31:1]) as a flag to indicate a compile for syntax. Otherwise the layout of the SOURCE_DECS, SOURCE_STMTS and ERRMSGS is a matter of agreement between the application program and the dynamic compiler.

Adding Entries to the D[1] Stack.

The SEGMENT_INDEX is the SDI of the next entry to be added to the D[1] Stack.

The D1_ADDITION array is used to return the additional entries.

The number of entries returned is specified in D1_ADDITION[0].

Each entry is the same as an entry in a compiler generated skeleton Segment Dictionary in a codefile.

<tag byte><flag byte><segdesc>

It is 8 bytes. The first byte is used to set the Tag. The second byte is used for Binder information, and is not used by the DYNAMIC_CODE_HANDLER. The remaining 6 bytes are the Segment Descriptor Word contents.

The DYNAMIC_CODE_HANDLER steps through the new D[1] Stack entries at 17930160. D1_WORDS is defined to be D1_ADDITION[0].

 IF D1_WORDS GTR D1_AVAILABLE(D1_SNR) THEN                           
 BEGIN                                                               
     IF EXPANDD1STACK(D1_SNR,(D1_WORDS+20)) THEN                     
         DYNERR(MSRCAN53,FATAL,BADD1STRETCHV);                       
 END;                                                                
 FOR TEMP := 0 STEP 1 UNTIL D1_WORDS -1 DO                           
     WORDSTACK[D1_SNR,SEGDICTMSCW+D1_NEXT_SEGMENT(D1_SNR)+TEMP] :=   
         D1_ADD(TEMP) & D1_ADD_TAG(TEMP) TAG;                        
 D1_AVAILABLE(D1_SNR) := D1_AVAILABLE(D1_SNR) - D1_WORDS;            
 D1_NEXT_SEGMENT(D1_SNR) := D1_NEXT_SEGMENT(D1_SNR) + D1_WORDS;      
 WORDSTACK[D1_SNR,TOSCWD].TSCWDELTAF:= * + D1_WORDS;

The D1_ADD and D1_ADD_Tag are defined as,

DEFINE D1_ADD(N)           = REAL(PTR_D1_ADDITION+(6+N*8+2),6) #,     
       D1_ADD_TAG(N)       = REAL(PTR_D1_ADDITION+(6+N*8),1) #,

The PCW for the Code.

The DYNAMIC_CODE_HANDLER passes the DYN_COMPILER a Skeleton Version 0 PCW in the first 6 bytes of the PCW_EBC_ARRAY, containing only the Lex Level of the original PCWREF.

REPLACE PCW_EBC_ARRAY BY (0&(WORD VIA PCWREF).LLF PCW_SKEL0_LLF)  
    FOR 1 WORDS;

The generated PCW may be returned as either a Skeleton Version 0 binding token, or as a Skeleton Version 1 Gamma PCW, since CONVERT_TO_GAMMA_PCW_FORMAT checks the PCW_SKEL_VERF = [47:3] field.

The MCP converts the PCW binding token to the appropriate PCW format at 17938960,

% CONVERT UNTOUCHED PCW TO TOUCHED PCW                               
THEPCW := REAL(PCW_EBC_ARRAY,6);                                     
CONVERT_TO_GAMMA_PCW_FORMAT (THEPCW, D1_SNR, FALSE);                 
WORD VIA PCWREF := THEPCW;

The PCW Skeleton 1 layout is in the MCP at 1035885,

PCW_SKEL_VERF =[47: 3]#        % skeleton version          
                                                           
PCW_SKEL1_PIRF=[41:13]#                                    
PCW_SKEL1_PSRF=[28: 3]#                                    
PCW_SKEL1_CSF =[25: 1]#                                    
PCW_SKEL1_LLF =[23: 4]#                                    
PCW_SKEL_BIT18=[18: 1]#        % BINDER requires bit 18 = 0
PCW_SKEL1_SDIF=[15:16]#

The PCW Skeleton 0 layout is in the MCP at 1035915,

PCW_SKEL0_PSRF=[35: 3]#                                    
PCW_SKEL0_PIRF=[32:13]#                                    
PCW_SKEL0_CSF =[19: 1]#                                    
PCW_SKEL_BIT18=[18: 1]#        % BINDER requires bit 18 = 0
PCW_SKEL0_LLF =[17: 4]#                                    
PCW_SKEL0_D1F =[13: 1]#                                    
PCW_SKEL0_SDIF=[12:13]#

The PCW contents are returned by the DYN_COMPILER in the first 6 bytes of the PCW_EBC_ARRAY, and are stored by the DYNAMIC_CODE_HANDLER into the PCW specified by the PCWREF, leaving the PCW Tag of the original PCWREF intact.

The Code.

The DYN_COMPILER is passed the logical RECORD_INDEX (segment address) where the generated code will be stored in the Virtual Codefile, and an array for the code.

The Virtual CodeFile segment addresses consist of the sectors up to DSKEOFV of the running codefile, rounded up to the next Row boundary, followed by the sectors of the Dynamic CodeFile.

On the first call to DYN_COMPILER, the logical RECORD_INDEX MOD ROWSIZE = 0.

The CODE array can be used to return up to 5 Rows of Code. A Row is assumed to be 504 Sectors (3024 Words).

Each Row in the CODE array is prefixed by a Word (CODE_RECORDS(Row)), which contains a count of the number of sectors in use in that Row.

The DYNAMIC_CODE_HANDLER copies the code, starting at RECORD_INDEX MOD ROWSIZE of the first Row, for the number of sectors given by CODE_RECORDS until the end of the Row, then each subsequent Row.

The Code is written to the Dynamic Codefile starting at the RECORD_INDEX.

The copying of code from the array to the Dynamic CodeFile stops when a Row after the first Row has a CODE_RECORDS = 0.