Data Structures & Files
Information Gathering for a File Repository
  Table of Contents  v-16.01.01 - datrak01.htm 
  Introduction
  Gather non-Relational Data Info
  Application Analysis Tools
  Scan JCL, PROC's & Control Files
  LISTCAT & DUMPCAT Utility Programs
  Spread Sheet Information
  Micro Focus Catalog Extract
  Ongoing Growth of FPMASTER.DAT
  Online Access of FPMASTER.DAT
  Build a Catalog for Micro Focus
  Create Catalog Entries
  Catalog Entries for PDS's
  Catalog Entries for GDG's
  Catalog Entries for Sequential Files
  Catalog Entries for VSAM Data Sets
  Repository Record Structure
  Summary
  Software Agreement and Disclaimer
  Downloads and Links
  Current Server or Internet Access
  Internet Access Required
  Glossary of Terms
  Comments or Feedback
  Company Overview
The SimoTime Home Page 

Table of Contents Previous Section Next Section Introduction

Creating and maintaining an inventory of data files can be a challenge. The challenge is not with the degree of difficulty or complexity. The challenge is with the realization of the value, a commitment to the task and elevating the priority of doing a very boring task. However, once it is accomplished it becomes a very worthwhile repository of information that can be used in the planning, development, testing and production environments.

Information about the characteristic (or properties) of a Data File, Data Structure (such as a PDS or GDG) or VSAM Data Set is not always readily available. In today's world when application processing is being moved between systems and data is being transferred, shared, converted and compared an inventory of data files can be quite helpful.

This example will use a Windows System running Micro Focus Enterprise Studio to create a Key-Sequenced-Data-Set (or VSAM, KSDS) containing an inventory of Partitioned Data Sets, Generation Data Groups, Sequential Files and VSAM Data Sets. Once the inventory of data sets has been created it will be possible to use this information to build a catalog for Micro Focus Mainframe Express or Micro Focus Studio/Server environments.

Note:  This document is currently a Work in Progress


We have made a significant effort to ensure the documents and software technologies are correct and accurate. We reserve the right to make changes without notice at any time. The function delivered in this version is based upon the enhancement requests from a specific group of users. The intent is to provide changes as the need arises and in a timeframe that is dependent upon the availability of resources.

Copyright © 1987-2017
SimoTime Technologies and Services
All Rights Reserved

Table of Contents Previous Section Next Section Gather non-Relational Data Info

There are numerous ways to gather information about the existence of files and the file properties. This section will discuss various methods for collecting information about files used by an application.

1 Create a Spreadsheet for the Inventory of the Partitioned Data Sets (PDS's)
2 Create a Spreadsheet for the Inventory of the Generation Data Groups (GDG's)
3 Create a Spreadsheet for the Inventory of the Sequential Files
4 Create a Spreadsheet for the Inventory of the VSAM Data Sets
  Build Four Excel Spreadsheets for the Data Set Inventory and Associated Properties

Table of Contents Previous Section Next Section Application Analysis Tools

WIP2

Table of Contents Previous Section Next Section Scan JCL, PROC's & Control Files

WIP2

Table of Contents Previous Section Next Section LISTCAT & DUMPCAT Utility Programs

WIP2

@echo OFF
rem  *
rem  * ESA, Enterprise Server Administration
rem  *      Catalog List Utility
rem  *
rem  * CATGET01 identifies the CATALOG.DAT file that will be used as input
rem  * CATTXT01 identifies the ASCII/Text file to be used as output
rem  * CATCTL01 identifies the control file, if null then default to list
rem  *
     call ..\..\ENV1BASE
     set CATGET01=%BASECAT%\CATALOG.DAT
     set CATTXT01=%BaseSYS%\LOGS\SYSOUT_SYS1CATLIST.TXT
rem  *
rem  * By setting the CATCTL01 to a null value the catalog list program will
rem  * use the program defaults and produce a list file with the binary
rem  * values removed prior to writing to the text file.
     set CATCTL01=%BaseSYS%\PARMLIB\CATLIST1.CTL
rem  *
     echo *******************************************************SYS1CATLIST
     echo Starting JobName SYS1CATLIST, User is %USERNAME%
rem  *
     run CBLCATC2
     if not "%ERRORLEVEL%" == "0" goto EOJNOK
     echo The Catalog list will be displayed in a separate window...
     START NotePad %CATTXT01%
rem  *
:EOJAOK
     echo Finished JobName SYS1CATLIST, User is %USERNAME%
     goto END
rem  *
:EOJNOK
     echo ABENDING JobName SYS1CATLIST, User is %USERNAME%
rem  *
:END

WIP2

@echo OFF
rem  *
rem  * ESA, Enterprise Server Administration
rem  *      Catalog List Utility, Dump Format
rem  *      The Control file is created using the Windows "echo" command
rem  *
rem  * CATGET01 - identifies the CATALOG.DAT file that will be used as input
rem  * CATTXT01 - identifies the ASCII/Text file to be used as output.
rem  * CATCTL01 - identifies the control file
rem  *
     call ..\..\Env1BASE
     set CATGET01=%BASECAT%\CATALOG.DAT
     set CATTXT01=%BASESYS%\LOGS\SYSOUT_SYS1CATDUMP.TXT
     set CATCTL01=%BASESYS%\PARMLIB\CATDUMP1.CTL
rem  *
     echo *******************************************************SYS1CATDUMP
     echo * Starting JobName ezCATDUMP, User is %USERNAME%
rem  *
     echo * Create the control file CATCTL01 to have the Catalog List
     echo * be created in a Hexadecimal Dump format.
rem  *
     if exist %CATCTL01% erase %CATCTL01%
     echo *>%CATCTL01%
     echo /DUMP>>%CATCTL01%
     echo /NOSHOWDUMP>>%CATCTL01%
     echo *>>%CATCTL01%
     echo /FLAG-HDR=Y>>%CATCTL01%
     echo /FLAG-EBC=N>>%CATCTL01%
     echo /FLAG-HEX=Y>>%CATCTL01%
     echo /FLAG-ASC=Y>>%CATCTL01%
     echo *>>CATCTL01.TXT
rem  *
     echo * Post Catalog information to a file and display to a screen
     run CBLCATC2
     if not "%ERRORLEVEL%" == "0" goto EOJNOK
     echo * The Catalog list with dump format will be displayed in a separate window...
     START NotePad %CATTXT01%
rem  *
:EOJAOK
     echo Finished JobName SYS1CATDUMP, User is %USERNAME%
     goto END
rem  *
:EOJNOK
     echo ABENDING JobName SYS1CATDUMP, User is %USERNAME%
rem  *
:END

Table of Contents Previous Section Next Section Spread Sheet Information

The following is the column structure for an Excel spreadsheet to be used as a starting point to populate the catalog.

Column - Label Description
A - Level Number This should be a "1" for the primary file definition information. A zero (0) is used to define the row that is used for column headers.
B - DSN This is a text string with a maximum length of forty-four (44) characters. It is the fully-qualified MVS Data Set Name (DSN) or VSAM Cluster Name
C - DD Name This is a text string with a maximum length of eight (8) characters. It is the DD Name.
D - DSORG This is the Data Set Organization.
E - RECFM This is the Record Format.
F - lrecl-MIN This is a numeric string with a maximum length of five (5) digits. It is the minimum or average record length.
G - lrecl-MAX This is a numeric string with a maximum length of five (5) digits. It is the maximum record length.
H - KeyPos This is a numeric string with a maximum length of five (5) digits. It is the key starting position. The first position in a record is 1
I - KeyLen This is a numeric string with a maximum length of five (5) digits. It is for the length of the key.
J - VSAM Data Name This is a text string with a maximum length of forty-four (44) characters. It is the VSAM Data Name for KSDS and ESDS
K - VSAM Index Name This is a text string with a maximum length of forty-four (44) characters. It is the VSAM Index Name for KSDS
L - Override Job This is a text string with a maximum length of eight (8) characters. It is the JCL Member name that will be used to override the generated JCL that is used to create the catalog entry.
M - User Job 1 This is a text string with a maximum length of eight (8) characters. It is the JCL Member name for a job that will be submitted to the internal reader after the catalog creation process is complete.
N - User Job 2 This is a text string with a maximum length of eight (8) characters. It is the JCL Member name for a job that will be submitted to the internal reader after the previous steps are completed.
O - Extended S1 This is a text string with a maximum length of four (4) characters. The content varies.
If DSORG=PO then this field contains the extension of the PDSM that is used for a dynamic PDS.
P - Extended S2 This is a text string with a maximum length of four (4) characters. The content varies.
Q - Extended M1 This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1
R - Extended M2 This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1
S - Extended M3 This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1
T - Extended M4 This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1
U - Extended X1 This is a text string with a maximum length of 256 characters. It is the path to the directory that contains the Members for a Job Override, User Job 1 and/or User Job 2.
V - Extended X2 This is a text string with a maximum length of 256 characters. The content varies.
If DSORG=PO then this field contains the path to the directory that contains the PDS Members.
Note-1: The Extended M1 trhough M4 fields are sixteen (16) byte fields. The content varies.
If DSORG=GDG the following values are accepted:
 LIMIT(nnn) where nnn is the number of active generations
 SCRATCH or NOSCRATCH, SCRATCH will physically delete the generation, NOSCRATCH will uncatalog the latest generation
 NOEMPTY or EMPTY, This specifies, what action is to be taken when the GDG becomes full base on the number in the LIMIT.
  Format of an Excel Spread Sheet

Table of Contents Previous Section Next Section Micro Focus Catalog Extract

WIP2

@echo OFF
rem  * *******************************************************************
rem  *             SYS1CATXPORT.cmd - a Windows Command File             *
rem  *         This program is provided by SimoTime Technologies         *
rem  *           (C) Copyright 1987-2017 All Rights Reserved             *
rem  *             Web Site URL:   http://www.simotime.com               *
rem  *                   e-mail:   helpdesk@simotime.com                 *
rem  * *******************************************************************
rem  *
rem  * Text   - Enterprise Server, Catalog Export Utility
rem  * Author - SimoTime Technologies
rem  * Date   - January 24, 1996
rem  *
rem  * LONAP  - an acronym for a List Of Names And Properties.
rem  * CSV    - an acronym for Comma-Separated-Values.
rem  * FFL    - an acronym for Fixed-Field-Length.
rem  *
rem  * INPUT... Description
rem  * CATGET01 identifies the CATALOG.DAT file.
rem  *
rem  * OUTPUT.. Description
rem  * LONAPCSV identifies the ASCII/Text file,
rem  *          records are text strings of Comma-Separated-Values (CSV).
rem  * LONAPFFL identifies the Sequential file,
rem  *          records are data strings of Fixed-Field-Length (FFL).
rem  *
     set CmdName=SYS1CATXPORT
     call ..\..\Env1BASE
     call SIMONOTE "*******************************************************%CmdName%"
     call SIMONOTE "Starting CmdName %CmdName%, User is %USERNAME% "
rem  *
rem  *             Control/Monitor
rem  *            12345678/12345678
     set SIMOCNTL=YYYNNNNN/NNNNNNNN
     set CATGET01=%BASECAT%\CATALOG.DAT
     set LONAPCSV=%BASECAT%\Wrk1\SIMOTIME.UTIL.LONAPCSV.CSV
     set LONAPFFL=%BASECAT%\Wrk1\SIMOTIME.UTIL.LONAPFFL.DAT
rem  *
     run MFCATXPT
     if not "%ERRORLEVEL%" == "0" goto EOJNOK
     call SIMONOTE "The Catalog Export Info will be displayed in a separate window..."
     START NotePad %LONAPCSV%
rem  *
:EOJAOK
     call SIMONOTE "DataTake is %CATGET01%"
     call SIMONOTE "DataMake is %LONAPCSV%"
     call SIMONOTE "DataMake is %LONAPFFL%"
     call SIMONOTE "Finished CmdName %CmdName%, User is %USERNAME% "
     goto END
rem  *
:EOJNOK
     call SIMONOTE "ABENDING CmdName %CmdName%, User is %USERNAME% "
rem  *
:END
     if "%SIMOMODE%" == "PAUSE" PAUSE

Table of Contents Previous Section Next Section Ongoing Growth of FPMASTER.DAT

WIP2

Table of Contents Previous Section Next Section Online Access of FPMASTER.DAT

Link to Internet   Link to Server   Explore a CICS Sample Program for accessing the FPMASTER File. The FPMASTER File contains information (or properties) about the non-relational data structures used by the sample applications. The repository is a VSAM, Key-Sequenced-Data-Set (KSDS)

Table of Contents Previous Section Next Section Build a Catalog for Micro Focus

A suite of programs is provided that will automate and document the process for creating and populating a catalog for the Micro Focus environment. The following sections provide the detailed information about the various jobs and programs.

Table of Contents Previous Section Next Section Create Catalog Entries

Link to Internet   Link to Server   Explore How to Create Catalog Entries for non-Relational Data Files and Structures and document the process.

Table of Contents Previous Section Next Section Catalog Entries for PDS's

Link to Internet   Link to Server   Explore How to Create Catalog Entries for Partioned Data Sets (PDS's) and Document the Process. This suite of scripts and documentation is for a Micro Focus Enterprise Server running on a Windows System.

Table of Contents Previous Section Next Section Catalog Entries for GDG's

Link to Internet   Link to Server   Explore How to Create Catalog Entries for Generation Data Groups (GDG's) and Document the Process.

Table of Contents Previous Section Next Section Catalog Entries for Sequential Files

Link to Internet   Link to Server   Explore How to Create Catalog Entries for Sequential Files and Document the Process. This suite of scripts and documentation is for a Micro Focus Enterprise Server running on a Windows System.

Table of Contents Previous Section Next Section Catalog Entries for VSAM Data Sets

Link to Internet   Link to Server   Explore How to Create Catalog Entries for VSAM, Key-Sequenced-Data-Sets (KSDS's) and Document the Process.

Table of Contents Previous Section Next Section Repository Record Structure

The following (LONAPFFL.cpy) is the COBOL copy file that defines the record structure of fixed length fields for the repository of application properties or the APMASTER File that is a VSAM, key-Sequenced-Data-Set (or KSDS).

      *****************************************************************
      *               LONAPFFL.CPY - a COBOL Copy File                *
      *     Record Layout for the File Properties Repository          *
      *  The record stucture is a fixed length of 1,024 characters    *
      *         Copyright (C) 1987-2017 SimoTime Technologies         *
      *                     All Rights Reserved                       *
      *              Provided by SimoTime Technologies                *
      *        Our e-mail address is: helpdesk@simotime.com           *
      *     Also, visit our Web Site at http://www.simotime.com       *
      *****************************************************************
      *  The information that is imported into this format originates
      *  from an Excel spreadsheet. The spreadsheet is saved to an
      *  ASCII/Text File with a Comma-Separated-Values (or CSV) record
      *  structure.
      *
      *  LONAP is an acronym for a List Of Names And Properties.
      *  CSV   is an acronym for Comma-Separated-Values.
      *  FFL   is an acronym for Fixed-Field-Length.
      *
      *  A conversion program is used to convert the CSV File to a
      *  Record Sequential File with the following Fixed Field format.
      *
      *  The fields are all fixed length.
      *  Numeric fields are right-adjusted with leading zeroes.
      *  Alpha-numeric fields are left adjusted with trailing spaces.
      *
      *  Positions 73-80 of the following statements are used to
      *  identify the original columns used in the Excel spreadsheet.
      *
       01  LONAPFFL-REC.                                                Excel
           05  LONAPFFL-LEVEL-CHR.
               10  LONAPFFL-LEVEL-NBR      PIC 9(2).                    col-A
           05  LONAPFFL-FILE-INFO.
               10  LONAPFFL-DSN            PIC X(54).                   col-B
               10  LONAPFFL-DD-NAME        PIC X(17).                   col-C
               10  LONAPFFL-DSORG          PIC X(4).                    col-D
               10  LONAPFFL-RECFM          PIC X(4).                    col-E
               10  LONAPFFL-LRECL-MIN      PIC 9(5).                    col-F
               10  LONAPFFL-LRECL-MAX      PIC 9(5).                    col-G
               10  LONAPFFL-KEYPOS         PIC 9(5).                    col-H
               10  LONAPFFL-KEYLEN         PIC 9(5).                    col-I
               10  LONAPFFL-VSAM-DAT       PIC X(58).                   col-J
               10  LONAPFFL-VSAM-IDX       PIC X(58).                   col-K
           05  LONAPFFL-JOB-EXEC.
               10  LONAPFFL-JCL-OVR1       PIC X(8).                    col-L
               10  LONAPFFL-JCL-USR1       PIC X(8).                    col-M
               10  LONAPFFL-JCL-USR2       PIC X(8).                    col-N
           05  LONAP-EXTENDED-GROUP.
               10  LONAPFFL-EXTENDED-S1    PIC X(4).                    col-O
               10  LONAPFFL-EXTENDED-S2    PIC X(4).                    col-P
               10  LONAPFFL-EXTENDED-M1    PIC X(16).                   col-Q
               10  LONAPFFL-EXTENDED-M2    PIC X(16).                   col-R
               10  LONAPFFL-EXTENDED-M3    PIC X(16).                   col-S
               10  LONAPFFL-EXTENDED-M4    PIC X(16).                   col-T
               10  LONAPFFL-EXTENDED-X1    PIC X(256).                  col-U
               10  LONAPFFL-EXTENDED-X2    PIC X(256).                  col-V
           05  FILLER                      PIC X(199).
      *
      ***  LONAPFFL - End-of-Copy File - - - - - - - - - - - LONAPFFL *
      *****************************************************************
      *

Table of Contents Previous Section Next Section Summary

This example will use a Windows System running Micro Focus Enterprise Studio to create a Key-Sequenced-Data-Set (or VSAM, KSDS) containing an inventory of Partitioned Data Sets, Generation Data Groups, Sequential Files and VSAM Data Sets. This document may nbe used as a tutorial for new programmers or as a quick reference for experienced programmers.

In the world of programming there are many ways to solve a problem. This document and the links to other documents are intended to provide a greater awareness of the Data Management and Application Processing alternatives.

The documentation and software were developed and tested on systems that are configured for a SimoTime environment based on the hardware, operating systems, user requirements and security requirements. Therefore, adjustments may be needed to execute the jobs and programs when transferred to a system of a different architecture or configuration.

SimoTime Services has experience in moving or sharing data or application processing across a variety of systems. For additional information about SimoTime Services or Technologies please send an e-mail to: helpdesk@simotime.com or call 415 883-6565. We appreciate hearing from you.

Table of Contents Previous Section Next Section Software Agreement and Disclaimer

Permission to use, copy, modify and distribute this software, documentation or training material for any purpose requires a fee to be paid to SimoTime Technologies. Once the fee is received by SimoTime the latest version of the software, documentation or training material will be delivered and a license will be granted for use within an enterprise, provided the SimoTime copyright notice appear on all copies of the software. The SimoTime name or Logo may not be used in any advertising or publicity pertaining to the use of the software without the written permission of SimoTime Technologies.

SimoTime Technologies makes no warranty or representations about the suitability of the software, documentation or learning material for any purpose. It is provided "AS IS" without any expressed or implied warranty, including the implied warranties of merchantability, fitness for a particular purpose and non-infringement. SimoTime Technologies shall not be liable for any direct, indirect, special or consequential damages resulting from the loss of use, data or projects, whether in an action of contract or tort, arising out of or in connection with the use or performance of this software, documentation or training material.

Table of Contents Previous Section Next Section Downloads and Links

This section includes links to documents with additional information that are beyond the scope and purpose of this document. The first group of documents may be available from a local system or via an internet connection, the second group of documents will require an internet connection.

Note: A SimoTime License is required for the items to be made available on a local system or server.

Table of Contents Previous Section Next Section Current Server or Internet Access

The following links may be to the current server or to the Internet.

Note: The latest versions of the SimoTime Documents and Program Suites are available on the Internet and may be accessed using the Link to Internet icon. If a user has a SimoTime Enterprise License the Documents and Program Suites may be available on a local server and accessed using the Link to Server icon.

Link to Internet   Link to Server   Explore the JCL Connection for more examples of JCL functionality with programming techniques and sample code.

Link to Internet   Link to Server   Explore the COBOL Connection for more examples of COBOL programming techniques and sample code.

Link to Internet   Link to Server   Explore An Enterprise System Model that describes and demonstrates how Applications that were running on a Mainframe System and non-relational data that was located on the Mainframe System were copied and deployed in a Microsoft Windows environment with Micro Focus Enterprise Server.

Link to Internet   Link to Server   Explore The ASCII and EBCDIC Translation Tables. These tables are provided for individuals that need to better understand the bit structures and differences of the encoding formats.

Link to Internet   Link to Server   Explore The File Status Return Codes to interpret the results of accessing VSAM data sets and/or QSAM files.

Table of Contents Previous Section Next Section Internet Access Required

The following links will require an internet connect.

This suite of programs and documentation is available to download for review and evaluation purposes. Other uses will require a SimoTime Software License. Link to an Evaluation zPAK Option that includes the program members, documentation and control files.

A good place to start is The SimoTime Home Page for access to white papers, program examples and product information. This link requires an Internet Connection

Explore The Micro Focus Web Site for more information about products and services available from Micro Focus. This link requires an Internet Connection.

Table of Contents Previous Section Next Section Glossary of Terms

Link to Internet   Link to Server   Explore the Glossary of Terms for a list of terms and definitions used in this suite of documents and white papers.

Table of Contents Previous Section Next Section Comments or Feedback

This document was created and is copyrighted and maintained by SimoTime Technologies.

If you have any questions, suggestions, comments or feedback please call or send an e-mail to: helpdesk@simotime.com

We appreciate hearing from you.

Table of Contents Previous Section Next Section Company Overview

SimoTime Technologies was founded in 1987 and is a privately owned company. We specialize in the creation and deployment of business applications using new or existing technologies and services. We have a team of individuals that understand the broad range of technologies being used in today's environments. Our customers include small businesses using Internet technologies to corporations using very large mainframe systems.

Quite often, to reach larger markets or provide a higher level of service to existing customers it requires the newer Internet technologies to work in a complementary manner with existing corporate mainframe systems. We specialize in preparing applications and the associated data that are currently residing on a single platform to be distributed across a variety of platforms.

Preparing the application programs will require the transfer of source members that will be compiled and deployed on the target platform. The data will need to be transferred between the systems and may need to be converted and validated at various stages within the process. SimoTime has the technology, services and experience to assist in the application and data management tasks involved with doing business in a multi-system environment.

Whether you want to use the Internet to expand into new market segments or as a delivery vehicle for existing business functions or need assistance with converting non-relational data structures simply give us a call at 415 883-6565 or check the web site at http://www.simotime.com


Return-to-Top
Data File Information Gathering, Create and Leverage a File Properties Repository
Copyright © 1987-2017
SimoTime Technologies and Services
All Rights Reserved
When technology complements business
http://www.simotime.com