File Conversion & Formatting When technology complements business   Mainframe and PC

Copyright © 1987-2008  SimoTime Enterprises, LLC  All Rights Reserved http://www.simotime.com

Introduction
(Next) (Previous) (Table-of-Contents)

There are many options available when moving and converting data between an IBM Mainframe and a Windows or Unix environment. This document will discuss the cycle of how to convert a VSAM, KSDS to a flat Sequential file on the mainframe, download the flat sequential file from the mainframe to the PC and create an Indexed file of ASCII content on the PC.

A general guideline to follow is, "The scope of effort for converting a mainframe legacy file from EBCDIC to ASCII is directly proportional to the size of the file, the age of the file and the number and type of numeric fields." Estimating the time for conversion between EBCDIC and ASCII can be done with a high degree of accuracy. Estimating the time for "scrubbing the data" may be a challenging task.

Additional programs are included for converting ASCII, Text files to Indexed files. These examples also show how to manage Packed-Decimal and BINARY or COMP fields.

In the world of programming there are many ways to solve a problem. This suite of programs is provided as an example of one of the possible solutions to the problems of file conversion. The intent is to keep the programming effort simple and do the conversion as quickly as possible.

The programs have been tested on an IBM mainframe, Mainframe Express from Micro Focus and Net Express from Micro Focus. Sample projects may be obtainf for Mainframe Express and Net Express by sending an e-mail to: helpdesk@simotime.com

Note: The source code and documentation for this suite of examples is available from the SimoTime Library under Download Directory at   www.simotime.com

This is an example of how a COBOL program can read a VSAM Keyed Sequential Data Set (KSDS) with EBCDIC content and create a sequential file with EBCDIC content. The example then describes how to translate the EBCDIC content to ASCII and write an ASCII file. At first glance the conversion between EBCDIC and ASCII appears to be a simple and practical task. Typically the EBCDIC files reside on a mainframe legacy system. These files may be very large and may have been updated over a period on many years. In addition to containing displayable EBCDIC characters the records may contain binary and packed decimal fields that cannot simply be converted using an EBCDIC/ASCII table. Also, some of the fields in a record may contains low-values (i.e. x'00') that are being handle as a space characters on the mainframe. The low-values remain low-values when converted to ASCII and when downloaded or ported to the PC these low-values may be treated as null characters and this could present a problem if not addressed in the initial conversion process.

The COBOL source code for the conversions programs was generated using SimoZAPS, a product of SimoTime Enterprises. The SimoZAPS utility program also has the capability of reading, writing or converting between other file formats. In the world of programming there are many ways to solve a problem. This suite of programs is provided as a COBOL example of one of the possible solutions to the problem of changing the contents and structure of data files.

Naming Convention
(Next) (Previous) (Table-of-Contents)

The programs and data files used in this example use a file naming format that is eight (8) characters long with a three (3) character extension.

Naming Convention for Programs
(Next) (Previous) (Table-of-Contents)

The member names used in this suite of examples are eight (8) characters long with a three (3) character extension.

Naming Convention for Data Files
(Next) (Previous) (Table-of-Contents)

The data files used in this example are eight (8) characters long with a three (3) character extension.

Flowchart for the File Conversion Cycle
(Next) (Previous) (Table-of-Contents)

The following shows the file conversion cycle or process when transferring files between the host system (in this example the host system is an IBM mainframe) and the PC/Windows environment. This illustration also shows the process for converting between ASCII/Text files and Indexed files.

Host         Data Set SEQUENTIAL ebcdic SQEKSEC1 Read SEQUENTIAL Write VSAM, KSDS Data Set VSAM, KSDS ebcdic KSESQEC1 Read VSAM, KSDS Write SEQUENTIAL Data Set SEQUENTIAL ebcdic   Step-6       Step-1   FTP binary Step-5 Upload to Host       Download to Server Step-2 FTP binary   Step-4   Server   Step-3   Data Set SEQUENTIAL ebcdic KSASQEC1 Read INDEXED Write SEQUENTIAL Data Set INDEXED ascii SQEKSAC1 Read SEQUENTIAL Write INDEXED Data Set SEQUENTIAL ebcdic                         COPY lan             Step-A3     Step-A1         TXAKSAC1 Read ASCII/Text Write INDEXED Data Set INDEXED ascii KSATXAC1 Read INDEXED Write ASCII/Text           Local PC           Data Set Text File ascii NotePAD Read ASCII/Text Write ASCII/Text Data Set Text File ascii           Step-A2           Step-A6       Step-A4         CBLCSVC2 Read CSV File Write Text File       CBLCSVC1 Read Text File Write CSV File                       Data Set CSV File ascii Excel Import CSV File Save As CSV File Data Set CSV File ascii             Step-A5

Data Conversion & File Formatting, Generating the Source Code
(Next) (Previous) (Table-of-Contents)

This section contains a number of example programs to do data file conversion between EBCDIC and ASCII. Two of the programs (TXAKSAC1.CBL and KSATXAC1.CBL) illustrate a process for converting packed-decimal and COMP fields to and from character-oriented numeric fields.

Once the COBOL source code has been generated it needs to be compiled. The following three .BAT files are provided to set the environment variables, generate the programs and compile the programs.

Setting the Environment Variables
(Next) (Previous) (Table-of-Contents)

The following is the .BAT file used to set the environment variables

@echo off
echo ****************************************************************
echo *                This program is provided by:                  *
echo *                 SimoTime Enterprises, LLC                    *
echo *        (C) Copyright 1987-2003 All Rights Reserved           *
echo *          Web Site URL:   http://www.simotime.com             *
echo *                e-mail:   helpdesk@simotime.com               *
echo ****************************************************************
echo * %0 is STARTING...
echo * %0 is STARTING...  >XENVST01.LOG
echo * This process will set the envariables for the demoST01.BAT
echo *
     SET  COBCPY=.;..\
     SET  DATACNTL=d:\simonxe1\st01\datacntl
rem  *
     set path=d:\Net Express;d:\Net Express\base\bin;path=%path%
rem  *
     set ENV4ST01=YES
rem  *
echo * %0 path is %path%
echo *                             >>XENVST01.LOG
echo * %0 path is as follows...    >>XENVST01.LOG
echo *                             >>XENVST01.LOG
echo * %path%                      >>XENVST01.LOG
echo *                             >>XENVST01.LOG
rem  *
echo * %0 is COMPLETE...
echo * %0 is COMPLETE...           >>XENVST01.LOG

Generating COBOL Source Code
(Next) (Previous) (Table-of-Contents)

The following is the .Bat file for generating the COBOL source code.

@echo off
echo ****************************************************************
echo *                This program is provided by:                  *
echo *                 SimoTime Enterprises, LLC                    *
echo *        (C) Copyright 1987-2003 All Rights Reserved           *
echo *          Web Site URL:   http://www.simotime.com             *
echo *                e-mail:   helpdesk@simotime.com               *
echo ****************************************************************
echo * %0 is STARTING...
echo * %0 is STARTING...  >>XGENST01.LOG
echo * This batch file generates the COBOL source code for the Demo.
echo *
     if exist XGENST01.LOG erase XGENST01.LOG
     if "%ENV4ST01%" == "YES" goto :SKIPENV
        CALL zENVst01.bat
:SKIPENV
rem  *
     setlocal
     call :GENERATE KSASQEM1 KSASQEC1
     call :GENERATE KSATXAM1 KSATXAC1
     call :GENERATE KSESQEM1 KSESQEC1
     call :GENERATE SQEKSAM1 SQEKSAC1
     call :GENERATE SQEKSEM1 SQEKSEC1
     call :GENERATE SQEKSEM2 SQEKSEC2
     call :GENERATE TXAKSAM1 TXAKSAC1
     call :GENERATE TXAKSAM2 TXAKSAC2
rem  *
     dir  XGENST01.LOG
     type XGENST01.LOG
echo * %0 is COMPLETE...
echo * %0 is COMPLETE...  >>XGENST01.LOG
     goto :END

:GENERATE
     echo * %1 COBOL Generation is STARTING...
     if exist %1.cbl erase %1.cbl
rem  *
     call %1
     if exist %2.CBL echo * 0% %1 generation SUCCESSFUL...
     if exist %2.CBL echo * 0% %1 generation SUCCESSFUL...    >>XGENST01.LOG
     if exist %2.CBL goto :END
     echo * 0% %1 generation FAILURE!!!...                    >>XGENST01.LOG
:END

Compiling the COBOL Programs
(Next) (Previous) (Table-of-Contents)

The following is the .BAT file for compiling the COBOL programs.

@echo off
echo ****************************************************************
echo *                This program is provided by:                  *
echo *                 SimoTime Enterprises, LLC                    *
echo *        (C) Copyright 1987-2003 All Rights Reserved           *
echo *          Web Site URL:   http://www.simotime.com             *
echo *                e-mail:   helpdesk@simotime.com               *
echo ****************************************************************
echo * %0 is STARTING...
echo * %0 is STARTING...                                   >>XCBLST01.LOG
echo * This process will compile all the COBOL programs for the Demo.
echo *
     if exist XCBLST01.LOG erase XCBLST01.LOG
     if "%ENV4ST01%" == "YES" goto :SKIPENV
        CALL zENVst01.bat
:SKIPENV
rem  *
     setlocal
     call :COMPILE KSASQEC1
     call :COMPILE KSATXAC1
     call :COMPILE KSESQEC1
     call :COMPILE SQEKSAC1
     call :COMPILE SQEKSEC1
     call :COMPILE SQEKSEC2
     call :COMPILE TXAKSAC1
     call :COMPILE TXAKSAC2
rem  *
     dir  XCBLST01.LOG
     type XCBLST01.LOG
echo * %0 is COMPLETE...
echo * %0 is COMPLETE...                                   >>XCBLST01.LOG
     goto :END

:COMPILE
     echo * %0 %1 COBOL Compile is STARTING...
     if exist %1.idy erase %1.idy
     if exist %1.obj erase %1.obj
     if exist %1.dll erase %1.dll
     if exist %1.gnt erase %1.gnt
     if exist %1.int erase %1.int
rem  *
     cobol %1.CBL noanim omf(gnt) assign(external);
     if exist %1.GNT echo * %0 %1 compiled with NO ANIMATE...
     if exist %1.GNT echo * %0 %1 compiled with NO ANIMATE... >>XCBLST01.LOG
     if exist %1.GNT goto :END
     echo * %0 %1 Compile failure!!!                          >>XCBLST01.LOG
:END

Step-01, Read from VSAM EBCDIC and Write to QSAM EBCDIC
(Next) (Previous) (Table-of-Contents)

Read a VSAM Keyed Sequential Data Set (KSDS) with EBCDIC content and create a sequential file with EBCDIC content. This program is intended to run on a mainframe but will also run on a PC with Micro Focus Mainframe Express. This program does not convert the content of the records within the file. It simply creates a new file type (i.e. VSAM, KSDS to a flat sequential file).

The KSESQEx1.ext files are used to generate, compile and execute the file conversion programs.

The following flowchart is a quick overview of the process for generating the COBOL source code to do the file conversion and formating.

Step-02, Download Sequential EBCDIC file to PC or Server
(Next) (Previous) (Table-of-Contents)

Download the sequential file from the mainframe to the PC using FTP (File Transfer Protocol). Since the file contains EBCDIC characters, decimal-packed and COMP data it will be necessary to download in a BINARY format.

Note: Micro Focus provides a Mainframe Access (MFA)option with their Mainframe Express product. The MFA option may be used to download a sequential or VSAM file from the mainframe to the PC with a simple point-and-click or command line procedure.

Step-03, Read from Sequential EBCDIC and Write to ASCII Indexed
(Next) (Previous) (Table-of-Contents)

This program is intended to run on the PC using Micro Focus Net Express (it will also run on the mainframe and the PC with Mainframe Express). It will convert both the file type (i.e. Sequential to Indexed) and the content of the records within the file. Packed-decimal and COMP fields are copied as appropriate. Character fields are converted from EBCDIC to ASCII.

The SQEKSA?1.ext files are used to generate the file conversion program.

The following flowchart is a quick overview of the process for generating the COBOL source code to do the file conversion and formating.

Step-04, Read from ASCII Indexed and Write to Sequential EBCDIC
(Next) (Previous) (Table-of-Contents)

This program is intended to run on the PC using Micro Focus Net Express (it will also execute on the PC with Mainframe Express or the mainframe). It will convert both the file type (i.e. Indexed to Sequential) and the content of the records within the file. Packed-decimal and COMP fields are copied as appropriate. Character fields are converted from ASCII to EBCDIC.

The KSASQE?1.ext files are used to generate the file conversion program.

The following flowchart is a quick overview of the process for generating the COBOL source code to do the file conversion and formating.

Step-05, Upload Sequential EBCDIC to Host
(Next ) (Previous) (Table-of-Contents)

Upload the sequential file from the mainframe to the PC using FTP (File Transfer Protocol). Since the file contains EBCDIC character, decimal-packed and COMP data it will be necessary to upload in a BINARY format.

Step-06, Read from Sequential EBCDIC and Write to VSAM EBCDIC
(Next) (Previous) (Table-of-Contents)

Read a Sequential file with EBCDIC content and create a VSAM Keyed Sequential Data Set (KSDS) with EBCDIC content. This program is intended to run on a mainframe and will run on a PC with Micro Focus Mainframe Express (it will also execute on the PC with Net Express). This program does not convert the content of the records within the file. It simply copies the records to the new file type (i.e. flat sequential file to a VSAM, KSDS).

The SQEKSEx1.ext files are used to generate, compile and execute the file conversion programs.

The following flowchart is a quick overview of the process for generating the COBOL source code to do the file conversion and formating.

File Conversion between ASCII Text and ASCII Indexed
(Next) (Previous) (Table-of-Contents)

The following examples show how to convert between Indexed files and ASC/Text files. Since the COBOL programs contains the file definition of "ORGANIZATION IS LINE SEQUENTIAL" on the "SELECT" statement this program will not compile on the mainframe. The programs have been compiled and executed on the PC using Micro Focus Net Express.

Step-A1, Read from an ASCII/Indexed and Write to an ASCII/Text
(Next) (Previous) (Table-of-Contents)

This generated sample program will read an Indexed file with ASCII content and create a Text file with ASCII content. This example will also convert a packed-decimal (i.e. COMP-3) and BINARY (i.e. COMP) field to a printable numeric field.

The KSATXA?1.ext files are used to generate the file conversion program.

The following flowchart is a quick overview of the process for generating the COBOL source code to do the file conversion and formating.

Step-A2, Edit the ASCII/Text File using NotePAD
(Next) (Previous) (Table-of-Contents)

Th