Application Migration When technology complements business   File Management & Data Integrity

Copyright © 1987-2008  SimoTime Enterprises, LLC  All Rights Reserved  http://www.simotime.com

Introduction
(Next) (Previous) (Table-of-Contents)

Ever since the second computer was introduced into the world the tasks involved with a System Upgrade or replacement, an Application Migration and Data File Management (data file sharing, transfer, conversion and validation or compare) have been technically challenging.

The term "Application Migration" may be defined as the moving of all the components of an application from an "existing system" to a "new system" and obtaining the same business results. The term "system" is used to refer to existing or new hardware, operating system software and sub-systems such as online transaction monitors or job schedulers. The system provides the foundation for the application to execute and deliver the critical business requirements.

If changes become necessary then identifying the "Application Change" tasks and making the changes after a successful migration would help differentiate "risk incurred through application change" vs. "risk incurred through application migration or system dependencies". Also, this process will identify and help to manage the "scope creep" that may occur during an "Application Migration" effort.

Maintaining Data Integrity during an application migration and data file conversion process should be the top priority. To succeed requires careful planning and analysis with constant monitoring of the data transfer, data sharing, data conversion and data validation processes.

This document discusses the "possibilities and considerations" aspects of an application migration and will focus on providing an overview of the data management process during a mainframe application migration effort.

Introduction, Application & Data Migration Process
(Next) (Previous) (Table-of-Contents)

The following is a list of the steps for an application migration effort.

When moving from the "application migration and unit testing" phase to the "application and system testing" phase the support requirements will change from a planned, proactive-oriented process to an on-demand, reactive-oriented process. During the migration-oriented phase we had hours or days to plan and resolve issues. As an alternative we could possibly bypass the issue (develop a temporary work around) until a fix was provided.

During the Testing phase individuals doing the testing will be reporting problems as they occur and will be expecting a quick response so they may continue with their testing effort. When a problem is identified in the testing phase it must be resolved quickly with a permanent, long-term fix (a work around or temporary fix with a follow-on permanent fix may invalidate the testing). If an issue is encountered that stops the testing from moving forward it must be resolved within a matter of minutes or hours. If the issue is not resolved in this timeframe it could result in a rescheduling of the testing phase (this could possible require a partial or complete synchronization of the data and other system resources).

Therefore, it is important to have the training for the support personnel completed and involved in the support process prior to starting the "application and system testing". This will provide the primary support required and offer an opportunity to gain hands-on experience with the application running in the new environment.

Introduction, Application & Data Characteristics
(Next) (Previous) (Table-of-Contents)

Many companies have successfully migrated business applications between Mainframe Systems and a Windows, UNIX or Linux Systems. Based on these successes companies are now motivated and actually pursuing the migration of additional applications and data between a Mainframe System and a Windows, UNIX or Linux System.

These additional applications continue to increase in size and complexity along with the transfer, share, convert and compare of data that is formatted in an ever-increasing variety of file structures, formats, encoding schemas, data bases and access methodologies.

The Mainframe System continues to evolve and customers are migrating applications that are an intermingling of reliable, twenty-year old programs and file structures combined with programs and data structures that incorporate the latest mainframe technologies.

Mainframe applications that are created using the COBOL programming language are referred to as "COBOL-oriented applications" in this document. Batch jobs require the use of JCL (Job Control Language) and on-line programs usually interface with a transaction system (for example CICS, Customer Information Control System) and screen handlers such as BMS (Basic Mapping Support).

Therefore, a mainframe application may be predominately COBOL but will have other source members such as JCL or BMS (hence the term COBOL-oriented-application vs. COBOL-application).

In addition, many years ago (prior to the release of the ANSI/85 standard and the release of IBM's COBOL/2 for the mainframe) the COBOL language did not have the functionality to meet some of the programming requirements needed to meet the business and system demands. The COBOL Compilers did not generate executable members that were finite-in-purpose and fast enough to meet the size and performance requirements. To solve these problems Mainframe Assembler programs and callable assembler routines were written and many are still in use today. As the mainframe has evolved the capability of Mainframe Assembler has evolved. Mainframe Assembler continues to be used to address "Systems-oriented" requirements resulting in the combined use of older and newer Mainframe Assembler technologies.

Consulting Engagements & Training Options
(Next) (Previous) (Table-of-Contents)

The following is an overview of some of the services and training options available for an application migration effort. The following is a list of the recommended consulting, services and training phases designed to minimize the risk and successfully complete an application and data migration effort.

Training will be addressed in the Analysis and Planning phase based on individual assignments and requirements. Each of the options is discussed in more detail in the following sections of this document.

When moving from the "application migration and unit testing" phase to the "application and system testing" phase the support requirements will change from a planned, proactive-oriented process to an on-demand, reactive-oriented process.

During the migration-oriented phase we had hours or days to plan and resolve issues. As an alternative we could possibly bypass the issue (develop a temporary work around) until a fix was provided. During the Testing phase individuals doing the testing will be reporting problems as they occur and will be expecting a quick response so they may continue with their testing effort. When a problem is identified in the testing phase it must be resolved quickly with a permanent, long-term fix (a work around or temporary fix with a follow-on permanent fix may invalidate the testing). If an issue is encountered that stops the testing from moving forward it must be resolved within a matter of minutes or hours. If the issue is not resolved in this timeframe it could result in a rescheduling of the testing phase (this could possible require a partial or complete synchronization of the data and other system resources).

System, Sub-System, Utility & Third-Party Software
(Next) (Previous) (Table-of-Contents)

Migrating an application from a mainframe requires lifting the source code for the application off of its current foundation of hardware (mainframe), system software (z/OS or VSE), sub-system (CICS) software and Data Base Management (DB/2) or File Management (VSAM or sequential files) software.

The source code for the application is then shifted to a new foundation of hardware (intel), system software (Windows), sub-system (Micro Focus ES/MTO with Net Express for development) and Data Base Management (MS/SQL) and File Management (Micro Focus for Indexed and Sequential files) software. The source code for the application is then compiled and we are almost ready for our first unit test.

At this point it is important to note that Micro Focus provides the following.

Refer to the Micro Focus documentation for more detail.

Third-Party software may be application oriented, sub-system oriented or utilitarian. Many software vendors now offer a Windows or Distributed version of the mainframe software packages. For example, a third-party package that manages and catalogs reports into a database and provides easy access by a business user would be sub-system oriented. A third-party package that provides a file copy function and is called from a JCL submission or command file execution would be utilitarian.

Security
(Next) (Previous) (Table-of-Contents)

For an "Application and Data Migration" project security requires a two-phased approach. The Micro Focus Enterprise Server, Mainframe Transaction Option (ES/MTO) provides sub-system level security in the Windows environment. To obtain system level security it will be necessary to use a combination of ES/MTO subsystem security and Active Directory Security provided by Microsoft. This is a separate discussion that is beyond the scope of this document.

Batch Job Scheduling and Job Monitoring
(Next) (Previous) (Table-of-Contents)

A number of software vendors including some of the following vendors now offer distributed (or Enterprise) products for Windows, UNIX and Linux (distributed) platforms. If the job scheduling requirements are relatively simple the Task Scheduler provided with Windows may satisfy the scheduling requirement.

The use of a Job Scheduler will require additional effort when Migrating an application to a distributed platform. This effort usually requires a replacement product (i.e. a vendor's distributed scheduler instead of the vendor's Mainframe scheduler) that will need to be installed and configured for the targeted distributed platform. It is important to note that many job schedulers provide additional capability beyond the simple scheduling of jobs. Jobs may be scheduled as follows.

A job scheduler may also do parameter substitution. For example, the use of the %argument syntax may be used as a substitution capability where the substitution value for the %argument is stored in an external table. This capability functions similar to the capability of JES to define and substitute values based on the &argument syntax to identify where the substitution is to take place. The difference is the value for the &argument is specified in the JCL stream and the value for %argument may be specified in an external table or database.

A job scheduler should be able to monitor job progress and take appropriate action based on certain conditions. A job scheduler can also track system usage based on user (or other criteria) that may be necessary for billing.

Report Management and Printing
(Next) (Previous) (Table-of-Contents)

This section will discuss Report Management Systems and Printing. Third Party Vendors supply sub-system or utilitarian software to extract printed reports from the mainframe output spool and place the reports into a repository for easy access by business users. Mobius Management Systems is one of the vendors that supply this type of software in both a mainframe and a distributed (i.e. Windows) environment.

A data string that is to be printed should not contain binary data (i.e. non-print characters). The binary characters may be the same value as a print control character that could result in the incorrect presentation of the data. For additional information about non-print character processing refer to the Additional Reference Materials section of this document.

Application Asset Migration
(Next) (Previous) (Table-of-Contents)

This section provides an overview of the tasks that will need careful planning in order to ensure a successful application asset migration (i.e. transfer and compilation) of the source code used to build the application. Most mainframe environments have implemented a Software Configuration Management (or SCM) system. The following is a partial list of SCM systems available on the mainframe.

Depending on the method used to transfer the source code between the Mainframe system and the Windows system it may be necessary to "check-out" a copy of the source members in the SCM to members in a Partitioned Data Set (or PDS). Once the source members are stored in a PDS they may be accessed as EBCDIC-encoded, record sequential files with 80 byte records. The files may be transferrred between the mainframe and the Windows system using the File Transfer Protocol (FTP).

Note: Source code members should not contain embedded hexadecimal characters. Refer to  http://www.SimoTime.com/utlhex01.htm for more information about embedded hexadecimal in source code.

Asset Migration, Prepare a Windows-oriented Environment for Application Assets
(Next) (Previous) (Table-of-Contents)

A variety of directory structures or configurations are available in the Windows environment. This section will describe a configuration that works well in the Windows environment and is complimentary to the Micro Focus environment and supports the concurrent existence of a development, test and production environment on a single Windows system or spread across muiltiple systems. The following is the sub-directory structure used by SimoTime for the samle programs. The primary directory is C:\SimoSAM1.

Note: The DataLibx directory or directories are defined in the Data Migration sections of this document.

Asset Migration, Transfer Source Members
(Next) (Previous) (Table-of-Contents)

The section briefly describes three of the popular methods for transferring source code between a Mainframe System and a Windows, Linuc or UNIX System.

A separate  Data File Transfer Document  is available on the Internet provides additional information about data file conversion technology.

Asset Migration, File Transfer using Micro Focus Mainframe Access (MFA)
(Next) (Previous) (Table-of-Contents)

When transferring source code from a Mainframe system to a Windows system running a Micro Focus sub-system this should be the preferred transfer method. Since this is a Micro Focus utility the file format conversion (Mainframe file format to Micro Focus file format) is done as part of the file transfer process. The transfer process may be performed on demand using the Graphical User Interface (GUI) with a point-and-click or drag-and-drop methodology. The transfer process may be scripted and automated using a Windows command file, a job scheduler and the MFA command line interface.

Asset Migration, File Transfer using File Transfer Protocol (FTP)
(Next) (Previous) (Table-of-Contents)

This file transfer methodology is usually available on Mainframe systems (both z/OS and VSE), Windows systems, UNIX Systems and Linux systems. Because it is readily available it is usually the default transfer technology.

FTP works quite well when transferring source members that are stored in a Partitioned Data Set (PDS) as sequential files with fixed-length records of eighty characters of printable text. FTP in ASCII mode will do the file transfer, the file content (i.e. EBCDIC and ASCII) and the file format (record sequential to line sequential) conversions.

Asset Migration, File Transfer using Shared or Removable Machine Readable Media
(Next) (Previous) (Table-of-Contents)

Since the introduction of high-speed networks and shared Disk Access Storage Devices (or DASD) the use of machine-readable media (or MRM) for file sharing or transferring has steadily increased. The actual hardware, software and network configurations may vary based on the type and volumes of data. A typical configuration would be a Storage Area Network (SAN) in a clustered environment over a network. As the data volumes increase the use of Fibre Channel Protocol (FCP) may be required. The challenge is to provide access to storage area network (SAN) connectivity that meets both the budgetary and performance requirements for interconnecting primary data centers to remote locations.

Asset Migration, Compile Source Members
(Next) (Previous) (Table-of-Contents)

WIP001...

Data File Migration
(Next) (Previous) (Table-of-Contents)

This section provides an overview of the tasks that will need careful planning in order to ensure a successful data migration (i.e. transfer and conversion) of the data files used by the application.

Data Migration, Prepare a Windows-oriented Environment for Data Files
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, Directory Structure for Windows Environment
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Transfer
(Next) (Previous) (Table-of-Contents)

WIP001...

A separate  Data File Transfer Document  is available on the Internet provides additional information about data file conversion technology.

Data Migration, File Transfer, FTP or File Transfer Protocol
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Transfer, MFA or Mainframe Access
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Transfer, MRM or Machine Readable Media
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Conversion
(Next) (Previous) (Table-of-Contents)

WIP001...

A separate  Data File Conversion Document  is available on the Internet provides additional information about data file conversion technology.

Data Migration, File Format Conversion
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Format Conversion, Sequential with Fixed Length Records
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Format Conversion, Sequential with Variable Length Records
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Format Conversion, KSDS to Micro Focus Key Sequenced
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Content Conversion
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Content Conversion, ASCII and EBCDIC
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, File Content Conversion, Maintain Numeric Integrity
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, Data File Comparison
(Next) (Previous) (Table-of-Contents)

WIP001...

A separate  Data File Compare Document  is available on the Internet provides additional information about data file conversion technology.

Data Migration, Data File Viewing & Editing
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Migration, Data File Formats (Micro Focus)
(Next) (Previous) (Table-of-Contents)

WIP001...

A separate  Document describing Micro Focus File Formats  is available on the Internet provides additional information about data file conversion technology.

Data File Migration Process (ongoing requirements)
(Next) (Previous) (Table-of-Contents)

This section provides an overview of the tasks that will need careful planning in order to ensure a ongoing data migration process(i.e. transfer and conversion) of the data files used by the application.

Data Conversion, Archived and/or Historical Data
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Conversion, AS/400 as a Client or Distributed System
(Next) (Previous) (Table-of-Contents)

If a communication or data sharing requirement with the AS/400 exist then it may be necessary to modify some parts of the application. The AS/400 is an EBCDIC-encoded architecture as is the Mainframe System. Therefore, data that is shared between the Mainframe and the AS/400 is compatible and does not require conversion. However, when moving to an ASCII-encoded environment such as Windows, Linux or UNIX it will be necessary to convert the data that is shared between the two system.

Data Conversion, Repeatable for Various Testing Cycles
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Conversion, Convert in Preparation for File Compare
(Next) (Previous) (Table-of-Contents)

WIP001...

Data Conversion, Process may be Reviewed & Audited after Completion
(Next) (Previous) (Table-of-Contents)

WIP001...

Summary
(Next) (Previous) (Table-of-Contents)

Before we can take these mainframe applications into a production environment the people that will be doing the day-to-day operations and the people that will be providing production support (at the system level or the application level) will need time to develop what we refer to as a minimum experience base. To develop this will require additional training and heavy involvement in the testing process and the support required during the testing process.

Involvement in the support (both problem determination and problem resolution) is key in order to:

Software Agreement and Disclaimer
(Next) (Previous) (Table-of-Contents)

Permission to use, copy, modify and distribute this software for any commercial purpose requires a fee to be paid to Simotime Enterprises. Once the fee is received by SimoTime the latest version of the software will be delivered and a license will be granted for use within an enterprise, provided the SimoTime copyright notice appear on all copies of the software. The SimoTime name or Logo may not be used in any advertising or publicity pertaining to the use of the software without the written permission of SimoTime Enterprises.

Permission to use, copy, modify and distribute this software for a non-commercial purpose and without fee is hereby granted, provided the SimoTime copyright notice appear on all copies of the software. The SimoTime name or Logo may not be used in any advertising or publicity pertaining to the use of the software without the written permission of SimoTime Enterprises.

SimoTime Enterprises makes no warranty or representations about the suitability of the software for any purpose. It is provided "AS IS" without any express or implied warranty, including the implied warranties of merchantability, fitness for a particular purpose and non-infringement. SimoTime Enterprises shall not be liable for any direct, indirect, special or consequential damages resulting from the loss of use, data or projects, whether in an action of contract or tort, arising out of or in connection with the use or performance of this software.

Additional Reference Materials
(Next) (Previous) (Table-of-Contents)

This section is intended to provide references to additional information for individuals that are migrating applications across systems or share data between systems. Many of the following references will require a connection to the Internet.

Additional Reference, Numeric Field Formats
(Next) (Previous) (Table-of-Contents)

The following items provide external links to reference material for various encoding formats used for numeric data strings.

1. This document describes the  Zoned-Decimal  format. This is coded in COBOL as USAGE IS DISPLAY and is the default format if the USAGE clause is missing.
2. This document describes the  BINARY or COMP  format. This is coded in COBOL as USAGE IS COMPUTATIONAL and is usually coded in its abbreviated form of COMP. This may also be coded with the keyword BINARY.
3. This document describes the  Packed-Decimal  format. This is coded in COBOL as USAGE IS COMPUTATIONAL-3 and is usually coded in its abbreviated form of COMP-3.
4. This document describes the  Edited Numeric  format. This is coded in COBOL using an edit mask in the picture clause. An example would be PIC ZZZ.99+.
5. This example with documentation describes the use of  Floating Point  data strings or fields.

Additional Reference, Data File Conversion
(Next) (Previous) (Table-of-Contents)

The following list provides external links to reference material and examples for data file conversion.

1. The  Data File Conversion section of the SimoTime Reference Manual  that is available on the Internet provides additional information about data file conversion technology.
2. An  ASCII/EBCDIC Table  with hex, decimal and binary values.

Additional Reference, Data File Comparison
(Next) (Previous) (Table-of-Contents)

The following list provides external links to reference material and examples for data file comparison.

1. The  Data File Comparison section of the SimoTime Reference Manual  is available on the Internet and provides additional information about data file comparison technology.
2. The  Data File Comparison Self-Study Course  provides additional information about using a generated COBOL program to compare two Micro Focus files in a Windows or UNIX environment. Since the generated COBOL program is OS/390 compliant it may be compiled and executed on an IBM Mainframe with ZOS or VSE.

Additional Reference, Data File Transfer
(Next) (Previous) (Table-of-Contents)

The following list provides external links to reference material and examples for data file transfer.

1. The following  white paper for Transferring Data Files  provides additional information about transferring files between a Mainframe and a Wintel platform.
2. The following is provided as a Reference for the File Transfer Protocol  commands. This document describes a typical process for an interactive and automated, batch session.

Additional Reference, Non-print or Non-display Characters
(Next) (Previous) (Table-of-Contents)

This link provides an example of  how to remove non-print and non-display characters  in a data string prior to printing or displaying.

Additional Reference, Vendors, Consultants and Service Providers
(Next) (Previous) (Table-of-Contents)

The following items provide external links to Internet sites for Software Vendors, Consulting Firms and Service Providers.

1. The  Internet Site for Micro Focus  provides a variety of information about their technology, services and training offerings.

Glossary of Terms
(Next) (Previous) (Table-of-Contents)

Check out  The SimoTime Glossary  for a list of terms and definitions used in the documents provided by SimoTime.

Comments or Suggestions
(Next) (Previous) (Table-of-Contents)

If you have any questions, suggestions or comments please call or send an e-mail to:  helpdesk@simotime.com  .

We appreciate your comments and feedback.

About SimoTime Enterprises, LLC
(Next) (Previous) (Table-of-Contents)

Founded in 1987, SimoTime Enterprises is a privately owned, Limited Liability Corporation located in Novato, California. We specialize in the creation and deployment of business applications using new or existing technologies and services. We have a team of individuals that understand the broad range of technologies being used in today's environments. This includes the smallest thin client using the Internet and the very large mainframe systems. There is more to making the Internet work for your company's business than just having a nice looking WEB site. It is about combining the latest technologies and existing technologies with practical business experience. It's about the business of doing business and looking good in the process. Quite often, to reach larger markets or provide a higher level of service to existing customers it requires the newer Internet technologies to work in a complimentary manner with existing corporate mainframe systems. Whether you want to use the Internet to expand into new market segments or as a delivery vehicle for existing business functions simply give us a call or check the web site at http://www.simotime.com