Chapter N7. AS/400 and Linux Integration

Chapter N7. AS/400 and Linux Integration

*"Magic Software Enterprises is very excited to see IBM embrace Linux with its*
                                                                                            *support of the iSeries platform. Already, we have demonstrated how*
                                                                               *easy it is for our Magic eMerchant highly customizable e-business*
                                            *solution to run in an IBM Linux iSeries partition.*
*We expect we will be able to leverage our development technology*
                                                *and business solutions that run on Linux to*
                                                                       *realize greater business opportunities."*
*- David Leichner*
                                                                                                         *VP Worldwide Marketing*
                                                                                                                 *Magic Software*
Goals for this chapter: Introduction The AS/400 Integrated File System The TCP/IP communications Sharing files with NFS Terminal Emulation Database File to Stream File Copy Stream File to Database File Copy

Introduction

The IBM AS/400 and Linux have been very successful for many years and it is increasingly important to find the best ways to integrate their usage. It should be noticed that what is described below about Linux is generally applicable also to UNIX.

IBM has done a lot of work from version 3.1 of the OS/400 to improve the coexistence with the Unix and PC applications.

Recently IBM has also announced that Linux will be installed as an option on the AS/400 by using a separate Logical Partition (LPAR).

The following point will examine in detail how the recent changes make the integration of these two worlds easier.

The AS/400 Integrated File System

The Integrated File System has been built in the operating system after the version 3.1 to allow an easier coexistence between the AS400 native applications and stream file oriented systems such as Linux, UNIX, DOS and OS/2.

The traditional AS/400 approach consists of fixed format records stored in database files residing in libraries (single level containers with the exception of QSYS that contains all other libraries).

Normally each file (stored in the DB2 UDB database for the AS/400) has an external definition of the component fields that is automatically copied into each program that uses the file.

This model is very good for business applications because it allows to identify and process each field of any database file easily, but is not very suitable for other files such as word processing documents, Web pages, Java classes and graphics.

The world of Unix, Linux and PCs has been instead traditionally oriented towards the use of stream files that have not a fixed structure and are well suited for documents processing.

IBM had tried in the past to use the traditional library structure also for stream files with the concept of 'shared folders', but they were difficult to manage and performed poorly.

The IFS was then introduced to allow a better coexistence on the AS/400 of the different approaches described above.

The IFS supports stream files and subdivides the space among different file systems that use different rules and conventions. For instance:

The top level, identified by /, uses the rules of PC style file systems (i.e hierarchical directories and DOS or OS/2 no case sensitive file names)

The QOpenSys is designed to be compatible with Unix based open systems standards such as POSIX and XPG. It is similar to the / file system, but it support also case sensitive file names.

The QDLS file system provides supports for folders and documents.

The QSYS.LIB file system implements the traditional AS/400 support of the library based database files.

The data in each file systems can be accessed by using the traditional commands associated with them (e.g. AS/400 CL commands and programs can still be used with the QSYS.LIB file system and commands from a PC can be used to access the / file system).

However IBM has introduced a common API interface that can be used with all file system.

Some examples of the new common commands are the following:

CRTDIR to create a Directory

RMVDIR to remove a Directory

CHGCURDIR to change the current Directory

DSPCURDIR to display the current Directory

WRKLNK to work with links in a Directory

CPY to copy a file

SAV to save an object

RST to restore an object

One interesting requirement is that all file paths of the new commands must be entered in the Unix style format. This is different from the traditional AS400 library system notation and includes (for original AS400 objects) a suffix that identifies the type of object. For instance the suffix is .LIB for libraries, .FILE for files and .MBR for file members.

The original name for a file could be SALESLIB/CUSTFILE, but this must be referred to in the common APIs as qsys.lib/saleslib.lib/custfile.file/custfile.mbr.

It should be noticed that applications stored in IFS (e.g. JAVA classes or Lotus Domino) can access data store in a different file system (e.g. database files in QSYS.LIB).

It is possible to copy files between file systems, but there are some limitations that must be considered. For instance:

For instance the command CPYTOSTMF (copy to stream file) allows to copy AS/400 native files into stream files, but it supports only source file members or files without an external definition. The same limitation exists for the command CPYFRMSTMF (copy from stream file).

The command CPYTOIMPF (copy to import file) allows to copy a database file into a field delimited format, but it does not allow to specify the code page to use when creating the files. Therefore file transferred from EBCDIC to ASCII are not translated correctly.

The commands CPYTOPCD (copy to PC documents) and CPYFRMPCD (copy from PC documents) allow to make copies between database files and PC stream files, but can be used only with stream files stored in folders of the document file system (QDLS)

Links are supported, but, whereas UNIX like file systems accept multiple hard or symbolic links, a unique link is associated to each database file in QSYS.LIB

Many utilities have been developed to make the transfers of data between AS/400 database and stream files easier and some of them will be discussed later.

The TCP/IP communications

The AS/400 originally supported only the IBM System Network Architecture (SNA) with APPN/APPC communications.

Different solutions were developed to allow the coexistence with other approaches such as TCP/IP or Novell IPX.

For instance Microsoft developed an SNA gateway that allow translating the original SNA protocols into TCP/IP protocols.

IBM decided to open its arms to TCP/IP and adopted TCP/IP as a brother to APPN/APPC after the version 3.1 of the Operating System (OS/400).

The main reasons for this development are the following:

AS/400 TCP/IP enables multivendor connectivity
AS/400 TCP/IP works with AnyNet to reduce network protocol requirements. It allows for instance conventional APPC traffic to travel over a TCP/IP network
AS/400 TCP/IP offers Internet capabilities. It becomes the foundation for services such as World Wide Web and Gopher.

Together with TC/IP the AS/400 has developed all the related services such as DNS, FTP, LPR, PPP, SLIP, SMTP, Telnet etc. and clearly this has made the communication with Linux and Unix machines much easier.

Data transfers are usually done with FTP, but some data format conversion is often needed as described below:

The AS/400 uses the EBCDIC character set, whereas Linux machines use ASCII. The conversion is normally done automatically by FTP.

Numbers are often stored on the AS400 in Packed Decimal format (2 digits per byte) that is not directly supported by Linux. Packed decimal fields must therefore be unpacked when copying data for Linux.

In the AS/400 files the decimal point is implied according to the field definition. For instance if a field is 11 digits long with 2 decimals, no decimal point exists in the representation of the numbers, but the system automatically treats the last 2 digits as decimals. When numeric fields with decimals must be transferred to Linux, an explicit decimal point must be added in the correct position.

The above differences suggest that a direct transfer from the AS/400 database files is not normally possible. It is better to use some utility to create an intermediate file (with unpacked numerical fields and explicit decimal points) and then use FTP to transfer the intermediate file.

When a logical structure of fields must be identified, it is convenient to use field separators in the stream files.

Later you will find a presentation of some utilities that allow to create a file suitable for FTP transfers and that maps fields from a stream file into corresponding fields of an AS/400 database file.

Sharing files with NFS

NFS allows access to remote file systems, which appear to applications as if they reside on the local machine. Therefore the need to duplicate and copy files among different machines on the net is eliminated.

Sun Microsystems originally developed NFS and released it in 1984. Since then Sun NFS has become a de facto standard within the Unix community.

The AS/400 has included NFS support with the V3R7 release of the OS/400 and the implementation is based on Sun's Version 2, which is one of the most commonly used.

NFS uses exporting from the server to make local file systems, directories and objects available to remote systems. The server administrator must decide what local file systems (i.e. subtrees of directories and files) may be exported as well as how they will be exported.

The administrator can choose which clients may access the files and how (e.g. read only).

The options chosen by the server administrator are typically stored in a file called /etc/exports and then the exports can be activated by using the command exportfs. This can be done at start-up and can be included in the AS/400 start-up file QSTRUP run automatically during IPL.

The clients can then add the exported file system to the local machine by using a mount command (normally on an empty directory). The exported file system can later be un-mounted by using the umount command.

If the mount command is done on an AS/400 the format is as follows:

MOUNT TYPE(*NFS) MFS('Sunset:/home/dir1')

MNTTOVRDIR('/Sunset/data')

OPTIONS('ro, retry=10, timeo=60, soft')

CODEPAGE(297 297)

After mounting, client users see the mounted remote file system and files as part of their local name space. The file system appears and functions just as any other local file system would.

The flow of requests over the network is usually transparent to the users.

The communications with the server are handled by Remote Procedure Call (RPC), which is a high-level stateless network protocol that saves no information from client/server communications.

RPC bundles up the arguments intended for a procedure call into a packet of data called a network datagram. The NFS client creates an RPC session with an NFS server by connecting to the proper server for the job and transmitting the datagram to that server. The arguments are then unpacked and decoded on the server. The operation is processed by the server and a return message (should one exist) is sent back to the client. On the client, this reply is transformed into a return value for NFS. The user's application is re-entered as if the process had taken place on a local level.

By using NFS the AS/400 can be included as a client or a server in the network and Linux and AS/400 machines can easily share their files.

An AS/400 can export any of the following file systems:

"Root" (/)

QopenSys

QSYS.LIB

QDLS

QOPT

UDFS

An AS/400 can choose its mount points in one of the following file systems:

"Root" (though not over the root directory itself)

QOpenSys

NFS

UDFS

If an AS/400 stream file is accessed remotely by a Linux machine, it can be edited by using any Unix editor such as vi.

The AS/400 library file system QSYS.LIB can be exported, but there are some limits as follows:

The QSYS.LIB cannot be mounted directly. It is possible instead to mount libraries of the QSYS.LIB file system.

You can access .LIB and .FILE objects, whereas attempt to access a .SAVE file will fail

A client AS/400 will be able to use all file operations on database members (.MBR) and user spaces (.USRSPC).

A Linux Client should be able to see files in QSYS.LIB, but, since they are not stream file, will not normally be able to access them properly. If data from QSYS.LIB are needed by a Linux machine, it would be better to copy the database file into a stream file in one of the stream oriented file systems and let Linux access the copy.

Terminal Emulation

Both the AS/400 and Linux can use Telnet to establish an interactive session.

However, when Linux wants to access an AS/400, Telnet is not the best option because the communication is oriented to single characters, whereas theAS/400 original 5250 data stream manages one full screen at a time.

Some interesting emulators exists for the Linux market and are described below:

Mocha Linux TN5250. This is a commercial product developed by MochaSoft, that produces terminal emulation software also for other environments such as Windows. The Linux version has been developed for the X Windows system and has been fully tested on RedHat Linux 6.1 with KDE and Gnome. The cost is very low and a Single User licence can be bought for 25 USD ($ 250 for a Company). Detailed information can be found at http://www.mochasoft.dk/linux.html

Linux 5250 by Michael Madore. The product has been developed as Open Software by a group of developers leaded by Michael Madore (mmadore@turbolinux.com). It can be freely downloaded at the Web site http://www.midrange.com. It is not yet completely stable, but many persons contribute to improving and testing it.

Some useful AS/400 utilities

We have seen that the different approach used by the AS/400 for the database files, makes a direct transfer between Linux and the AS/400 inadvisable.

It is normally difficult to use direct transfers, because of packed fields and fixed length field requirements on the AS/400 files.

We have therefore developed some utilities that allow to make easier transfer from AS/400 database files to stream files and from stream files to the AS/400 database files.

They are briefly described below. If you are interested to have more details and a copy of the programs send an e-mail to mario@datamission.co.uk

Database File to Stream File Copy

The utility consists of a command (UCRTFTPFIL) and some programs to perform the conversion.

The AS/400 packed fields are converted into numeric fields and an explicit decimal point is added when there are decimals.

Moreover a field separator is added to identify the end of each field.

The command prompt the data in the following mask:

Create FTP file (UCRTFTPFIL)

Immettere le scelte e premere Invio.

AS400 DB File . . . . . . . . . Valore carattere

AS400 Library . . . . . . . . . *LIBL Valore carattere

AS400 Record . . . . . . . . . . Valore carattere

Separator . . . . . . . . . . . '|' Valore carattere

Decimal Point . . . . . . . . . ',' Valore carattere

FTP File . . . . . . . . . . . . Valore carattere

FTP File . . . . . . . . . . . . 0 Numero

It should be noticed that the user can select the separator (e.g. the pipe character) and the decimal point to use (dot or comma).

Stream File to Database File Copy

The utility assumes that the stream file contains fields separated by a separator. It is not requested to have the fields in the same sequence as those on the AS/400 database file.

The mapping of the fields can be entered by using the utility program UFMAP01L, that displays the following screen:

UFMAP01 MAPPING FILE MAINTENANCE 23.01.01 12:26:46

Select one of following values and press ENTER

2=Select 4=Delete

- - - - F T P - - - - - - - - - A S 4 0 0 - - - - -

Sel. File Library File Library Record Last User

EXCTST PESCE EXCTST2 PESCE UCITRA

FTPTRAN PESCE CPTRAN GLSDGERT CPTRAN01

GL_ACT_XRF GL_ACT_XRF CPTABU PESCE CPTABU01 PESCEMA

NRGTRAEXC PESCE NRGTRA UCI NRGTRA00

When one select one row by entering a '2', the program displays a list of the fields in database files and allows to enter the sequence number of the corresponding field on the stream file (described on the screen as FTP file because it is normally received by FTP).

See the example below:

UFMAP30 MAPPING FILE MAINTENANCE 23.01.01 12:31:58

AS400 Database File: CPTRAN FTP File: FTPTRAN

Seq AS400 Fld Description                                      Typ Len D From
No. Name                                                                FldNo
1 CT0010     COUNTRY-CODE                                       P   2     1
2 CT0020     BRANCH CODE                                        P   2     3
3 CT0030     VENDOR CODE                                        P   2     2
4 CT0040     VENDOR PGM-NO                                      P   2     0
5 CT0050     VENDOR POOL-NO                                     P   2     0
6 CT0060     TRANSACTION-NO                                     P   3     0
7 CT0061     STATUS CODE                                        P   1     0
8 CT0080     CRL-NUMBER LESSEE                                  P   5     0
9 CT0110     TRANSACTION FILE STATUS                            A   1     0
10 CT0111     RENTAL STR. FILE STATUS                            A   1     0
11 CT0112     DOCUMENT FILE STATUS                               A   1     0
12 CT0113     EQUIPMENT FILE STATUS                              A   1     0
13 CT0114     PROPOSAL FILE STATUS                               A   1     0
14 CT0115     TRANSACTION FILE ERROR-KZ                          A   2     0
15 CT0116     RENTAL STR. FILE ERROR-KZ                          A   2     0
                                                                     Segue...
F3=Exit          F6=Update Mappings           F9=Assign sequential field no.

In the example above, the field number 1 of the stream file (see From FldNo) is mapped to the field CT0010 of the AS/400 database file, the field number 2 is mapped to CT0030 and the field number 3 is mapped to CT0020.

Once the mappings have been defined, it is possible to perform the copy by using the command UCPYFTP that prompts the user for the following data:

Copy FTP file to AS400 DB file (UCPYFTP)

Immettere le scelte e premere Invio.

AS400 DB File . . . . . . . . .                 Valore carattere
AS400 Library . . . . . . . . .   *LIBL         Valore carattere
AS400 Record . . . . . . . . . .                 Valore carattere
Separator . . . . . . . . . . .   '|'           Valore carattere
Decimal Point . . . . . . . . .   ','           ,, .
FTP File . . . . . . . . . . . .                 Valore carattere
FTP Library . . . . . . . . . .                 Valore carattere
Up to Record No. . . . . . . . .   0             Numero

F3=Exit F6=Add

Also in this case the user is required to specify the separator and the decimal point used for the numbers.

It should be mentioned that the field mappings must be specified only once for each couple of files and that the copy command uses them automatically.

Exercises (Only if you have an AS/400 system in your enviroment)

Ping your AS/400 system
Open a telnet session on it.

Tests

Is possible the integration between AS/400 and Linux ?
Is possible to run Linux on AS/400 systems ?
Is possible to share AS/400 disks with Linux ?
What character set uses the AS/400 system ?
What character set uses the Linux system ?

Read the answers to the exercises.

Check the Interactive Exam Cram Networking:

Internet Resources for this Chapter.