Executing File Transfers

Exchanging data between different system platforms requires the character sets to be converted. This is done automatically using the Code Table Object (CODE). The transfer can be structured, thus enabling the exchange of files with packed and binary fields in heterogeneous system environments. All network transfers are encoded by default.

This topic provides information on the following:

Background

For each file transfer, a separate connection is established between the agents. Agents for the OS/400, Windows, Unix, z/OS and NSK operating systems handle file transfers through extra threads. As soon as a file transfer has ended, the corresponding connection is automatically closed. If there are still open and unused connections, they are closed after the period of time that is defined in the FT_CONNECTION_TIMEOUT key (variable UC_HOSTCHAR_DEFAULT, see UC_HOSTCHAR_DEFAULT - Host Characteristics).

The old file transfer protocol is used if one or both of the affected agents use Automation Engine version 8.00A or earlier. In this case, only one connection is established between the agents for all file transfers. The system checks if a connection has already been established between the two agents before a file transfer starts. If so, this connection is used and no attempt is made to establish a separate connection. After the file transfer has ended, the system ensures that a connection that is still used is not closed.

Please take the following into account:

Transferring Files

Each file transfer is started by establishing a connection (see File Transfer (FT) Procedure) between the source and destination Execution. The file can be transferred as soon as a connection can be established. There are two methods of transferring files:

  1. With a temporary file:

    The file that should be transferred obtains a specific name in the destination system. This name is composed of the letter "T" and a 7-digit alphabetic string (such as TAAFLEFM). The string is based on the file transfer RunID, which ensures that the file name is unique.

    Also refer to the documentation about script functions RUNNR2ALPHA and ALPHA2RUNNR .

    In the new FT protocol, a separator and a file ID are appended in addition to the string. Doing so makes a distinction possible because all files are handled via the file transfer task's RunID.

    The file obtains the name specified in the FileTransfer object after a successful transfer.

    The temporary file is created in the file transfer's destination directory.

  2. Without a temporary file:

    The file that should be transferred obtains the name that is specified in the FileTransfer object in the beginning.

The method that should be used depends on the specifications that the administrator has made in the destination agent's INI-file parameter ft_temp_file=.

A temporary file is not created if an existing file is extended or overwritten. Contents are directly inserted in the relevant files.

The huge advantage of temporary files lies in the simplified handling of further processing because the files only obtain their specified names after they have successfully been transferred.

If an error occurs, the temporary file is deleted in the destination system. Files that could not completely be transferred are not removed on UNIX and Windows computers. Therefore, the agent does not have to newly start the file transfer if the file transfer is restarted (see Restarting File Transfers).

Temporary Files On the File Structure of the Source Agent

Each time an source agent does a file transfer, it produces two temporary files:

The location of these files is determined by the UC_EX_PATH_JOBREPORT variable set in the [VARIABLES] section of the agent .ini configuration file.

The agent uses these files to reconstruct the file transfer if the agent is restarted or if a transfer fails. If the file transfer completes successfully, the files are automatically deleted. If the file transfer fails to make a restart, the files remain on the file system. The number of days the status files remain is determined by the value of the FT_RESTARTINFO_LIFETIME key in the UC_HOSTCHAR_DEFAULT variable.

Agent Groups

An agent group can either be used as the source or as the target. This results in two possible ways of usage:

  1. File transfers from a single host to many target hosts.
  2. File transfers from many hosts to a single target host.

Pay particular attention to the latter one, as the files must not have the same name. Otherwise, the file transfer overwrites the files or it cannot transfer the file if it is being deployed by parallel processes at that point in time. Insert the following variable in the target file name: &<FT_SRC_HOST>. The file transfer replaces this variable with the name of the file's original host.

For example:

C:\temp\&<FT_SRC_HOST>_test*.txt

Note that agent groups cannot be selected in file transfers as a source and a target at the same time!

Compression

Files can also be transferred in compressed form. You can activate this option in the File Transfer definition page (Defining File Transfer Objects).

If you select the compression type "Default value" in the file transfer, the system uses the value that the administrator has defined in the UC_HOSTCHAR_DEFAULT variable , setting FT_COMPRESS. The files are transferred in compressed mode if the two agents have different settings.

For example:

The agent uses the LZP+Huffman procedure for the compression.

When using compression, make sure that effort (time required for compression) and benefit (gain of time because of reduced transfer time) are balanced against each other. An important component for your decision is the available bandwidth of your network connection. You cannot increase performance with fast networks (100 Mbit)because the compression process is more time-consuming than the gain achieved by reducing the network time. With an available bandwidth of 10MBit or less, compression will definitely result in a performance increase.  

End Status

Transferring a Particular File

A file transfer obtains the status ENDED_NOT_OK if the relevant file is not available.

Use case: A file transfer is to transfer a particular file which should then further be processed.

Transferring Files Using Wildcard Characters

A file transfer obtains the status ENDED_EMPTY if no file is available which complies with the filter specifications. It obtains the status ENDED_OK if at least one matching file is found.

Use case: A file transfer is to transfer several files whose naming convention follows a particular pattern . The focus is on file names and not on the number of files.

UNIX

The administrator can use the parameter login_check=yes to specify in the INI files of the UNIX agents that the Login object's password is checked. For reasons of security Automic recommends using this setting. Login_check=no only checks the user name.

VMS

In VMS, each text file requires a delimiter to be appended. The administrator can define in the INI-file parameter FileEndDelimiter= whether the VMS agent should append one.

See also: