Archiving Operational Data

As an Automic Automation administrator, archiving operational data is one of your database maintenance duties. Archiving is also necessary to comply with legal, audit and reporting obligations that might require you to keep records for several years. Automic Automation lets you extract and store system messages, execution data (formerly called statistics), reports and comments, using the following methods:

In either case, you must first configure the archiving parameters in a variable called UC_UTILITY_ARCHIVE that is available in Client 0. For more information, see UC_UTILITY_ARCHIVE - Configuring the Archiving Parameters.

The archiving process results in a ZIP file that contains various index and data files. This package is stored in a default location as soon as the archiving process is done. You can then move those files to a directory of your choice. The default store location is the following directory, depending on the method you used to archive the data:

  • AWI: The directory that you have specified as default in your browser.

  • REST API

  • AE DB Archive Utility: The path you specify in the path parameter in the UC_UTILITY_ARCHIVE variable.

Requirement: To be able to extract data, your User definition must have the Database Maintenance privilege. For more information, see Granting Automation Engine Privileges.

Recommendations:

  • Archive your data regularly and in short intervals, ideally once a day. You specify the intervals in the UC_UTILITY_ARCHIVE variable.

  • Configure UC_UTILITY_ARCHIVE carefully to make sure that you only archive the data you need. Avoid cluttering the resulting archive files with unnecessary data.

Archiving Operational Data through AWI

You trigger the data archiving process from the list of Clients, either in Client 0 or in a production Client. As soon as an archiving run starts, an entry is written to the Client History list displaying its runID, its status and additional information such as name, type (which is always CLNTARCH for archiving operations), and so forth. Once the archiving process has finished, you can download the resulting files.

To Archive Operational Data through AWI

  1. Configure the UC_UTILITY_ARCHIVE variable.

  2. Go to the Administration perspective and open the list of Clients.

  3. Select the Client whose operational data you want to archive. If you are working in a production Client, this list contains only one entry. In Client 0 it contains multiple entries, however, you cannot select more than one Client to archive data.

  4. Right-click and select Start Archiving. A dialog is displayed that prompts you to confirm that you want to archive the data.

  5. The zip file containing the archive files is now available in the directory that you have specified as default download folder in your browser.

You can see the list of all your archive runs on the Client History page in Client 0 too. For more information, see Monitoring the Client History.

Archiving Operational Data Using the AE REST API

You use the REST API to archive operational data in on-premises, AAKE and Automic SaaS environments. Depending on your type of environment, the procedure is slightly different.

For more information, see AE REST API - Database Maintenance.

Archiving Operational Data Using the REST API in On Premises Environments

Follow these instructions:

  1. Configure the UC_UTILITY_ARCHIVE variable in Client 0.

  2. Configure the outputPath parameter in the Automation Engine INI file. This is where the resulting archive package is stored by default.

  3. Trigger the archiving process.

  4. Download the files from the default location to a directory of your choice. You do this from your REST API client.

Archiving Operational Data Using the REST API in AAKE Environments

Follow these instructions:

  1. Configure the UC_UTILITY_ARCHIVE variable in Client 0.

  2. Configure the pvc in the values.yaml file. This is where the resulting archive package is stored by default.

  3. Trigger the archiving process.

  4. Download the files from the default location to a directory of your choice. You do this from your REST API client.

Understanding the Archive Files

The directory in which the archive files are stored contains subfolders, one per Client whose data have been archived, named after the Client. Each Client sub-folder contains subfolders, one per archive run, with the following naming convention:

UC_ARCHIV_<YYYYMMDD - date of the archive run>_<sequential number>

If you archive data on the same Client on the same date, each run will generate a sub-folder under that Client.

If you archive Client data and the resulting files exceed the maximum size set in SIZE in the UC_UTILITY_ARCHIVE variable, a new sub-folder is created and the remaining files are stored there.

This is what the archive directory could look like:

  • 0100 (name of the Client)

    • UC_ARCHIV_20241101_0 first archive set for Client 0100 in November 1st 2024

    • UC_ARCHIV_20241101_1 second archive set for Client 0100 in November 1st 2024

    • UC_ARCHIV_20241102_0 first archive set for Client 0100 in November 2nd 2024

  • 0200 (name of the Client)

    • UC_ARCHIV_20241101_0 first archive set for Client 0200 in November 1st 2024

    • UC_ARCHIV_20241101_1 second archive set for Client 0200 in November 1st 2024

    • UC_ARCHIV_20241102_0 first archive set for Client 0200 in November 2nd 2024

The following archive files are stored within each sub-folder:

  • UC_ACMT.TXT contains the comments.

  • UC_AH.TXT contains the execution data.

  • UC_AH.IDX execution data are indexed, this is the corresponding index file.

  • UC_MELD.TXT contains the messages.

  • UC_RH.TXT contains the reports.

  • UC_RH.IDX reports are indexed, this is the corresponding index file.

The content of the database fields that are written to the archive files is separated by semicolons. If the archive keywords also use semicolons, they are automatically converted to commas to avoid invalid files.

Structure of the Index Files

The structure of the archive files is always the same, no matter how you have started the archive process (from AWI, through the REST API or through the DB Archive Utility).

The index files are generated when the execution data of the AH and RH tables are archived. Each index line refers to a data record in the archive. The individual positions of the data records are described below:

Index for AH: UC_AH.IDX

  • 1
    Block code H2
  • 2
    Client
  • 3
    User
  • 4
    Department
  • 5
    AType
  • 6
    SType
  • 7
    RunID
  • 8
    Host name
  • 9
    Start time/start date
  • 10
    Parent ACT
  • 11
    Parent PRC
  • 12
    Archive Key 1
  • 13
    Archive Key 2
  • 14
    Object name
  • 15
    Byte position in the archive
  • 16
    Line position in the archive
  • 17
    Archive number
  • 18
    Record length in bytes

Example:

H2;1;LF;PROG;JPAK;GRP;1302137;;2021-05-23 10:31:42;0;0;;;LF.JP.GROUP;73837;2555;7;148

Index for RH: UC_RH.IDX

  • 1
    Block code H3
  • 2
    Client
  • 3
    Report type
  • 4
    Object name
  • 5
    RunID
  • 6
    Point in time when the report was created
  • 7
    Point in time when the report was last updated
  • 8
    Byte position in the archive
  • 9
    Line position in the archive
  • 10
    Archive number
  • 11
    Record length in bytes

Example:

H3;98;JE01;MM.CHECK.FREE.SPACE;1000020;2020-05-24 12:56:03;2021-05-24 12:56:10;2820;39;12;151

Structure of the Message and Report Files

Message Archive: UC_MELD.TXT

  • 1
    Block code M
  • 2
    Client
  • 3
    User
  • 4
    Department
  • 5
    Point in time when the message was generated
  • 6
    Source
  • 7
    Category
  • 8
    Type
  • 9
    Host
  • 10
    Message Text

Example:

M;1;LF;PROG;2024-10-18 09:43:54;Automation Engine;Report;Error;;U0011021 Host 'LFT40' is not active!

Comment Archive: UC_ACMT.TXT

  • 1
    Block code M
  • 2
    Client
  • 3
    User/Department
  • 4
    Object name
  • 5
    RunID
  • 6
    Point in time when the message was generated
  • 7
    Record length in bytes
  • 8
    Message text

Example:

M;100;MH/PROG;TESTJOB;454711222;2024-10-18 09:43:54;nnn;This task was modified by (MH).

See also:

Monitoring the Client History