ETL Process

The command line program of Process Analytics Admin starts the ETL process (data-loading process). The process consists of the following steps:

  1. Extraction

    The AE database stores the meta data of tasks (executable objects such as workflows or jobs) in the AE system. It is stored in the following AE database tables and can be called by using the ETL process.

    • A* Tables: Information about tasks that have already run (statistics).
    • E* Tables: Information about active tasks (such as the runtime).
    • F* Tables: Information about scheduled tasks that have not yet been processed (forecasts).
    • O* Tables: Runtime information (minimum, maximum) of tasks that have already run.

  2. Transformation

    Depending on the task model, the data is converted accordingly. Each task is converted in a way that it can be displayed in Process Analytics as a long-running task, a single task or as a parent object.

  3. Loading

    The task model data is loaded to the Process Analytics Database. The following Process Analytics database tables refer to the ETL process and store the following information:

    • ACT Table: Information about long running tasks such as Schedules and period containers.
    • CON Table: Data of all individual tasks and parent objects.
    • DEP Table: Information about explicit and external task dependencies.
    • LI Table: Information about executed ETL processes.
    • LO Table: Information about currently active loading processes. This ensures that only one loading process is executed at a time.
    • TACT, TCON, TDEP (Temporary Tables): Buffer for the data of the ACT, CON and DEP tables.

      The AE system data is initially buffered in the temporary tables when a loading procedure is triggered. This serves the following purposes:

      • Consistency: The data that AE Process Analytics displays remains valid during the ETL process because this data does not change until the data has completely been loaded from the AE database. Afterward, all data is moved to the tables ACT, CON and DEP and can be accessed by Process Analytics.
      • Reliability: A failed data-loading process does not affect the displayed data. You can restart the data-loading process after the temporary tables have been cleared in order to avoid inconsistencies.
      • Performance: Data is only retrieved from the AE database during this process. There is no need to merge or delete data. The time that is required to access the AE database is reduced to a minimum.

      ETL Process

      You can determine the source database (AE database) and the target database (Process Analytics database) during the Process Analytics Admin configuration process. See: Configuring Process Analytics Admin.