Defining the Runtime Page

Runtime data is necessary for the various planning and monitoring functions available in the system. Automic Automation collects this data from the Real Runtime (RRT) information available from the executions of the tasks. Based on this information, Automic Automation can calculate the Estimated Runtime (ERT) of the tasks. As a developer and object designer you need this information for scheduling your tasks. You specify the method that Automic Automation will use to calculate the runtimes of executable objects on their Runtime definition pages.

This page describes both the RRT, the ERT and the options that you have to calculate the ERT.

This page includes the following:

Real Runtime (RRT)

The RRT is the period of time during which a task is active, that is, between its start and its end.

The following time lapses are not considered in the task RRT:

• The time it needs for its activation.
• The time it spends with the Waiting for host status.

The last 25 real runtimes of each executable object are saved with the object data. This data is used as basis for the calculation of the Estimated Runtime (ERT). Only the runs that ended with ENDED_OK status are considered.

The smallest possible real runtime amounts to one second. Tasks with a shorter real runtime are rounded up to one second.

Note: Setting the ERT_CALCULATION mode in the UC_CLIENT_SETTINGS system VARA object to BATCH also prevents RRT values from being updated.

Estimated Runtime (ERT)

The ERT is the expected runtime for the next execution of a task. This data is a key value for dynamic runtime monitoring and forecast calculations, as well as for calculating the most recent ending of a task. The ERT is calculated immediately after a task run using its past 25 RRTs.

Important:

• The quality of the ERT calculation is dependent on the available data. In case of a reorganization of the database there could be not enough datasets available. The calculation might be wrong in this case.
• A lot of historical data will be read from the object for the calculation. Therefore this function should be only activated for individual objects, not client-wide.
• If does not often occur that a task runs and ends several times at the same time. In this case it could happen that one of the executions is not included in the ERT calculation. This happens because of the locking mechanisms that are provided in the AE database.

The UC_CLIENT_SETTINGS system VARA object contains settings that apply Client-wide. In its ERT_METHOD key, system administrators can specify an ERT calculation method to be applied to the objects in the Client. This is the method that is used if you select Method as set in UC_CLIENT_SETTINGS in the Calculation Method dropdown list.

Important! The ERT calculation settings that you define on this page override the default defined in UC_CLIENT_SETTINGS.

To Define the Runtime Settings

1. In the Return Code Handling section you specify the exact statuses (return codes) that tasks must have to be considered as ended normally (ENDED_OK). Optionally, you can also specify that another object should be executed in case the task ends with a different status.

   1. To specify the statuses that you want to consider as ENDED_OK, do the following

      • (All objects) Select Consider: Return code and enter the return code in the Return Code for ENDED_OK Status <=. Tasks with a return code equal or smaller than the one you enter here will be consider to complete with the ENDED_OK status.

      • For Windows Jobs, if the return code is set to -1 in Windows, the Job status is ENDED_OK only if the return code equals 0. This allows you to react to negative exit codes in Windows jobs (although exit codes on Windows platforms are in within the 0 to 4294967295 range, the return code is treated as a signed integer, so large exit codes are displayed as negative return values).

      • (OS/390 Jobs only) Select Consider: zOS completion code if you want to specify different return codes for each step in an OS/390 Job. Consider the following:

        • The completion code is specified as a decimal.
        • The allowed codes are 0-2147483647
        • You can define ranges by using a hyphen (for example, 0-8). Separate ranges with a comma (for example, 1,2,10-12).

        Alternatively, in OS/390 the return code of each step can be compared with an entry in a step list. As result of this checks the job will be evaluated as normally ended or aborted.

        This is how return codes are checked when a z/OS job ends:

        • The system verifies if the job completed with an abnormal status (ABEND) caused by the system (first) or a user (second). If so, the job is regarded as aborted and the maximum return code is used.

        • If no step list has been defined, the job's return code is checked and used.

        • If a step list has been defined, the return codes of the individual steps are compared to the step list. Only steps whose return codes are not equal to 0 are checked. Return code 0 always signals OK.

        • If a step is included in the step list, it depends on the definition whether the step is OK or not OK. The first step that is not OK considers the job to be aborted. This step's return code is used as the job's return code.

        • If a step is not included in the step list, its return code must be 0. Otherwise, the job is considered as aborted.

      • You can set the return code using the :MODIFY_STATE script element.

   2. Optionally, in Otherwise: Execute object specify which object should be executed if that task ends with none of the OK statuses you have just defined. For example, this could be a Notification object that is sent to a particular group of users informing them of the failure.

   Note: For File Transfer objects, the actual status of the transfer does not affect its return code. If, for any reason, a file transfer fails, the return code is still 0.

2. In the Estimated Runtime (ERT) section you define the method with which to calculate the ERT. You have the following options:

   ERT calculation,calculate ERT,Estimated Runtime calculation,calculate Estimated Runtime

   • Method as set in UC_CLIENT_SETTINGS

     Select this option if you want to apply the method and parameters that your system administrator has specified in the UC_CLIENT_SETTINGS VAR object.

     • New objects

       There are no past runs for calculating the ERT. Since you need them for forecasts and runtime monitoring, for example, the Current/Fallback ERT fields are enabled to allow you to enter a suggestion. This value is overwritten after the task has completed its first run. It is then replaced by the ERT value that is calculated using the method specified in the VARA object.

     • Already executed objects

       After the object has been executed, these fields are disabled and display the current, real values.

       To reset them click the Reset Values button below the Runtime Statistics graphic. This enables these fields again.

   • Fixed value

     In Duration enter a static value and review it from time to time to make sure that it still meets all your requirements.

     The Current ERT fields are disabled when the duration is saved.

     Note: Forecast always uses the current ERT for forecast calculations. If a fixed value is set, it is used for further task executions but not for the forecast. If you want to use a fixed value, you must reset the ERT calculation first and also after the next execution. The fixed value is also the current ERT.

     Important!

     Consider the impact of setting the fixed value too high:

     • Forecasts: The completed execution is assessed too high because it always starts from the maximum allowed runtime (worst case). Therefore, the forecast cannot give realistic values.
     • Minimum runtime monitoring always activates the actions defined in case of Runtime deviation because the RRT is always smaller than the ERT.
     • Maximum runtime monitoring never activates the actions defined in case of Runtime deviation because the ERT is never reached.

     Consider the impact of setting the fixed value too low:

     • Forecasts: The completed execution is assessed too low because the RRT is mostly above the set value. Therefore, the forecast cannot give realistic values.
     • Minimum runtime monitoring activates the actions defined in case of Runtime deviation sometimes because the RRT is usually above the specified ERT.
     • Maximum runtime monitoring activates the actions defined in case of Runtime deviation much too often because the specified ERT is exceeded.

   • Dynamic

     The following dynamic methods are available:

     • Adaptive, see Calculating the ERT with the Adaptive Method

     • Linear regression, see Calculating the ERT with the Linear Regression Method

     • Average, see Calculating the ERT with the Average Method

     • Maximal value, see Calculating the ERT with the Maximal Value Method

3. The Runtime Statistics section provides a graphical interpretation of the real (in blue) and of the estimated (in green) runtimes that help understand the relationship between both. You can display both at a time, which allows you to compare their behavior, or you can disable one of them by clicking on the corresponding area of the box below the chart.

4. If necessary, click the Reset Values button at the bottom of the graphic. This deletes all your runtime data.

   Important!

   • Resetting values deletes the runtime data even before saving the object.
   • Real runtime data from archived data cannot be restored.
   • This function is only useful if you expect major runtime deviations for future runs (for example, you have changed the task).

5. In the Max & Min Runtime section activate Monitor MRT and/or Monitor SRT if you want to monitor the maximum and/or the minimum runtime.

   maximum runtime, minimum runtime

   The following calculation methods are available:

   • Fixed value: Enter a fixed duration.

   • ERT + additional duration / ERT - additional duration

     Select this option to add or subtract the ERT duration time and enter the correction time in %. The value you enter here is added to (for max. duration) or subtracted from (for min. duration) the ERT.

   • Real date + additional duration (for the maximum runtime only)

     You can define a particular time until which the task must be finished. You can combine the following options:

     •  The number of days in the Additional Duration field.
     •  A specific time in the Finish Time input fields.
     • A Time Zone object.

6. Optionally, define the Actions on Runtime Deviations.

   Tip: Use Service Level Objective (SLO) objects to monitor runtimes and trigger actions in case of deviations.

   Note: When "Cancel or quit this task if runtime has been exceeded" is used on a Workflow, all inactive and not started tasks are canceled. Currently active tasks are allowed to finish, and not canceled immediately.

7. In Forecast Settings select the end status that a task should return in case of forecast calculations.

Calculating the ERT with the Adaptive Method

The Adaptive ERT method estimates how long an executable object will take to run. Unlike the classical ERT methods, which compute a fixed value, an average, a maximum, or a linear regression solely from raw past runtimes, the Adaptive method uses a machine learning model that considers the full execution context. This allows it to produce different estimates for the same object depending on who runs it, where it runs, when it runs, and what input data it processes.

Requirements:

• At least 1 Java-based work process (JWP) must be available and running.

• An alternative calculation method is defined in the ERT_ADAPTIVE_FALLBACK_METHOD key (UC_CLIENT_SETTINGS) if no JWP is available.

• ERT_ADAPTIVE is set to Y (UC_CLIENT_SETTINGS).

See ERT Parameters.

Notes:

• The ERT is not calculated until the JWP is running. This can take a while. In ERT_ADAPTIVE_JWP_TIMEOUT (UC_SYSTEM_SETTINGS), administrator users can define how long the system should wait for the JWP after starting and before an alternative method should be used.

• Tasks that are waiting for the adaptive ERT calculation have the Waiting for ERT status. This applies to tasks for which the minimum or maximum runtime is calculated depending on the ERT.

• Object variables can be used as external factors for calculating the Estimated Runtime of the object with this method.

• Monitoring minimum or maximum runtime must be activated for the task in order to use the Adaptive method.

In Fallback ERT you can enter a hard-coded value that will be used in case the ERT cannot be calculated with the available attributes.

What the Model Learns from

Each completed execution produces a training record consisting of two parts:

• Internal context

  A set of contextual attributes that describe the circumstances of the execution. Which attributes are included is configurable via the system setting ERT_ADAPTIVE_DEFAULT_CONTEXT in UC_CLIENT_SETTINGS. For more information, see ERT Parameters.

• User-defined ERT variables

  When you define a variable that is specific to an object, you can specify that the variable be used as external factor in the calculation of that object's ERT. You do so on the Variables definition page, Defining the Variables Page.

  If this is the case, any object variable marked as ERT-relevant this way is included as additional attribute. These variables can be of type text, number, timestamp, date or time. This allows business-specific dimensions (customer ID, a batch size, a processing date, and so forth) to influence the estimate.

  The target value the model learns to predict is the actual runtime in seconds of each completed execution.

Training the Model

Automic Automation maintains one trained model per executable object. The model is built from the most recent 200 completed executions of that object, reading both the execution metadata and any ERT-tagged variables from the executions database.

Training happens automatically:

• The first time an ERT estimate is requested for an object that has no existing model, the system trains one immediately from available history.

• Subsequently, the model is refined over time through a feedback mechanism. For more information, see Self-Improvement Through Feedback.

The trained model is persisted in the database so that it survives server restarts without requiring retraining from scratch.

How an Estimate is Produced

When an execution starts, Automic Automation assembles the current execution's context (the same attributes described above — parent alias, host, start time, ERT variables, etc.) and passes them to the trained model.

The model internally runs two independent algorithms against these inputs:

• A k-Nearest Neighbor algorithm, which finds the most similar past executions by comparing their attribute values and averages their actual runtimes.

• A Regression Tree algorithm, which navigates a tree of decision rules learned from training data to arrive at an expected runtime for the given combination of attribute values.

A third component, the meta-learner, then decides which of the two algorithm outputs to trust for this particular input. The meta-learner is itself trained during the model-building phase to learn which algorithm performs better for which kinds of inputs.

The final estimate is always bounded by the minimum and maximum runtimes observed during training, ensuring the result remains within a historically plausible range even if the model extrapolates unexpectedly.

Self-Improvement Through Feedback

Every time an execution finishes, Automic Automation receives the actual runtime and compares it against the predicted value. This feedback is used to keep the model current. A full model retraining is triggered automatically when any of the following conditions is met:

• Sufficient new data has accumulated: A configurable number of completed executions have provided feedback since the last training.

• Prediction accuracy has degraded: The difference between predicted and actual runtimes has grown beyond an acceptable threshold, indicating the model is no longer reliable.

• New attribute values have been encountered: The execution contained a context value never seen during training, such as a new Agent name or a new parent Workflow alias. Because the model was not trained on this value, a retrain incorporating the new data is scheduled.

When a retrain occurs, Automic Automation reads the latest history, builds a new model, and stores it in the database. The old model is replaced.

Model Persistence and Caching

To minimize database access, trained models are held in memory on the server for the duration of its runtime. The in-memory copy is used for all predictions and is written back to the database periodically and at server shutdown. If the server restarts, models are reloaded from the database, avoiding a cold-start retrain for all objects.

Calculating the ERT with the Linear Regression Method

This is the default method. It emphasizes runtime increases or decreases.

• If you enter a value in Current ERT, it applies until an ERT is calculated after the next run.
• Enter the No. of past runs that should be used for calculating the ERT. Allowed values: 1 to 25
• To avoid the evaluation of extreme deviations, enter the +ERT Correction. This is the maximum deviation level. If the RRT exceeds this limit, it will not be taken into account in the calculation of the ERT. Additionally, the number of runs to be used can also be specified.

Calculating the ERT with the Average Method

Calculation of the average ERT based on the last RRTs of a task. You define here the number of RRTs to be considered.

• If the number of RRTs is too low, the ERT will follow any existing tendency in runtime behavior.
• If the number of RRTs is too large, extreme deviations with RRTs are not emphasized.

To preserve a distance between ERT and the RRT, it is possible to specify a correction value (+ERT Correction). This value is added to the ERTs to ensure that the ERT follows the RRT of a task at the specified distance.

As with the linear regression method, here you can define the Current ERT and No. of past runs. If No. of past runs is set to 0, then the Client settings are used.

In addition, to avoid the evaluation of extreme deviations you can define thresholds in which you want to ignore them. If the real runtime exceeds this limit, it will not be taken into account in the calculation of the ERT:

• Deviation extent: If this limit (in %) of the actual runtime is exceeded, the estimated runtime is ignored.
• Min. runs: Number of past runs to be considered regardless of deviations. Allowed values: 0 to 25.

Calculating the ERT with the Maximal Value Method

The highest value from the list of RRTs is used to calculate the ERT. If a large number of runs is to be calculated, the ERT immediately adjusts to increasing runtimes. However, if the RRT decreases, this results in a slow adjustment.

For the decreasing tendencies to be recognized faster, the number of runs to be calculated can be reduced.

To preserve a distance between ERT and RRT, a correction value in percent can be defined. This value is added to the ERTs. This allows you to set an alarm only with extreme increases in runtime.

The UC_JOB_CHECKINTERVAL variable defines the interval that is applied system-wide for checking times. This check is necessary when time dependencies are defined in objects. Take this interval into account when defining the MRT settings. For more information, see UC_JOB_CHECKINTERVAL - Periodic Time Check in AE.