Jobstreams

Jobstreams are containers of jobs that are related to each other, that run together and that end in the execution of the target job. The jobs are the batch processes that are automated by the underlying scheduler. The target job is the end point that you are interested in monitoring and for which you have set up an SLA. The jobstream collects the runtime data of the target job and its direct upstream predecessors. As the individual jobs execute, AAI keeps track of how long it takes for each job to execute. Then, it aggregates those times over the entire jobstream, providing an up-to-date picture of what is going on: Runtime analysis, statistics, averages, and so forth. This information is the basis of the AAI's monitoring and supervision tools such as SLA management and alerting.

Scheduler

In AAI, a scheduler is a representation of an automation engine that regularly provides data that populates AAI's time-based dashboards, charts, reports and so forth. AAI gathers data from these instances at predefined intervals.

For more information, see Schedulers.

Schedulers, Batch Processes and Jobstreams

Once AAI is installed and integrated with the scheduler(s), it can start extracting the data that it needs to build the jobstreams. AAI extracts two types of data:

• Object definitions

  Static object definitions that describe the jobs: Their schedules, dependencies and resource conditions, how they are connected, when they start, and so forth. AAI converts this data into jobstream definitions.

  When installing and configuring AAI and the connectors, the technical teams have set up the regular intervals at which AAI polls this data. Since these definitions do not change very often, the usual polling cycle is once or twice a day.

• Run data and metadata

  Job events, status changes that occur as executions take place in the scheduler. AAI recreates those job events and status changes and generates jobstream runs accordingly.

  When installing and configuring AAI and the connectors, the technical teams have set up the regular intervals at which AAI polls this data. Run data is dynamic, it changes constantly so the polling cycles are short in this case (every 20 seconds, every minute and so forth).

Once the scheduler and AAI are fully integrated, AAI automatically extracts the data it needs to build the jobstreams. Schedulers can potentially handle millions of processes a day, however, the processes that are actually relevant for AAI are a small fraction of them. The next thing that you need to so is to identify the processes that are business-critical and that you want AAI to analyze and monitor. You do this by identifying the target jobs.

Basics of Jobstream Creation

AAI creates the jobstreams automatically as soon as the scheduler and its connector (if applicable) are defined and configure and the data transfer between the workload automation solution (through the scheduler) and AAI takes place. However, it is also possible to create jobstreams using the following methods:

• Manually through the user interface, see Adding/Editing/Deleting Jobstreams.

• Automatically using the CLI, see Jobstream Commands.

When you create a jobstream manually, the first thing you do is to define its target job. AAI automatically discovers the target job's upstream dependencies (its predecessors) and builds the actual jobstream. As soon as a jobstream is created, AAI starts monitoring it at runtime and performing calculations that lead to predictions. If delays occur along the way, AAI sends alerts that let users remediate the cause before the timely completion of the jobstream is jeopardized.

Target job

The target job is the end point in the jobstream. It is the end job against which you track an SLA. It is typically a job that you consider critical to your business: either the job needs to start or finish by a particular time, or it is critical to your environment in some other way. AAI automatically creates historical views of the jobstream by tracing the upstream dependencies of the target job. It is used as criterion to determine lateness; if the target job is late or predicted to be late, alerts are triggered.

Fine-Tuning Jobstream Definition

Whether you have created a jobstream manually or if it has been created automatically by AAI, you can fine-tune its definition. AAI provides two powerful tools that let you strip irrelevant jobs from the jobstream. Decluttering data this way results in graphs and report that are easier to understand.

You have the following options:

• Trimming Jobstreams

• Start Jobs in Jobstreams

Critical path

The critical path is a specific sequence of jobs within a jobstream that directly affect or are predicted to affect the completion time of a jobstream. It singles out every job in the jobstream that comprises the longest path required for a particular jobstream run to be successful. This means that if any of the jobs on the critical path is delayed, the jobstream is also delayed. If a job is not on the critical path, any delay associated with this job does not necessarily affect the completion time of the jobstream. However, if a job that is not on the critical path is delayed enough to affect the completion of the jobstream, AAI adds it dynamically to the critical path in real-time. Therefore, the critical path can change from run to run.