Adding/Editing/Deleting Jobstreams

There are several ways to create a jobstream in AAI. From the Analysis tab a user can click on the button above the Jobstream List or right-click on a Business Area folder in the Categorized Jobstream List, and select Add Jobstream. In either case, a New Jobstream dialog will appear, then a Select Target Job dialog will pop-up.

As a business area coordinator, you can add or edit jobstreams from various places in the user interface:

  • Admin tab

  • Analysis tab

This page includes the following:

Accessing the Add Jobstream Dialog

As an AAI administrator, you have various options to add a jobstream:

  • From the Admin tab

    Select Jobstreams in the left pane and click the plus icon on the upper right corner of the Admin tab.

  • From the Analysis tab

    You have two options:

    • Right-click a business area and select Add Jobstream.

    • Click the plus icon on the Jobstreams toolbar

For example:

Selecting the Target Job

As soon as you click on the plus icon or select Add Jobstream, the Select Target Job dialog is displayed.

The target job is the end point in the jobstream. It is the end job against which you track an SLA. It is typically a job that you consider critical to your business: either the job needs to start or finish by a particular time, or it is critical to your environment in some other way. AAI automatically creates historical views of the jobstream by tracing the upstream dependencies of the target job. It is used as criterion to determine lateness; if the target job is late or predicted to be late, alerts are triggered.

  1. Use Find jobs where name (starts with, ends with, contains, matches) to restrict the list of potential target jobs to the ones that comply with the string you enter here. This field is case sensitive.

  2. Select a target job is selected from the resulting list. AAI displays the number of Jobs in that jobstream and the number of Historical Runs of that jobstream.

    For CA7 only

    A CA7 jobstream is defined by the combination of the target job and the SCHID. Multiple jobstreams can have the same target job with different SCHIDs. However, jobstream names must be unique.

    If you select a CA7 job, the Schedule ID drop-down list is displayed showing all SCHIDs for which an actual run for the selected target job selected is available. If AAI does not have a run on the SCHID for which you need to create a jobstream, type the specific SCHID here.

  3. Click OK to go to the New Jobstream dialog.

Note:

You can have multiple jobstreams per target job. For example, you can have several jobstreams where you define different trim points but where you use the same target job. For more information about trimming, see Trimming.

Naming the Jobstream

Give jobstreams meaningful names that users can easily recognize. An option could be to give them the same name as their target jobs, provided the target job names understandably describe the process they execute. Ideally, jobstream names should describe the processes that operators will analyze and monitoring through the jobstream, for example End of Day Processing.

If you assign the jobstream a name that is different from its target job, add a jobstream description that provides users with the information they need to understand its purpose.

Configuring the Statistics Section

The Statistics section shows the number of jobs and job runs in a particular jobstream. During jobstream creation, the information here helps you decide whether you need to add a trim point to the jobstream. For very large jobstreams, AAI asks you whether you want to create a trim point to reduce the number of upstream jobs in the jobstream.

Select one or more of the following:

  • Make this jobstream available for monitoring and alerts

    This option is selected by default.

    If you do not want to monitor the jobstream you are creating, deselect this option. If at a later time you want to enable monitoring and alerting, you can do so in the Edit Jobstream dialog.

  • Make this jobstream available for forecasting

    Select this option to forecast jobstream runs. By default it is NOT selected.

  • Is critical jobstream

    This option is selected by default. If selected, the following happens:

    • This jobstream is considered critical to a business area, allowing users to filter by critical jobstream in the Monitoring and Dashboard views.

    • If desired, AAI can limit the calculation of End of Business Area Cycle to only the jobstreams that are considered critical.

Defining the Jobstream Properties

You have defined the basic jobstream properties so far. Click More to expand the dialog and display the following tabs where you continue configuring the jobstream:

  • General

  • Trimming

  • Description

  • Alert Criteria

General

On the General tab you specify where the jobstream runs (business area) and which job runs are included in the jobstream.

Business Areas

By assigning jobstreams to business areas you group them logically in whatever way it makes sense in your enterprise. Business areas are most often organized around business units or application areas and they can have several levels of nesting.

You can assign one jobstream to one or more business areas.

Historical Runs

Specify the number of days of historical runs available to AAI that will be included in the jobstream.

Default: 30 days.

To deviate from the default, in Include runs since enter the date as of which the historical runs should be included in the jobstream.

Jobstream average end-time and duration are calculated from the included runs only. However, job-level duration and delay statistics always include all available runs.

(AutoSys) You can also add archive files.

Notes:

  • You can modify the number of days of historical run data to be included during jobstream creation in the client.jobStreamRunHistoryInitialLimit parameter in the Configuration Tool - Params Tab.

  • Limiting the number of runs can improve performance if a very large number of runs exist.

Days of run history to keep

Specify the number of days of run history that AAI should keep on the server. Select Custom and enter a number to change the server default.

Advanced Settings

This section is slightly different between schedulers.

  • AutoSys

    Allow new run based on force started job

    Force starting a job in AutoSys may end up causing the Target Job to run. AAI will see this as a new run of a jobstream and, usually, this is the desired behavior. Deselect this option that is not your desired behavior.

    (Click on the image to expand)

  • CA7, IWSd and Tidal

    Maximum Jobstream Run Duration

    There may be jobstreams in which job runs take place long before the target job runs. Including these runs may make for an unnecessarily complicated jobstream. If such runs occur early enough, you can exclude them from the jobstream. Use this option to limit the job runs in a jobstream to those within a specific period (in hours and minutes) before the end of the target job run.

    CA7:

    IWSd:

    IWSz:

    Tidal:

  • Tidal

    Allow Group to Start a Jobstream

    By default, the start time of a jobstream run is the same as the start time of the earliest job or group run in that jobstream. In some scenarios, a container can start well before (potentially hours) any job of the batch actually runs. This can lead to situations where the container starts the jobstream, which then shows as having run for many hours before the first job runs, thus cluttering up monitor views and resulting in misleading statistics.

    Use these options to specify the jobstream behavior in these situations, whether a container start time should indicate the start of a jobstream run or not.

    The client.allowGroupStartJobstreamRun server parameter controls the default behavior of this functionality for all jobstreams. The default for the server can be modified through the Params tab of the Configuration Tool.

    • Server default

      This option is selected by default. The behavior specified by the value of client.allowGroupStartJobstreamRun is applied to the jobstream.

      Deselect this option to be able to modify this behavior for this jobstream. Deselecting it enables the Allow option.

    • Allow

      If you select this option, parent group(s) start times are used as the start time of a jobstream run.

      If you do not select this option, parent group(s) runs will NOT be used as the start time for a jobstream run. Instead, the start time of the earliest non-group job is considered the start time of the jobstream run.

    Important!

    • Changes to the server default setting (client.allowGroupStartJobstreamRun) will only take effect for future jobstream runs. If you must apply the changes to historical runs too, you have to rebuild the jobstream run history.

      To do so, open the Edit Jobstream dialog and, in the General tab, select a date in the past.

    • Modifying the Allow option on a per jobstream basis will automatically take effect for both historical and future runs of the jobstream.

  • Universal schedulers (Automic Automation, Control-M, ESP, CA7)

    By default, the start time of a jobstream run is the same as the start time of the earliest job or group run in that jobstream. In some scenarios, a container can start well before (potentially hours) any job of the batch actually runs. This can lead to situations where the container starts the jobstream, which then shows as having run for many hours before the first job runs, thus cluttering up monitor views and resulting in misleading statistics.

    Use these options to specify the jobstream behavior in these situations, whether a container start time should indicate the start of a jobstream run or not.

    In Jobstream Run Starting Job you define which will be the first job in the jobstream.

    Consider the following scenario:

    Where:

    • Workflow 1 starts at midnight

    • Workflow 2 starts at 8:00 am

    • Job 11 starts at 9:00 am

    • All jobs in Workflow 2 complete by 10:00 am, which causes Workflow 3 to start

    • Workflow 3 starts at 10:00

    • Job 1 starts at 11:00 am

    • You create a jobstream and identify job 7 as target job

    Depending on the option you select here, the following happens:

    • Can be any job/parent

      Any job run for any job in the jobstream is a candidate to be the first job run in the jobstream. This is the default behavior.

      The universal.jobStreamRunStartingJobOption server parameter controls the default behavior of this functionality for all jobstreams. The default for the server can be modified through the Params tab of the Configuration Tool.

      Considering our scenario, the run for the start of Workflow 1 comes in at 00:00. At this time a jobstream run is created with a start time of 00:00 and the run for Workflow 1 is added. The jobstream run will show up in monitoring with an actual start time for Workflow 1 and predictions for everything else in the jobstream. All other job runs will be added to the in-progress jobstream run until it completes.

    • Ignore if a container

      The first job in the jobstream run cannot be a container.

      In our scenario,

      • The run for the start of Workflow 1 comes in at 00:00 but since Workflow 1 is a container, no jobstream run is created.

      • The run for the start of Workflow 1 comes in at 08:00 am but since Workflow 1 is a container, no jobstream run is created.

      • The run for the start of Job 11 comes in at 09:00 am. Since Job 11 is NOT a container, jobstream run is created with start time 9:00 am.

      The runs for Workflow 1 and Workflow 2 are included in the jobstream run only so that the container can be drawn on the Gantt chart. Their start times on the Gantt chart are 9:00 am (the start time of Job 11).

    • Ignore if an outermost container

      The first job in the jobstream run cannot be an outermost container. Any container that does not itself have a container cannot start a jobstream run.

      In our scenario:

      • The run for the start of Workflow 1 comes in at 00:00. Since Workflow 1 does not have a container, no jobstream run is created.

      • The run for the start of Workflow 2 comes in at 8:00. Since Workflow 2 has a container, a jobstream run is created with start time 8:00.

      • The run for Job 11 comes in at 9:00. It is added to the existing jobstream run as are all other subsequent runs of the jobs in the jobstream.

      The run for Workflow 1 is included in the jobstream run only so that a container can be drawn on the Gantt chart. Its start time on the Gantt chart is 8:00 (the start time of Workflow 2).

Trimming

With the trimming function you optimize AAI's calculations and predictions by removing unwanted predecessors for the target job. Trimming is a key jobstream function that is described in a dedicated topic. For more information, see Trimming Jobstreams.

Description

Use this tab to provide a description of the processes executed by this jobstream. This information should be useful for your end users, that is, operators who are interested in the outcome of the processes.

Start Jobs

By default, when you add a jobstream, AAI determines that the jobstream starting point (that is, the job that triggers the jobstream run) is the target job's earliest predecessor. However, this automatically designated first job might not be the actual start of the process that you want to monitor. Setting a hard start job is a key jobstream function that is described in a dedicated topic. For more information, see Start Jobs in Jobstreams.

Late Criterion

The Late Criterion tab is available when you edit a jobstream. You determine here when AAI will consider this jobstream to be late. By default, AAI calculates the late criterion automatically, however you can override it by specifying a different way to calculate it.

AAI provides the following options to determine when a jobstream is considered to be late:

  • System-Calculated Late Criterion

    Select this option if you want AAI to decide which method is the best one to determine when it will consider this jobstream to be late. AAI's algorithm chooses the best criteria based on the historical data that it has collected in its database. Depending on the method that it has chosen, AAI then displays either the late time, the average end time or the average duration. It also indicates what method the late criterion calculation is based on.

  • End Time (user supplied)

    Enter the end time as of which you want AAI to consider the jobstream to be running late.

    If you need to display the times in a time zone that is different from the target job's time zone, select it from the dropdown list. If the list does not show the time zone you need, select Show all time zones.

    Once you have selected the end time, AAI displays the average end time (and the standard deviation) for the jobstream based on the historical data contained in its database. This information helps you select a reasonable end time for the jobstream.

    The Options button opens the End Time Options dialog, where you can further manipulate the end time. Since the AAI prediction engine calculates this value automatically, we do not recommend to use this option. In Jobstream completes nn calendar day(s) after jobstream starts you can enter the number of calendar day(s) after the jobstream starts that the jobstream end time will be determined by. For example, if a jobstream starts to run at 11:00 p.m. on Monday, and the end time should be at 5:00 a.m. on Tuesday, then you would enter a 1 in the calendar day(s) after jobstream starts field in order to accurately reflect the proper end time.

    Tip:

    You may want to define jobstreams that are based on daily and end of month (or other similar criteria). Using the same Target Job, you can create different jobstreams that have different user-defined end times (i.e., with End Of Month having a later end time). This can help prevent unnecessary alerts for Month End processing.

  • Fixed duration (user-supplied)

    Specify how long the jobstream can execute before AAI considers it to be late. AAI displays the average duration (and the standard deviation) for the jobstream based on the historical data contained in its database. This information helps you select a reasonable duration for the jobstream.

Alert Criteria

On this tab you configure the alerts that you want AAI to send as soon as it detects that the timely completion of the jobstream is jeopardized. All alerts are displayed in the Monitoring console but you can also add SNMP traps (provided you have the required infrastructure) and emails. The alerts include detailed information about the jobstream run.

Alerts are a key jobstream function that is described in a dedicated topic. For more information, see Configuring Alerts in Jobstreams.

Deleting Jobstreams

To delete a jobstream, select it from the list and click the delete button. If a jobstream exists in more than one business area, it is deleted from ALL business areas.

See also:

Jobstreams