The Processing Load Data Insight
The Processing Load data insight is a tool to monitor and analyze trends in the processing load on your schedulers and their agents (machines) by showing you load information in both a graphical and a tabular form, as well as giving you totals of execution delays by time interval.
This topic describes the Processing Load data insight, what it contains, and what filter and view options it offers.
For general information about how to work with an individual data insight, see Viewing and Using a Data Insight. For information about creating and maintaining data insights, see Adding, Editing, Copying, and Deleting Data Insights.
This page includes the following:
Purpose
The Processing Load data insight type gives you a clear picture of scheduler performance by showing you the number of jobs that are running during an interval of time as well as the total time in execution delays for each scheduler or machine during each slice of time. The graph, which you can zoom into and out of, reveals spikes and valleys of processing load to help you discover where you might optimize your scheduler and machine loads.
Furthermore, because you can create a processing load data insight for totals by machine, it becomes a tool to support charge-back models to application teams.
Sample
The following is an example of a Processing Load data insight that shows the scheduler load over the last 15 days. Each line represents one scheduler and it plots the number of job starts on the vertical, y-axis at various time intervals on the horizontal x-axis. The differences in the processing load among the schedulers become apparent when you see them visualized like this. The peaks and valleys of the loads also points to possible opportunities for improving your resource distribution.
The table below the graph shows the details per scheduler at each time interval. The data on the table and graphic are synchronized, so that when you change filters or zoom in or out on the graph, the two cover the same data.
Best Practice: Schedule Daily Downloads
Because the Processing Load data insight is a tool for monitoring scheduler and machine performance over time, it can be good to schedule a CSV file to be regularly produced and emailed to yourself and possible also the appropriate departments.
Tips for success:
-
Processing load data is voluminous. Therefore, when you want to email processing loads by machine, create multiple data insights with different filters to reduce the data focus and the volume. Also schedule the emails at short and regular intervals so that the attachments are smaller and easier to receive.
-
Use a clearly identifying subject and description, so that your recipients can set up their email program to automatically archive them to a specified folder.
Configuration Settings
The Processing Load data insight offers data columns, filter options, and group-by options that are relevant to understanding your processing load volume and performance.
Available Columns for the Data Table
The following are the data columns that are available in a Processing Load data insight. You can define filters based on the data in these columns and you can choose to show or hide them in the table. The ones marked with an asterisk (*) are required columns and cannot be hidden. They contain key data or data that is essential to the purpose of a Processing Load data insight.
-
Machine*: Only available when the Group By option in the data insight's Filters settings is set to Machine,. Sometimes this is referred to as a scheduler agent .
-
Scheduler*: The AAI scheduler name for the workload scheduler on which the jobs ran.
-
Start Time*: The time that the event occurred, in the time zone that is defined for this data insight.
-
End Time*: The user whose actions triggered the event.
-
No. of Job Starts: The number of jobs that started within the filtered time interval.
-
Finish-Start Delay: The total amount of time of finish-to-start delays for the jobs on the scheduler or machine in the filtered time interval. A finish-to-start delay is the time that passes after the last predecessor of a job finishes and before the next job starts. High finish-to-start delays indicate scheduler inefficiencies because there is too much wait time that could be used in processing time. You might want to redistribute the workload on your agents (machines).
-
Start-Running Delay: The total amount of time of start-to-running delays of the scheduler or machine in the filtered time interval. This delay time indicates how long the scheduler takes to prepare resources to run the job. High start-to-finish delays can point to scheduler inefficiencies or suboptimal load distribution.
Filter Options
You can define Filters settings for a Processing Load data insight based on the following values:
-
Time: The time span for the data to include in the data insight. Note the following restrictions:
-
Whether you define your time filter with a relative or an absolute date range, you cannot include more than 15 days of load data on a Processing Load data insight.
-
You can only include dates in the past until now. There is no processing load data for a future date.
-
-
Scheduler: Scheduler names
-
Machine: Machine names. This is not available when the Group By option is set to Scheduler. Because the number of agent machines is typically very large, the filter menu for Machine has a search field to help you find machine names.
How Group By affects filtering
In the case of the a Processing Load data insights, you can only filter on Machine names when the Group By option is set to Machine. When it is set to Scheduler, the Machine data is excluded. This is a special characteristic of the behavior of the Processing Load data insight type. For more information, see Group-By Options.
Sort the No. of Job Starts column in descending order to see where your heaviest load is. Sort primarily by Scheduler and then secondarily by No. of Job Starts to see the load amounts per scheduler.
Remember that you or any shared user can modify the sort order dynamically when viewing the data insight. Users who can only view the data insight cannot, however, save a sort order different to the one in the data insight definition. For more information, see Sharing Data Insights.
Group-By Options
The following are the options for the Group By setting of the Processing Load data insight:
-
Scheduler (default)
-
Machine
Although, as with other data insight types, the Group By setting of the Processing Load determines what you see on the graph, this setting works differently than in other data insights in the following ways:
-
You can choose the option only on the Filters tab in its definition. You cannot change it dynamically when viewing the data insight. The reason for this is that the enormous volume of job processing data would take too long to re-group on demand.
-
The option you choose effectively determines the subtype of the Processing Load data insight type, that is, whether it is a scheduler processing load or the machine (agent) processing load data insight.
When you choose Scheduler, you create a scheduler processing load data insight. This means the following:
-
You do not see the Machine names on the data insight graph nor the table. It is not included in the columns for viewing, sorting, nor filtering.
-
The graph shows the number of total jobs starts (on the x-axis) per scheduler (on the y-axis).
When you choose Machine, you create a machine processing load data insight. This means the following:
-
A column for the Machine name is added to the data insight. It is a key value, so the column is always shown. You can sort and filter on machine name.
-
The graph shows the number of jobs starts (on the x-axis) per machine (on the y-axis).
-
Although the scheduler names are always included on the table, you do not see the total job starts for the schedulers. You see those only when the Group By option is Scheduler.
-
If you change the Group By option, it does not remove filter settings for any of the other columns (Scheduler, Start Time, and so on).
Understanding and Using the Processing Load Data Insight
A Processing Load data insight includes a line graph that shows the total number of job starts (the vertical y-axis) during a specific time interval (the horizontal x-axis). Each line on the graph shows the total job starts over time for a scheduler or a machine, depending on the Group By setting. The spikes and valleys show when processing load is particularly high or low. Use that insight to determine where you can redistribute your processing load.
What You See on the Processing Load Data Insight
The following are parts of the Processing Load data insight that are characteristic to it:
-
The Group By option is not editable. It can be set to Scheduler or Machine. You can set this option only on the Filters tab of the data insight definition.
-
The graph shows you the number of job starts on the vertical y-axis and time intervals on the horizontal x-axis. The time intervals are fractions of the time period in the main filter. For example, if the filter is for the last 15 days, the x-axis intervals are days; whereas, if the filter is for the last 24-hours, the x-axis intervals are hours.
-
Each line is a scheduler or machine and each data point is the number of jobs started in the corresponding interval for a specific scheduler.
-
The legend above the graph shows which line color and data point symbol on the graph corresponds to each scheduler or machine (depending on the Group By setting). The six schedulers or machines with the highest number of job starts have their own line, and all the remaining schedulers or machines are summarized in n Schedulers or n Machines. The n number is the total minus the six that are showing.
Tip:If you want to see specific schedulers or machines that are not in the top six and therefore in the summary amounts, use filters to select them.
In general, use filters to manage the large number of Machines when viewing a data insight grouped by Machine.
-
The data table shows the corresponding data to the table. For information, see Available Columns for the Data Table.
What You Can Do on the Processing Load Data Insight
Both the graph and the data table on the Processing Load data insight have interactive features that help you focus on the schedulers or machines and time intervals whose processing load you want to investigate, as well as link you to relevant details. At all times the graph and the data table remain synchronized as you shift your focus. For example, when you change the filter settings, the data in both the graph and table are aligned accordingly.
Interactive features on the graphic
The following are the interactive features on the graph:
-
On the legend, each scheduler name is a toggle. Click the name to remove its data from the graph and table. Click a grayed out scheduler name to include its data in the graph and table again.
-
Mouseover a data point to see a pop-up with details of the time interval and number of jobs started in that scheduler or machine in that time.
-
Use the zoom slider under the graph to focus on a on smaller or greater time range within the filtered time period.
-
You can move from either the left or right to define the span of time that you want to focus on.
-
Alternatively, mouseover anywhere onto the graph, stop and roll your mouse wheel up or down to zoom in or out on the time period, equally from left and right. The upper limit is the time period setting on the mail filter.
-
The interval scale on the horizontal x-axis (daily, hourly, and so on) does not change.
-
The table is filtered to the time span that the slider shows. (Hint: You see the number of rows for the table change accordingly.) Whereas the main filter for the data insight remains unchanged.
-
-
Click a data point to drill down to smaller time intervals.
-
The first click takes you from days to hours, zooming in to the hour of the data point you clicked. The second click takes you from hours to 5-minute intervals of the hour of the data point that you clicked. Notice that the main filter for the data insight changes accordingly. Click Reset next to the main filters to return to days.
-
Notice that, as you move in, the schedulers that are shown in the legend might change because the data insight always shows the schedulers with the six highest job starts and summarizes the rest as the total for n Schedulers or n Machines.
-
The time period filter for the data insight adjusts accordingly, as does the table data.
-
Interactive features on the data table
The following are the interactive features that you can use on the data table:
-
Click the Scheduler name to open a window with its scheduler connection definition. This includes both the Monitoring Details and the Connection Configuration. From that window you can edit or even delete the scheduler definition, if you have rights to do that. After closing the scheduler definition, use the back button on your browser to return to the data insight. For more information, see Schedulers.
-
Click the number in the No. of Job Starts column to open the Job Starts Details windows that contains a list of all the jobs that were included in the total.
-
When the window opens the table is already filtered for the time period that you were viewing on the Processing Load data insight when you opened the window, and for the scheduler that the jobs started on.
-
You cannot change the Time Period filter, but you can filter on Job Name, Machine, and Scheduler.
-
-
The columns that the table can include are Job Name, Start Time, End Time, Machine (this in included regardless of the Group By option set for the data insight), Scheduler.
You can filter the list and configure the columns like any other table. For information, see Using Filters (Web Interface) and Working with Tables (Web Interface).