Connectors for Universal Schedulers

The AAI integrations for Airflow, Automic Automation, Control-M, and ESP Workload Automation with schedulers (workload automation engines) happens through connectors, which use the same Universal Framework.

As a mainframe scheduler, ESP requires you to install additional ESP specific components on the mainframe to provide a data feed mechanism to the External Connector. The Connector passes the structured scheduler data onto AAI.

As an AAI administrator, you make sure your system meets the necessary requirements. You also install and configure the relevant connector. The installation of these connectors is very similar and they can be installed in Windows and Unix systems. For installation and configuration instructions, see:

• Setting Up the Airflow Connector

• Setting Up the Automic Connector

• Setting Up the Control-M Connector

• Setting Up the ESP Connector

Overview

The Connector for a universal scheduler establishes the communication between the respective workload automation engine (Airflow, Automic Automation, ESP, Control-M) and AAI. It is a stand-alone component and, as such, it runs in its own process space, has its own installer and writes its own log files. It consists of two main parts:

1. The universal connector framework, which handles the communication between the Connector and AAI. It also triggers, or determines, when the Connector fetches job definitions and events (executions) from the target system.

2. A mapper, which extracts job definitions and events (executions) from the target automation engine and translates them into a format that AAI can process.

The connector has different settings that can be modified at any time. For example, you can define the following behaviors:

• The interval in which the job definitions and events (executions) are fetched. You can set the interval for fetching both individually.

• Upon starting, how far back should the Connector look the first time it fetches information.

• How far forward should the Connector look, thus gathering information on planned start times.

Once the Connector is installed and running for the first time, it reaches to AAI and registers itself. It also reads either job definitions or events (executions) from the target system, translates them into the relevant format and passes the information to the framework, which then passes the information to AAI.

Downloading the Product or Solution

Make sure you have downloaded the relevant product bundle or solution. For more information on how to download the relevant component(s), see Downloads.

Java Requirements

Make sure that Java JDK 1.8.x is installed on the AAI application server and configure the JAVA_HOME environment variable accordingly.

For more information, see Compatibility Information.

Automic Automation Performance

In Automic Automation, be aware that the database increases in size when using AAI in combination with the security audit log function in Automic Automation. For more information, see the SECURITY parameters in the UC_CLIENT_SETTINGS variable in the Automic Automation documentation at https://docs.automic.com/.

If you use the AE DB Unload utility to reorganize your Automic Automation database, make sure that you turn off the Automic Connector in AAI while the utility runs.

Retrieving Data

While the Control-M Connector connects with the Control-M database, the Airflow and Automic Automation Connectors use the corresponding REST API and AAI to periodically extract job definitions and executions (current and historical runs) from the respective target system and import them into AAI.

The AAI / ESP integration is unique. It requires some work in USS and an AAI connector to be installed in a distributed system, similar to the Automic Automation and Control-M integrations. However, it also requires the deployment of a number of components on the mainframe, similar to the CA7 and IWSz integrations.

The ESP Connector resides in the distributed system and submits data request to ESP and handles deliveries in return.

AAI uses the respective connector to request the following types of data:

• Definition data, which comprises jobs, schedules, and events that are defined in the target system.

  ESP produces two reports that, combined, contain the definition data that AAI requires:

  • Schedule Activity Report (SADGEN and JOBMAP commands) produce a data set based on job and application names, triggering events, external jobs, and so on.

  • Job Activity Report (MAPGEN and JOBMAP commands) produce a data set of definitions of jobs mapped to events. The data used in this case also includes agents, predecessors, successors, and so on.

• Event data, which is operational data such as execution, times and status, that is fed to AAI in real-time from the target system.

  ESP uses the LiveData Service, the ESP Adapter and the Message Service Hub, all running on the mainframe.

  Important!

  The LiveData Service is AAI specific and is disabled by default. Make sure you enable it on the ESPPARM data set.

Supported Automic Automation Objects

The Automic Automation integration into AAI supports the following objects:

• Events (EVNT)

  Event objects allow you to monitor certain conditions and, if they apply, to automatically trigger actions. These actions are usually the execution of other objects, as defined for the specific Event object.

  Note:

  The execution of an Event object triggers the execution of other objects, which in turn trigger the execution of further tasks. The tasks resulting from them are children of the Event task and are flagged with a special tag type called !EVNT. They are not supported in AAI.

  For more information, please refer to the Events (EVNT) section of the Automic Automation documentation at https://docs.automic.com/.

• C_PERIOD

  Runtime objects that are executed at intervals that are shorter than one day. C_PERIOD stands for period container and refers to the cyclical executions of objects that have the execution type Execute Recurring.

• File Transfers (JOBF)

  File Transfer objects let you exchange any file from one system to another. The transfer can be structured, thus enabling the exchange of files with packed and binary fields in heterogeneous system environments.

• Jobs (JOBS)

  An Automic Automation Job executes commands on computers or in enterprise business solutions (SAP, PeopleSoft, Oracle Applications, etc.). These solutions differ from each other; therefore, specific Job templates are available for each of them. Jobs are always assigned to Agents and they always need a Login object that submits the necessary credentials to the target system (Agent).

• Job Groups (JOBG)

  A Job Group is a container for other objects. It helps you manage the execution of the individual objects it contains. Job Groups can be stand-alone objects that you execute manually or you can insert them in Schedules or Workflows.

• Workflows (JOBP)

  Workflows serve as containers for objects that must be executed in a specific sequence and with specific parameters. An object that is inserted in a Workflow is called task. A Workflow can also be embedded in another Workflow. By linking the tasks in a Workflow, you establish the sequence of the executions.

  Note:

  Only STANDARD workflows are supported. JOBP IF (IF statements) and JOBP FOREACH (loops) are not supported. For more information, please refer to the Automic Automation documentation at https://docs.automic.com/.

• Schedules (JSCH)

  Schedule objects allow you to design time and event-driven task management. They are frames where you collect tasks that you want to execute automatically at regular intervals. Schedule objects determine scheduling parameters, such as the periodicity with which the tasks are executed and the times at which they start. They also let you modify the properties of the tasks that they contain. Such changes apply to the tasks only when processed within that particular Schedule; the objects themselves are not affected.

• Scripts (SCRI)

  Script objects let you write and reuse scripts that provide internal processing instructions. The scripts in Script objects are executed in the Automation Engine itself, and not on target systems.

• Sync (SYNC)

  Sync objects help coordinate the processing of multiple executable objects. They are synchronization mechanisms that let you assign logical resources to certain processes based on how much they consume. Then they regulate the process flow by consuming and releasing these resources.

  You use Sync objects to define states and status transitions that you link to actions and (optionally) to values. These definitions control when and how executable objects will be processed. You then assign the Sync object to the executable objects you want to control this way. You can assign a Sync object to as many other objects as you need. As soon as the executable objects are activated, the system checks what is the current state of the Sync object and, depending on the action (and value, if set) assigned to that state, the executable object is processed or not.

For more information, please refer to the Automic Automation documentation at https://docs.automic.com/.

You can use those Automic Automation objects to carry out the following actions in AAI:

• Create (SLA) for different jobstream setups (simple, complex and/or with external dependencies)

• Locate a possible bottleneck in a jobstream (critical path)

• Create dashboards

Supported Control-M Objects

The Control-M integration into AAI supports the following objects:

• Jobs

  A Control-M Job is a definition of a command to run. This definition includes the conditions, execution frequency, and the target machine in which to run the command.

• Events

  A Control-M Event is the definition of a Job's state change, error, or success. These are used to determine if jobs succeeded, failed, and/or were retried. Dependent jobs occasionally use these events as conditions.

• Folders (Smart, Simple, Complex and so on)

  A Control-M Folder is a container definition for Jobs and subfolders. Folders enable you to configure various settings such as scheduling, event management, resources, or notifications. Folder-level definitions are inherited by the Jobs or subfolders within the folder.

• Resources

  A Control-M Resource is something that a Control-M Job depends on and use. Control-M Resources can be Logical or Quantitative. Logical Resources include physical drives, tables, or databases. Quantitative Resources is a physical or logical entity that you can count, such as a CPU, or Table/Data set.

• Node

  A Control-M Node is a host computer that a Job is sent to, to be executed. Each Node is running a Control-M Agent that orchestrates and monitors the Job and its result.

For more information, please refer to the Control-M documentation at documents.bmc.com.

You can use those Control-M objects to carry out the following actions in AAI:

• Create (SLA) for different jobstream setups (simple, complex and/or with external dependencies)

• Locate a possible bottleneck in a jobstream (critical path)

• Create dashboards

See also:

• Preparing for the Installation

• Schedulers