Implementation Overview for AWS Glue

There are different components required to integrate your Automation Engine and AWS Glue so that you can trigger and monitor Glue Crawler, Job Run, Trigger, Blueprint and Workflow jobs from your Automic Automation system.

As a system administrator, you carry out the steps required to integrate Automic Automation and AWS Glue.

These are the steps you have to follow for the integration:

Download the AWS Glue Agent Installation Package

You can download the relevant Agent installation package from our download center at https://docs.automic.com/documentation or from our marketplace at https://marketplace.automic.com/.

For more information, see Downloads and Marketplace.

Install the AWS Glue Agent

After downloading the relevant installation package, you have to install the Agent so that you can establish the communication between the two systems.

Important!

Before installing the Agent, check the version compatibility information at compatibility matrix. For information about how to use the compatibility matrix, see Compatibility Information.

You can install the Agent manually or in a container. The containerized Agent installation refers to building and running a Docker image hosting Java Agents in a container.

Regardless of how you install the Agents, you can use them in on-premises Automic Automation systems as well as in an Automic Automation Kubernetes Edition system.

For more information, see the Automic Automation documentation:

For more information on how to install the AWS Glue agent, see Installing the Agent for AWS Glue.

Define the AWS Glue Agent

Once you have installed the Agent, you have to configure its INI file to cover your system's requirements, for example, the connection between the Agent and the Automation Engine instance. Then, you define the Agent object in AWI. There are definition pages that are common to all Agents and some that are Agent-specific.

The Agent definition allows you to configure the communication between the Automation Engine and the target system on which your Glue jobs run. The Agent starts the execution of these jobs and makes both their monitoring and the corresponding reporting possible.

For more information, see Defining the AWS Glue Agent.

Establish the Connection to AWS Glue

Now you have to make sure that the log in data required to establish the connection to the AWS Glue system is available, otherwise you cannot create and execute Glue jobs without leaving the Automic Web Interface.

To do so, you have to create and define AWS Glue Connection objects.

For more information, see Establishing the Connection to AWS Glue.

Define the AWS Glue Jobs

Now that the Agent is installed and defined and both systems can communicate with each other, you have to create and define the Glue jobs so that you can execute the relevant processing steps in the target system.

Once the jobs are defined, you can execute it and monitor its progress from the Automic Web Interface.

For more information, see Defining AWS Glue Job Properties.

See also:

Automic Automation / AWS Glue Agent Integration