Automic Automation / AWS EMR Agent Integration

Amazon EMR ( Elastic Map Reduce ) is an AWS-based platform service that processes large-volume datasets using shared computing frameworks such as Apache Hadoop and Apache Spark. It facilitates the users in quickly setting up, configuring, and scaling virtual server clusters for analyzing and processing vast amounts of data efficiently.

An Amazon EMR Studio is an Amazon EMR resource that you create for a team of users. Each Studio is a self-contained, web-based integrated development environment for Jupyter notebooks that run on Amazon EMR clusters. Users log in to a Studio using corporate credentials.

Workspaces are the primary building blocks of Amazon EMR Studio. To organize notebooks, users create one or more Workspaces in a Studio.

When you use a Workspace, EMR Studio autosaves the cells in notebook files at a regular cadence in the Amazon S3 location that is associated with your Studio.

Integrating Automic Automation with AWS EMR allows you to start, cancel, and monitor an AWS EMR Job from a definition and, if required, to override parameters set in the job definition from Automic Automation.

Important!

The AWS EMR Agent integrates the target system with Automic Automation. This means that the Agent resides within an Automic Automation system, and it gives you access to all its functions and features. If you are not familiar with Automic Automation or if you need specific information about any of its functions, use this Agent-specific help in combination with the Automic Automation product documentation that you can find at Automic Automation documentation.

The Agents and the Automation Engine communicate using TLS/SSL, which requires the use of trusted certificates to verify the identity of the respective communication partners.

When an Agent communicates with an on-premises Automic Automation system, it establishes the connection to the Java communication process (JCP). When communicating with the Automic Automation Kubernetes Edition, it establishes the connection to an ingress / HTTPS load balancer. Regardless of the environment you use, you have to make sure that you have the required certificates in place.

Important!

Make sure you are familiar with the TLS/SSL and certificate implementation before installing and/or upgrading the respective component. For more information, see the Automic Automation documentation at:

Note:

This integration capability, like all integrations of the Automic system, can support service orchestration Workflows. Such Workflows orchestrate automated processes that run across multiple platforms, domains, and applications to deliver a specific IT service. For more information, see About Service Orchestration in the Automic Automation help, which is available on docs.automic.com.

Important Consideration

To be able to implement (install and configure) and work with the Agent Integration, a sound knowledge of Automic Automation is required. If you are new to Automic Automation, we strongly recommend that you take the following courses before beginning with this integration:

For all users:

For administrators:

For developers and object designers:

Accelerated Learning with Hands-on Simulation Labs:

Beyond the product documentation and courses, Broadcom offers a range of educational materials designed to guide you through Automic Automation's features and functions. Our interactive learning center is a dedicated platform offering immersive simulation labs to help you learn by doing. They are dynamic, hands-on exercises that complement the content of our courses. They don't just tell you how to use a feature; they let you experience it directly.

Visit the Broadcom's Simulation Labs website to explore the available labs and start practicing: https://simlabs.aod.broadcom.com/simulation/automation?product=Automic.

This documentation describes how to set up the integration and how to work with EMR Add Steps to Cluster Jobs and Start Notebook Execution Jobs.