Automic Automation / Google Cloud Dataflow Agent Integration

Dataflow is a Google Cloud service that provides unified stream and batch data processing at scale. Use Dataflow to create data pipelines that read from one or more sources, transform the data, and write the data to a destination.

Typical use cases for Dataflow include the following:

  • Data movement: Ingesting data or replicating data across subsystems.

  • ETL (extract-transform-load) workflows that ingest data into a data warehouse such as BigQuery.

  • Powering BI dashboards.

  • Applying ML (machine learning) in real time to streaming data.

  • Processing sensor data or log data at scale

Integrating Automic Automation with Google Cloud Dataflow allows you to start Template Jobs in your Dataflow system from Automic Automation.

The Agents and the Automation Engine communicate using TLS/SSL, which requires using trusted certificates to prove the identity of the respective communication partner.

When an Agent communicates with an on-premises Automic Automation system, it establishes the connection to the Java communication process (JCP). When communicating with the Automic Automation Kubernetes Edition, it established the connection to an ingress / HTTPS load balancer. Regardless of the environment you use, you have to make sure that you have the required certificates in place.

Important!

Make sure you are familiar with the TLS/SSL and certificate implementation before installing and/or upgrading the respective component. For more information, see the Automic Automation documentation at:

Note:

This integration capability, like all integrations of the Automic system, can support service orchestration Workflows. Such Workflows orchestrate automated processes that run across multiple platforms, domains, and applications to deliver a specific IT service. For more information, see About Service Orchestration in the Automic Automation help, which is available on docs.automic.com.

Release Cycles

The release cycles of this Agent Integration are independent of the Automic Automation ones.

For more information, see Compatibility Information.

Important Consideration

To implement (install and configure) and work with the Dataflow Agent Integration (configure Run jobs and include them in other Automic Automation objects), a sound knowledge of Automic Automation is required.

If you are new to Automic Automation, we strongly recommend that you take the following courses before beginning with this integration:

For all users:

For administrators:

For developers and object designers

This documentation describes how to set up the integration and how to work with Dataflow jobs.