Automic Automation / Databricks Agent Integration

Databricks is a cloud-based, fully managed first-party service that utilizes an open Lakehouse architecture for building, deploying, sharing, and maintaining enterprise-grade data solutions at scale. The Databricks Lakehouse Platform integrates with cloud storage and security in your cloud account and manages and deploys cloud infrastructure on your behalf.

Integrating Automic Automation with Databricks allows you to run jobs, start and stop cluster jobs, and monitor them in your Databricks workspace from Automic Automation.

Important!

The Databricks Agent integrates the target system with Automic Automation. This means that the Agent lives within an Automic Automation system and it gives you access to all its functions and features. If you are not familiar with Automic Automation or if you need specific information about any of its functions, use this Agent-specific help in combination with the Automic Automation product documentation that you can find at Automic Automation documentation.

The Agents and the Automation Engine communicate using TLS/SSL, which requires using trusted certificates to prove the identity of the respective communication partner.

When an Agent communicates with an on-premises Automic Automation system, it establishes the connection to the Java communication process (JCP). When communicating with the Automic Automation Kubernetes Edition, it established the connection to an ingress / HTTPS load balancer. Regardless of the environment you use, you have to make sure that you have the required certificates in place.

Important!

Make sure you are familiar with the TLS/SSL and certificate implementation before installing and/or upgrading the respective component. For more information, see the Automic Automation documentation at:

Note:

This integration capability, like all integrations of the Automic system, can support service orchestration Workflows. Such Workflows orchestrate automated processes that run across multiple platforms, domains, and applications to deliver a specific IT service. For more information, see About Service Orchestration in the Automic Automation help, which is available on docs.automic.com.

Release Cycles

The release cycles of this Agent Integration are independent of the Automic Automation ones.

For more information, see Compatibility Information.

The Databricks Integration version 21.1.0 will support Automic Automation version 21.0.8 and later.

Important Consideration

To be able to implement (install and configure) and work with the Databricks Agent Integration to configure Run Jobs, Start or Stop Cluster Jobs, and include them in other Automic Automation objects, a sound knowledge of Automic Automation is required.

If you are new to Automic Automation, we strongly recommend that you take the following courses before beginning with this integration:

For all users:

For administrators:

For developers and object designers

This documentation describes how to set up the integration and how to work with Databricks jobs.