Establishing the Connection between the MWAA Agent and the Target Environment

{"URL":["/*.*/awa/pa_view_pa_view_CONN_mwaa"],"heroDescriptionIdentifier":"ice_hero_CONN_MWAA","customCards":[{"id":"ice_specific_CONN_MWAA","title":"Defining the Connection Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_MWAA/*.*/Agent%20Guide/Content/Airflow_MWAA/MWAA_Connection.htm","languages":["en-us"]},{"id":"ice_GoogleCloudStorage_S3_Authentication_video_CONN","title":"Watch the Video: Authentication Methods","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_MWAA/*.*/Agent%20Guide/Content/_CommonTopics/Video_GoogleAuthMethods.htm","languages":["en-us"]},{"id":"ice_permission_GCP_IAM_CONN_MWAA","title":"IAM Permissions for MWAA","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_MWAA/*.*/Agent%20Guide/Content/Airflow_MWAA/MWAA__IAM_Permissions.htm","languages":["en-us"]},{"id":"ice_Proxy_CONN_MWAA","title":"Defining the Proxy Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_MWAA/*.*/Agent%20Guide/Content/Airflow_MWAA/MWAA_Connection.htm","languages":["en-us"]},{"id":"ice_related_information_CONN_MWAA","title":"Related Information","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_MWAA/*.*/Agent%20Guide/Content/Airflow_MWAA/MWAA_Connection.htm","languages":["en-us"]}]}

The communication between the MWAA Agent and the target system is established by Connection objects. The MWAA Connection object contains the parameters (target system endpoint, credentials and so on) required to authenticate on and connect to both the Airflow instance and the AWS environment on which Airflow is running.

As an administrator user, you create the MWAA Connection object. As a developer or object designer, you assign the respective MWAA Connection object to the Automic Automation Run DAG Jobs to execute and monitor the jobs on the target cloud solution without leaving Automic Automation. This topic explains how to configure the MWAA Connection object.

Tip:

To configure an Automic Automation MWAA Connection object you will need the authentication data (credentials, endpoints, tokens and so forth) that enable the login to the target cloud solution. You can get this data from the team in your organization that is responsible for maintaining the target cloud solution.

The Connection object definition consists of an Agent-specific page and pages that are common to all Connection objects.

Adding an MWAA Connection Object

  1. In the Automic Web Interface, go to the Process Assembly perspective. It opens to the Explorer view that contains the list of jobs that are available to you in your system.

  2. You have two options:

    • Right-click anywhere on the list and select Add > Add Object.

    • Click the Add Object button on the toolbar.

  3. On the Add Object dialog, start typing MWAA in the Search field.

  4. Select MWAA Connection and click Add. The Object Name dialog is displayed.

  5. Enter a descriptive Name.

  6. Optionally, enter a short and descriptive Title that helps you recognize the purpose of the object.

  7. Click OK. A new page opens where you can start with the object definitions.

MWAA Page

The MWAA page consists of two sections where you can define the relevant parameters, the Connection section and a Proxy section.

Connection Section

Here you define the parameters relevant for the connection to the AWS system.

  • Endpoint

    Airflow URL as available in your AWS > MWAA > Environments > Environment detail page. For example:

    https://71d5e1b4-9c6a-47d9-995e-8824934d2.c46.us-east-1.airflow.amazonaws.com/aws_mwaa/cli

  • Region

    Optionally, define the region in the AWS account where Airflow resides. For example:

    us-east-1

    If you do not define the region, the system uses the URL defined in the Endpoint.

  • Authentication Endpoint

    URL that identifies the Environment where the Airflow instance resides. It contains the Environment name and the AWS region. The Authentication Endpoint is necessary to authenticate to the environment and to create the cli token that will be used for performing further actions on the DAGs.

    For example:

    https://env.airflow.us-east-1.amazonaws.com/clitoken/MyAirflowEnvironment

  • Authentication Type

    Method with which you authenticate to AWS. The following options are available:

    AWS Credentials File Path

    • Profile Name

      AWS profiles store multiple AWS credentials. Enter the name of the profile that contains the credentials for your AWS system.

    • Credentials File Path

      Location of the AWS credentials file on the Agent machine.

      Example:

      Windows: C:\Users\user\Documents\AWS\credentials

      UNIX:/home/user/aws/credentials

    Secret Access Key

    Private unique identifier of user accounts that is used to sign requests to AWS services. It is displayed to the user only during its creation. It consists of an Access Key and a Secret Access Key.

    EC2 Profile Instance

    Allows you to connect to an EC2 VM within an AWS cloud application. In Profile Instance Name enter the name of the profile available on the VM.

    Note:

    To use this option, you must have an EC2 instance profile. For instructions on how to set it up, please refer to the official AWS documentation.

    External Provider

    Allows you to set up single sign-on (SSO) with SAML using either a service or an identity provider.

    • Tenant ID

      Identifier of the AWS tenant.

    • Authentication URL

      URL that identifies the network address of the external authentication provider used to secure the application. Currently, only Azure is supported as external provider so, by default, the URL (https://login.microsoftonline.com) points to the Azure log in. If you do not want to use this URL, make sure you change the default definition.

    • SAML Username

      Username used for SAML authentication when setting up Azure as your AWS identity provider.

    • SAML Password

      Password for the user used for SAML authentication.

    • Principal ARN

      Amazon Resource Name (ARN) of the account's principal.

    • Role ARN

      Amazon Resource Name (ARN) of the role to be assumed by the user.

    • Identity Provider

      Currently, this integration supports Azure only.

    Assume Role

    Allows you temporarily to assume a role and access different services with the credentials that you are assuming.

    • Access Key: Define the access key.

    • Secret Access Key: Define the encrypted access key secret value.

    • Role ARN: Enter the Amazon Resource Name (ARN) of the role to be assumed by the user.

    • Role Session Name: Define a name for the respective session to differentiate it from other sessions that you might have using the same assumed role.

Proxy Section

If the AWS system is behind a proxy server, you define the parameters relevant for the connection to the AWS system in this section.

Specify the following and save the object:

  • Proxy Host Name

    Host name or IP address of the proxy server to which you want to connect.

  • Proxy Port

    Port used by the proxy server.

  • Proxy Username

    User name used to authenticate the proxy server.

  • Proxy Password

    Password of the user used to authenticate the proxy server.

Common Definition Pages

In addition to the Agent-specific connection parameters, you can also specify optional properties. You do so on the following definition pages:

Next Step

As soon as the Connection object is configured, developers and object designers can select it when defining the Run DAG Jobs.

See also: