Defining Azure Synapse Run Pipeline Job Properties

Automic Automation Synapse Run Pipeline Jobs allow you to run and rerun a pipeline and, therefore, all the activities grouped in the pipeline, in your Synapse workspace from Automic Automation.

To run a pipeline successfully, you have to define the pipeline parameters (name, type, default value) that you want to pass to the application in JSON format. These parameters allow you to control the pipeline's behavior and activities, such as passing dataset connection details or the path to a file to be processed.

This page includes the following:

On the Run Pipeline Job page, you define the parameters relevant to run a pipeline in Azure Synapse.

• Connection

  Select the Azure Synapse Connection object containing the relevant information to connect to the application.

  To search for a Connection object, start typing its name to limit the list of the objects that match your input.

• Pipeline Name

  Enter the name of the pipeline that you want to run.

• Pipeline Parameters (Optional)

  Select one of the options available:

  • JSON

    Use the JSON field to enter the JSON payload definition.

    Important!

    There are many options available to define the JSON payload. For more information and examples of the JSON definition, see Defining the JSON for Azure Synapse Run Pipeline Jobs.

  • File Path

    Use the JSON File Path field to define the path to the JSON file containing the attributes that you want to pass to the application. Make sure that the file is available on the Agent machine (host).

  Note:

  The Pre-Process page allows you to define the settings of the Jobs using script statements. These statements are processed before the Pipeline Run Job is executed, see Setting Azure Synapse Job Properties Through Scripts.

This section gives you examples of how you could define the JSON field when defining a Run Pipeline Job. You have different options available.

Simple JSON Definition

The first option to define the JSON field is a simple payload definition. To do so, make sure you define the parameters required to define the pipeline, such as name, type, default value, and so on.

Using Variables

You can also use variables in the payload definition.

Example

In the Pipeline Parameters field, enter the following:

&PIPELINEPARA#

If the variable is not defined yet, you must define it now. You do it on the Variables page of the Run Pipeline Job definition:

(Click to expand)

When you execute the Job, the variables will be replaced with the value you have just defined. This is visible in the Agent log (PLOG), see Monitoring Azure Synapse Jobs.

Run Pipeline Job in a Workflow

You can also use the JSON field if you want to include a Run Pipeline Job in a Workflow and you want to use Automation Engine variables in it.

Example

In the Workflow, a Script object (SCRI) with the variable definition relevant for the Pipeline Name and the Pipeline Parameters precedes your Run Pipeline Job:

(Click to expand)

In the Run Pipeline Job definition, you include those variables:

(Click to expand)

When the Job is executed, the variables will be replaced with the value you have just defined. This is visible in the Agent (PLOG) report, see Monitoring Azure Synapse Jobs.

• Defining Azure Synapse Job Properties

• Defining RA/ Integration Reports Properties

• Defining Common Job Properties

• Automic Automation / Azure Synapse Agent Integration