Defining S3 Download File Jobs

This job allows you to download the S3 file from the specified bucket into your local machine. If the file already exist in the host, the file is overwritten.

This page includes the following:

On the Download File Job section, you define the parameters relevant to run that job on the S3 system from Automic Automation.

• Connection

  Select the S3 Connection object containing the relevant information to connect to the Simple Storage Service system.

  To search for a Connection object, start typing its name to limit the list of the objects that match your input.

• Region

  Defining the region in the job is optional and only relevant for AWS. If you choose not to define it, the job takes the URL defined in the Connection object.

  However, if you decide to define the region in the job, make sure that the definition matches the one defined in the Connection object that you have selected for the job. If both region definitions do not match, the job execution fails and an error message is logged in both, the Agent (PLOG) and the Job (REP) reports (see Monitoring S3 Jobs).

• Bucket Name

  Define the bucket name from where the file must be downloaded. You can click the browse button to the right of the field to open a picker dialog where you can select the relevant name.

• Use Regex

  This option is not selected by default. Select the checkbox if you want to use a regular expression to download multiple files at the time.

• Source File Path

  This option is available only if you have not selected the Use Regex option.

  Define the source file name or the path of where the source file is located. You can define the path using either <file name> or <folder name>/<file name>

  You can also click the browse button to the right of the field to open a picker dialog where you can select the relevant file path.

• Source

  This option is available only if you have selected the Use Regex option.

  Define were the relevant files are located in your bucket, for example MYFILES/*.

• Destination File Path

  This option is available only if you have not selected the Use Regex option.

  Define the local destination file name (for example, C:\temp\text.xml) or the path on the Agent machine to where the file should be downloaded.

• Destination Path

  This option is available only if you have selected the Use Regex option.

  Define the path to the destination to which you want to move the files. You can define the path using <folder name>/.

  Warning!

  Make sure you define a folder and not a single file (path). Otherwise, you will overwrite the contents of the folder with a single file.

• Query Param

  Allows you to filter the query and therefore the query response.

  Examples

  When using a regex, the prefix parameter allows you to optimize the search and get results that are more efficient. For example, if your bucket has the following files:

  /opt/files/example_04_08.pdf

  /opt/files/example_05_08.pdf

  /opt/files/example_06_08.pdf

  /opt/files/demo_07_08.pdf

  /opt/files/demo_08_08.pdf

  If you want to check for files starting with example and with a .pdf extension, you can specify .*.pdf on the File Name field and enable the Use Regex option.

  You can further specify the query using the following query parameter:

  prefix=/opt/files/example.

  You can also send multiple query parameters using the format <param1>=<value1>&<param2>=<value2>.

  For example, you can add the list-type=2 parameter to use version 2 of the AWS API operation:

  prefix=/opt/files/example&list-type=2

  There are no restrictions on the parameters that you can use in a query. All URI request parameters for the Download File Job are supported. For more information, refer to the official AWS S3 and GCS documentations, respectively.

The Failed Operation Parameters section allows you to define if you want to try to execute failed operations again or not. There are a number of parameters that you can define to do so:

• Retry

  Select the checkbox if you want to retry to execute failed operations. Once selected, you can define the following:

  • Retry Count: Define the number of retry attempts.

  • Retry Delay: Define the time in seconds between attempts.

In this section, you define the parameters that determine who can access and modify the downloaded files and which permissions they will have.

• File Permission

  File permissions differ depending on the OS you are using. Those OS differences also apply when granting permissions to a user other than the one running the job.

  Windows

  These options are available:

  • Allowed values:

    • R = Read

    • W = Write

    • X = Execute

  • Format: Any combination of the allowed values (RWX, RW, R,W, X), in uppercase

  UNIX

  Each file is associated with an owner and a group and assigned with permission access rights for three different users: file owner, group members, and others, in that order.

  • Format:chmod

  • Allowed values: 000 to 777, where:

    • 0 = ---

    • 1 = --x

    • 2 = -w-

    • 3 = -wx-

    • 4 = r-

    • 5 = r-x

    • 6 = rw-

    • 7 = rwx

  Using this numeric method allows you to specify which users are allowed to read the file, write to the file, or execute the file.

  Examples

  The combination 640 gives the file owner read and write permissions, read permissions to group members and no permissions to all other users:

  • 6 (rw-) on the first position refers to the file owner permissions

  • 4 (r-) on the second position refers to the group members

  • 0 (---) on the third position refers to others

  Important!

  The job fails when diverging from the allowed file permissions. Also, on Windows, the job fails when both file owner and file owner group are not filled in but there are file permissions specified.

• File Owner

  Owner of the downloaded file who will have the specified permissions.

• File Owner Group

  Owner group as specified on your Unix environment. This group will have the specified permissions.

• On Invalid Permission Settings

  Select the relevant option to define the job behavior when the permission settings are not set correctly.

  • job fails, in these cases:

    • When diverging from the allowed file permission parameters

    • On Windows, when the file owner and the file owner group are not filled in but there are file permissions specified

  • job succeeds, thus ignoring incorrect permission settings

  Default: Job fails

Amazon S3 encrypts your objects at their destination as it writes them in the respective AWS S3 data center and decrypts them when you access them. You can set a default encryption configuration for your buckets. However, you can also override the default bucket encryption and define a different one per object to be stored in an AWS S3 bucket.

You can only apply one type of server-side encryption to an object at the time.

If the file that you want to download was originally uploaded to the bucket using a custom encryption type (SSE-C), you need to provide an algorithm and a key to be able to download it.

Specify Encryption Key Allows you to define that you want to use a custom server-side encryption, when the file that you want to download was uploaded using custom encryption.

Select the check box to define the following parameters:

• Customer Algorithm: AES256 is the only supported algorithm.

• Customer Key: Enter the encryption key that you want to use to execute the job.

• Defining RA/ Integration Reports Properties

• Defining Common Job Properties

• Executing S3 Jobs

• Monitoring S3 Jobs

• Defining S3 Download File Jobs

• Defining S3 Delete File Jobs

• Defining S3 Exist File Jobs

• Defining S3 Monitor File Jobs

• Defining S3 Move File Jobs

• Defining S3 Upload File Jobs