Defining S3 Download File Jobs

{"URL":["/*.*/awa/pa_view_pa_view_DOWNLOAD_JOB_s3"],"heroDescriptionIdentifier":"ice_hero_Download_File_Job_S3","customCards":[{"id":"ice_specific_Download_File_Job_S3","title":"Download File Job Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DownloadFile_Job.htm","languages":["en-us"]},{"id":"ice_failed_Download_File_Job_S3","title":"Failed Operations Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DownloadFile_Job.htm","languages":["en-us"]},{"id":"ice_permissions_Download_File_Job_S3","title":"File Permission Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DownloadFile_Job.htm","languages":["en-us"]},{"id":"ice_AWS_Encryption_Download_File_Job_S3","title":"Defining AWS S3 Server-Side Encryption Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DownloadFile_Job.htm","languages":["en-us"]},{"id":"ice_RA_Integration_Report","title":"RA / Integration Reports","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_RA_Page.htm","languages":["en-us"]},{"id":"ice_script_Download_File_Job_S3","title":"Setting Download File Job Properties through Scripts","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_Script.htm","languages":["en-us"]},{"id":"ice_related_information_Download_File_Job_S3","title":"Related Information","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DownloadFile_Job.htm","languages":["en-us"]}]}

This job allows you to download the S3 file from the specified bucket into your local machine. If the file already exist in the host, the file is overwritten.

On the Download File Job section, you define the parameters relevant to run that job on the S3 system from Automic Automation.

Note:

Some fields allow you to open a picker dialog from where you can select the file and the bucket. By default, only 200 entries are displayed. If the relevant file or bucket is not displayed among those 200, please type in the relevant name on the Search field to narrow down the list.

  • Connection

    Select the S3 Connection object containing the relevant information to connect to the Simple Storage Service system.

    To search for a Connection object, start typing its name to limit the list of the objects that match your input.

  • Region

    Defining the region in the job is optional and only relevant for AWS. If you choose not to define it, the job takes the URL defined in the Connection object.

    However, if you decide to define the region in the job, make sure that the definition matches the one defined in the Connection object that you have selected for the job. If both region definitions do not match, the job execution fails and an error message is logged in both, the Agent (PLOG) and the Job (REP) reports (see Monitoring S3 Jobs).

  • Bucket Name

    Define the bucket name from where the file must be downloaded. You can click the browse button to the right of the field to open a picker dialog where you can select the relevant name.

  • Source File Path

    Define the source file name or the path of where the source file is located. You can define the path using either <file name> or <folder name>/<file name>

    You can also click the browse button to the right of the field to open a picker dialog where you can select the relevant file path.

  • Destination File Path

    Define the local destination file name (for example, C:\temp\text.xml) or the path on the Agent machine to where the file should be downloaded.

S3 File Permission Parameters

In this section, you define the parameters that determine who can access and modify the downloaded files and which permissions they will have.

  • File Permission

    File permissions differ depending on the OS you are using. Those OS differences also apply when granting permissions to a user other than the one running the job.

    Windows

    These options are available:

    • Allowed values:

      • R = Read

      • W = Write

      • X = Execute

    • Format: Any combination of the allowed values (RWX, RW, R,W, X), in uppercase

    UNIX

    Each file is associated with an owner and a group and assigned with permission access rights for three different users: file owner, group members, and others, in that order.

    • Format:chmod

    • Allowed values: 000 to 777, where:

      • 0 = ---

      • 1 = --x

      • 2 = -w-

      • 3 = -wx-

      • 4 = r-

      • 5 = r-x

      • 6 = rw-

      • 7 = rwx

    Using this numeric method allows you to specify which users are allowed to read the file, write to the file, or execute the file.

    Examples

    The combination 640 gives the file owner read and write permissions, read permissions to group members and no permissions to all other users:

    • 6 (rw-) on the first position refers to the file owner permissions

    • 4 (r-) on the second position refers to the group members

    • 0 (---) on the third position refers to others

    Important!

    The job fails when diverging from the allowed file permissions. Also, on Windows, the job fails when both file owner and file owner group are not filled in but there are file permissions specified.

  • File Owner

    Owner of the downloaded file who will have the specified permissions.

  • File Owner Group

    Owner group as specified on your Unix environment. This group will have the specified permissions.

  • On Invalid Permission Settings

    Select the relevant option to define the job behavior when the permission settings are not set correctly.

    • job fails, in these cases:

      • When diverging from the allowed file permission parameters

      • On Windows, when the file owner and the file owner group are not filled in but there are file permissions specified

    • job succeeds, thus ignoring incorrect permission settings

    Default: Job fails

Note:

The Pre-Process page allows you to define the settings all S3 Jobs using script statements. These statements are processed before the Schedule Job is executed, see Setting S3 Job Properties Through Scripts.

Working with S3 File Permissions

There are different scenarios that could be relevant when downloading jobs and working with file permissions. The table below outlines the most common ones.

Important!
  • When assigning ownership to user B, consider that User A can be either a privileged or a standard user. If user A is an admin or root user, changing the file ownership to user B will not impact user A's rights. However, in Windows OS, if it is a standard user, then A will no longer be able to access the file.

  • In Windows OS, while designing the job, make sure that the directory file system permissions for the file owner or Agent user (default) are set correctly so that they have RWX access on the respective destination. If the relevant users do not have read permissions, then they will have problems later on accessing that file.

  • In UNIX OS, while designing the job, make sure that the Agent user (process owner) has root access rights; otherwise, the file ownership cannot be changed and the job will fail.

Scenario Permission Owner Group Behavior
1 blank blank blank

Windows and UNIX: No impact of file permissions or ownership.

  • The Agent user is set as the file owner

  • The Agent group is set as the file group

2 blank blank provided

Windows:

  • The Agent user is set as the file owner of the file with the permissions inherited from the parent directory (as set in Windows OS)

  • The new group is added with the default permissions (as set in Windows OS, by default read and execute)

UNIX: The permissions are set based on the Agent definition.

  • The Agent user is set as the file owner

  • The specified group is set as the file group

3 blank provided blank

Windows:

  • The new owner is the file owner of the file with the permissions inherited from the parent directory (as set in Windows OS)

  • The Agent group is the file owner group of the file with the permissions inherited from the parent directory (as set in Windows OS)

UNIX: The permissions are set based on the Agent definition.

  • The specified user is set as the file owner

  • The Agent group is sent as the file group

4 blank provided provided Windows:
  • The new owner is the file owner of the file with the permissions inherited from the parent directory (as set in Windows OS)

  • The new group is added with the default permissions (as set in Windows OS, by default read and execute)

UNIX:

  • The new owner is the file owner of the file with the umask setting of the Agent definition

  • The new group is the file owner group of the file with the umask setting of the Agent definition

5 provided blank blank

Windows: The job fails.

UNIX: The permissions are set as specified.

  • The Agent user is set as the file owner

  • The Agent group is set as the file group

6 provided blank provided Windows:
  • The Agent user is set as the file owner of the file with the specified permissions

  • The new group is added with the specified permissions

UNIX: The permissions are set as specified.

  • The Agent user is set as the file owner

  • The specified group is set as the file group

7 provided provided blank Windows:
  • The new owner is the file owner of the file with the specified permissions

  • The new group is added with the specified permissions

UNIX: The permissions are set as specified

  • The specified user is set as the file owner

  • The Agent group is set as the file group

8 provided provided provided Windows:
  • The new owner is the file owner of the file with the specified permissions

  • The new group is added with the specified permissions

UNIX: The permissions are set as specified

  • The specified user is set as the file owner

  • The specified group is set as the file group

9 invalid provided blank

Windows and UNIX: The job fails.

 

See also: