Defining S3 Delete File Jobs

{"URL":["/*.*/awa/pa_view_pa_view_DELETE_JOB_s3"],"heroDescriptionIdentifier":"ice_hero_Delete_File_Job_S3","customCards":[{"id":"ice_specific_Delete_File_Job_S3","title":"Delete File Job Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DeleteFile_Job.htm","languages":["en-us"]},{"id":"ice_failed_Delete_File_Job_S3","title":"Failed Operation Parameters","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DeleteFile_Job.htm","languages":["en-us"]},{"id":"ice_RA_Integration_Report","title":" RA / Integration Reports","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_RA_Page.htm","languages":["en-us"]},{"id":"ice_script_Delete_File_Job_S3","title":"Setting Delete File Job Properties through Scripts","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_Script.htm","languages":["en-us"]},{"id":"ice_related_information_Delete_File_Job_S3","title":"Related Information","type":"customize","url":"https://docs.automic.com/documentation/webhelp/english/ALL/components/IG_S3/*.*/Agent%20Guide/Content/S3/S3_DeleteFile_Job.htm","languages":["en-us"]}]}

This job allows you to delete a file from the S3 bucket. A file can only be deleted if it exists in the specified bucket. However, if the file that you have specified does not exist, the job does not fail. It only fails if the region or bucket name do not exist.

This page includes the following:

S3 Delete File Job Parameters

On the Delete File Job section, you define the parameters relevant to run that job on the S3 system from Automic Automation.

Note:

Some fields allow you to open a picker dialog from where you can select the file and the bucket. By default, only 200 entries are displayed. If the relevant file or bucket is not displayed among those 200, please type in the relevant name on the Search field to narrow down the list.

Warning!

When using regular expressions, make sure you consider the following issues:

The File Name changes to a File Pattern. This means that you can no longer define a single file name.
The AE REST API handles the regex on the background. That means that executions of jobs with regular expressions might take longer. You can use the Query Params to refine the search and reduce the matches thus significantly increasing the performance.

Connection

Select the S3 Connection object containing the relevant information to connect to the Simple Storage Service system.

To search for a Connection object, start typing its name to limit the list of the objects that match your input.
Region

Defining the region in the job is optional and only relevant for AWS. If you choose not to define it, the job takes the URL defined in the Connection object.

However, if you decide to define the region in the job, make sure that the definition matches the one defined in the Connection object that you have selected for the job. If both region definitions do not match, the job execution fails and an error message is logged in both, the Agent (PLOG) and the Job (REP) reports (see Monitoring S3 Jobs).
Bucket Name

Define the bucket name from where the file must be deleted. You can click the browse button to the right of the field to open a picker dialog where you can select the relevant name.
Use Regex

This option is not selected by default. Select the checkbox if you want to use a regular expression to delete multiple files at the time.
File Name

This option is available only if you have not selected the Use Regex option.

Define the name of the file to be deleted on the bucket. The file name definition is case sensitive. You can click the browse button to the right of the field to open a picker dialog where you can select the relevant name.

If the bucket has folders and sub-folders, you must enter the entire path here. For example <folder name>/<file name>.
File Pattern

This option is available only if you have selected the Use Regex option.

Enter the regular expression that define the file pattern matching the files that you want to delete.
Query Param

Allows you to filter the query and therefore the query response.

Examples

When using a regex, the prefix parameter allows you to optimize the search and get results that are more efficient. For example, if your bucket has the following files:

/opt/files/example_04_08.pdf

/opt/files/example_05_08.pdf

/opt/files/example_06_08.pdf

/opt/files/demo_07_08.pdf

/opt/files/demo_08_08.pdf

If you want to check for files starting with example and with a .pdf extension, you can specify .*.pdf on the File Name field and enable the Use Regex option.

You can further specify the query using the following query parameter:

prefix=/opt/files/example.

You can also send multiple query parameters using the format <param1>=<value1>&<param2>=<value2>.

For example, you can add the list-type=2 parameter to use version 2 of the AWS API operation:

prefix=/opt/files/example&list-type=2

There are no restrictions on the parameters that you can use in a query. All URI request parameters for the Delete File Job are supported. For more information, refer to the official AWS S3 and GCS documentations, respectively.

Note:

The Pre-Process page allows you to define the settings all S3 Jobs using script statements. These statements are processed before the Schedule Job is executed, see Setting S3 Job Properties Through Scripts.

S3 Failed Operation Parameters

The Failed Operation Parameters section allows you to define if you want to try to execute failed operations again or not. There are a number of parameters that you can define to do so:

Retry

Select the checkbox if you want to retry to execute failed operations. Once selected, you can define the following:
- Retry Count: Define the number of retry attempts.
- Retry Delay: Define the time in seconds between attempts.