Creating MapReduce Jar Jobs

To run a MapReduce Jar Jobs with the RA Hadoop Agent, you need to create MapReduce Jar Jobs.

To create an RA Hadoop Agent MapReduce Jar Job:

  1. Add a Job object of type Hadoop > MAPREDUCEJAR and select your RA Hadoop Agent in the Host field on the Attributes page.
  2. Go to the Hadoop page.
  3. Respond to the fields in the General section.
    • Connection

      • An RA Hadoop Agent Connection object. For more information on RA Hadoop Agent Connection objects, see topic Creating RA Hadoop Agent Connection Objects.

    • Write Response to Job Log

      • When checked, the RA Hadoop Agent prints the whole response to the Job log.

    • Create Request Reports

      • Creates and registers a report with the request data.

    • Create Response Report

      • Creates and registers a report with the response data.

    • Connection Timeout

      • The number of seconds before timing out while attempting to connect to the URL endpoint. When set to 0, the connection never times out.When no timeout is specified, the Job's Connection object setting is used.

    • Read Timeout

      • The number of seconds before timing out when waiting for a method call. When set to 0, the read never times out.When no timeout is specified, the Job's Connection object setting is used.

    • Trace Performance Metrics

      • When checked, statistics on how long the call took are written to the Job report. Additionally, the following object variables are set with the metrics that can be reported upon:

      • COMPILE_REQUEST_DURATION
      • COMPILE_RESPONSE_DURATION
      • ROUNDTRIP_DURATION

    • URL Endpoint

      • Dynamically lists the full URI in a non-editable field.

  4. Respond to the fields in the Parameters section.
    • Jar File

      • The .jar file that has the code that extracts the map reduce you want to do. You can use the folder icon to browse to the file.

    • Class Name

      • The class within the selected .jar file.

    • User Name

      • The Hadoop user name.

    • Status Directory

      • The Hadoop status directory. You can use the replacement value {runid} to resolve to the run Id of the Job. You can use the folder icon to browse to the file.

    • Files

      • Comma separated files to be copied to the map reduce cluster.

    • Lib Jars

      • An additional .jar file, if required for the class. You can use the folder icon to browse to the file.

    • Enable Log

      • Enables Hadoop logging.

  5. Optionally define arguments and response defines.
  6. Click Save to save the Job.

Post Processing Variables

The following variables are available in the Job's Post-Process page.

For the: Use the Variable:
Exit value &exitValue#
Status name &statusName#

Workflow Variables

The following variables are available for the next Job, when this Job is included in a Workflow.

For the: Use the Variable:
Standard output &stdout#
Standard error output &stderr#
Status directory &statusdir#