Accessing Software Logs

Cause

It might be necessary to review or monitor (such as using tail, Logstash, Splunk, or Graylog) software logs for error and status information.

Steps

This logging capability is configurable but by default:

  1. See Installed Defaults for location of log file directory.

  2. The most current log file within this directory is named jobtrac-connector.log.

  3. Historical log files are named jobtrac-connector.log.<yyyy-mm-dd> where <yyyy-mm-dd> refers to the date of creation.

  4. Historical log files are preserved for 30 days and then automatically removed.

Advanced options:

  • Logging severity level is controlled by the following setting found in the application.yml file co-located with the software’s JAR file and supporting scripts: logging.level.com.termalabs.

    Note: This file is created automatically when either the configuration script or software is first run.

  • Other logging behavior may be managed by co-locating a log4j2.xml file with the software’s JAR file and supporting scripts.

    Note: See document appendix for format references and an example log4j2.xml file.

Output

Monitoring the log using tail might result in output similar to the following:

$ cd /var/logs/jobtrac-connector
$ tail -f remoteConnector.log
2019-05-28 08:54:18,405 INFO [SchedulerConnectionService] Local scheduler
configuration matches remote.
2019-05-28 08:54:18,405 INFO [SchedulerConnectionService] ...Configuration
for scheduler verified.
2019-05-28 08:54:18,405 INFO [SchedulerConnectionService] ...JAWS
contacted and scheduler configuration verified (pre-publish).
2019-05-28 08:54:19,850 INFO [JobParserWorkflowService] JAWS host:
http://172.30.4.151:8080, connector id: AWr59_dNOGE=, transaction id:
AWr59_dNOGE=1559058859815
2019-05-28 08:54:19,853 INFO [JobParserWorkflowService] Adding 568 jobs to
JAWS...
2019-05-28 08:54:20,405 INFO [JobParserWorkflowService] ...568 jobs added
to JAWS.
2019-05-28 08:54:20,406 INFO [JobParserWorkflowService] Adding 180
conditions to JAWS...
2019-05-28 08:54:20,722 INFO [JobParserWorkflowService] ...180 conditions
added to JAWS.
2019-05-28 08:54:21,017 INFO [JobParserWorkflowService] ...Published jobs
and conditions to JAWS.
2019-05-28 08:54:21,701 INFO [FileReaderMonitorService] File parsed:
/opt/jobtrac/inbound/2019-05-28T15:53:58Z-POSD0125

Searching the log using grep may result in output similar to the following (note “-a” argument may be necessary, as shown):

$ cd /var/logs/jobtrac-connector
$ grep -a ERROR remoteConnector.log
2019-05-28 09:42:02,901 ERROR [JobParserWorkflowService] Unable to parse      
file (message: Invalid input line:@F@'!@8 @@@@''888', source:
2019-05-28T16:41:43Z-POSD0126). (will report to JAWS; schedulerId=(none),
transactionId=(none), errorType=GENERAL).
2019-05-28 09:42:04,294 ERROR [FileReaderMonitorService] Can not parse file
(stream error): ParserWorkflowException(lineCtr=1, inputText=[])
$