Sample Database Reconnect Logs
Sample text from logs where the database connection was lost and the Agent attempted to reconnect to it, are shown below.
If the application database goes down while the Agent is running, you will see errors like the following:
20120831/133922.152 - 13 RA_OEBS_SP4Status: OEBSAgentImpl: uc4_oae_pk.getStatus 0 timeout : 10 : i : class java.lang.Integer :
20120831/133932.147 - 13 RA_OEBS_SP4Status: DBCall: close connection 1089 ORA-01089: immediate shutdown in progress - no operations are permitted
uc4_oae_pk.getStatus
0 timeout : 10 : i : class java.lang.Integer :
20120831/133932.148 - 13 RA_OEBS_SP4Status: DBCall: isConnectError errorCode 1089 SQLState 72000 ORA-01089: immediate shutdown in progress - no operations are permitted
ORA-06512: at "APPS.UC4_OAE_PK", line 932
ORA-06512: at line 1
20120831/133932.149 - 13 RA_OEBS_SP4Status: DBCall: isConnectError true
20120831/133942.153 - 13 getConnection: retry (SQLState 72000) ORA-01089: immediate shutdown in progress - no operations are permitted
ORA-06512: at "APPS.UC4_OAE_PK", line 932
ORA-06512: at line 1
uc4_oae_pk.getStatus
0 timeout : 10 : i : class java.lang.Integer :
Additionally, you will see a message similar to the one below that will show how many total attempts the Agent will make to reconnect to the database. In the example below, it will retry 20 times with 30 seconds sleep time between each try. This would match a Reconnect Timeout of 10 minutes and a Reconnect Sleep of 30 second set in the Agent's General Parameters settings.
20120831/133942.155 - 13 RA_OEBS_SP4Status: DBCall: runFunction: uc4_oae_pk.getStatus
20120831/133942.162 - 13 JDBCConnection reconnect: retries 20 sleep 30 jdbc:oracle:thin:@hostname:1521
Next, you will see messages similar to the ones below that show the Agent trying to reestablish the connection.
20120830/134739.097 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/134809.120 - 13 JDBCConnection reconnect retry 1 jdbc:oracle:thin:@hostname:1521:VIS
20120830/134809.122 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/134838.998 - 10 U2000015 Periodical job test started.
20120830/134839.130 - 13 JDBCConnection reconnect retry 2 jdbc:oracle:thin:@hostname:1521:VIS
20120830/134839.131 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/134909.140 - 13 JDBCConnection reconnect retry 3 jdbc:oracle:thin:@hostname:1521:VIS
20120830/134909.141 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/134939.148 - 13 JDBCConnection reconnect retry 4 jdbc:oracle:thin:@hostname:1521:VIS
20120830/134939.149 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/134940.153 - 10 U2000015 Periodical job test started.
If the connection is successfully reestablished during the retry attempts, you will see messages similar to the following:
20120830/135009.159 - 13 JDBCConnection reconnect retry 5 jdbc:oracle:thin:@hostname:1521:VIS
20120830/135009.160 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/135039.168 - 13 JDBCConnection reconnect retry 6 jdbc:oracle:thin:@hostname:1521:VIS
20120830/135039.169 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/135041.331 - 10 U2000015 Periodical job test started.
20120830/135109.179 - 13 JDBCConnection reconnect retry 7 jdbc:oracle:thin:@hostname:1521:VIS
20120830/135109.180 - 13 RA_OEBS_SP4Status: JDBCConnectionImpl: JDBCConnection createConnection apps jdbc:oracle:thin:@hostname:1521:VIS
20120830/135109.224 - 13 JDBCConnection reconnect successful, elapsed ms 210114
20120830/135109.225 - 13 RA_OEBS_SP4Status: DBCall: initStatement
20120830/135109.226 - 13 RA_OEBS_SP4Status: OEBSAgentImpl: uc4_oae_pk.getStatus 0 timeout : 10 : i : class java.lang.Integer :
If the connection is not successfully reestablished, you will see messages similar to the following. When the maximum number of retries are exhausted, an error is printed to the log and written to the Messages dialog. After which, any newly submitted Jobs will go to a Waiting for Host status. Once the maximum number of retries are exhausted, you need to restart the Agent to reconnect it. Once the Agent restarts, all Jobs that had previously been running are allowed to finish, and new Jobs can start as expected.
20121213/200452.085 - JDBCConnection reconnect: retries 10 sleep 30 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200522.096 - JDBCConnection reconnect retry 1 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200550.495 - U2000015 Periodical job test started.
20121213/200552.107 - JDBCConnection reconnect retry 2 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200622.116 - JDBCConnection reconnect retry 3 jdbc:oracle:thin:@hostname:1521:VIS
.20121213/200651.660 - U2000015 Periodical job test started.
20121213/200652.126 - JDBCConnection reconnect retry 4 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200722.137 - JDBCConnection reconnect retry 5 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200752.161 - JDBCConnection reconnect retry 6 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200752.822 - U2000015 Periodical job test started.
20121213/200822.172 - JDBCConnection reconnect retry 7 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200852.206 - JDBCConnection reconnect retry 8 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200853.988 - U2000015 Periodical job test started.
20121213/200922.216 - JDBCConnection reconnect retry 9 jdbc:oracle:thin:@hostname:1521:VIS
20121213/200922.227 - JDBCConnection reconnect retries failed 10 jdbc:oracle:thin:@hostname:1521:VIS
Io exception: The Network Adapter could not establish the connection
20121213/200922.235 - SQL Exception
20121213/200922.235 - (SQLState 72000) uc4_oae_pk.getStatus
0 timeout : 10 : i : class java.lang.Integer :
20121213/200922.236 - U0003620 Routine 'UCEX_R' forces trace because of error.
20121213/200922.237 - U0003449 Output to the TRACE file is finished.
20121213/200922.241 - U0003450 The TRACE file was opened with the switches '0000000000000001'.
20121213/200922.325 - U0003449 Output to the TRACE file is finished.
20121213/200955.151 - U2000015 Periodical job test started.
20121213/201056.314 - U2000015 Periodical job test started.
.20121213/201158.661 - U2000015 Periodical job test started.