Upgrading an AE System from Version 10

To ensure that your upgrading process is successful Automic recommends that you strictly follow the steps that are described in this guide.

Never upgrade a productive AE system without having it tested extensively in a separate environment.

For details about installing hotfixes, see the related topic hotfix installation.

It is very important that you follow a structured procedure when you upgrade an AE system, because it guarantees that processing can be continued as soon as possible.
The following guidelines explain the required steps in detail and especially address areas that need special attention. Follow the recommended steps for a smooth conversion process and you will soon be able to use all the new functions of your new Automation Engine version.

When changing the version of the Automation Engine, the utility AE DB Load processes and monitors all the required steps for changing the databaseA database is an organized collection of data including relevant data structures.. This is necessary, because it also modifies data that cannot be changed with SQL. All steps that the utility processes during the upgrading process are logged in the file uc_upd.txt. The file chngdb.sql also informs about the database-relevant statements that were set. These statements must be processed by the utility. Refer to the Release Notes of the relevant Automation Engine version for changing the database and preparing the necessary steps.

The upgrading process comprises of several stages:

1. Installation and comprehensive testing in a test environment.
2. Planning the conversion time and creating a plan for a possible re-conversion.
   Never change to a new Automation Engine version without having planned a re-conversion scenario.
3. Make a backup of the AE database and all the components' directories.
4. Upgrade your system step by step WITHOUT using the new functions.
5. Only use the new functions when every component has been converted to the new version and after a particular system-monitoring period.

Each stage consists of many small steps. Your AE system is not upgraded all at once but step by step. The old components are not overwritten and the new files are installed in separate directories. Only the AE database is directly upgraded with the utility AE DB Load. This method has two main advantages:

1. You can quickly re-convert to the old AE-component version if problems occur in your system environment. Therefore, there is almost no risk for your processing.
2. Depending on the size of your AE system, the complete upgrading process can take some time. The advantage of upgrading step by step is that you can do so in individual and shorter periods of time which makes it easier to coordinate the upgrading process with other departments and processes. And it is also easier to locate errors that might occur.

First upgrade your test system. Possible problems can so be recognized and solved before they occur in your production system. Test the individual upgrading steps, thereby setting up a plan for upgrading your production system.

The AE system is not available while the database is upgraded.

After each step of the upgrade installation, it is essential to monitor the new components for some time. Only continue the upgrading procedure when they have proven to run stably.

Note that the upgrading process Automic recommends enables production to be continued in the old Automation Engine version at any time. The only requirements are a parallel installation of the components and the provision of a second database instance.

Use the message-comparing program if you require a list of all changed messages.

Our consultants are experts in upgrading AE systems. Contact Automic, our experts will be pleased to assist you whenever it is necessary.

The table below lists new features that might lead to compatibility issues or should be taken care of when upgrading - it does not list all new features of this AE version.
New features are described in full in the release notes.

Description of the table columns:

• Topic - Name of the general topic or new feature
• Changed behavior - What has been changed
• Possible incompatibilities - Impact the change may have
• Actions/Countermeasures - What can be done to identify and/or remove possible incompatibilities

General DB change

Information and the checking instructions apply to all versions, between your existing installation and the latest you want to upgrade to, respectively.

The DB scheme/structure has been changed.

Custom SQL queries on AE DB do not work anymore.

• Check chngdb.sql for general changes
• Check and adapt relevant SQL/SQLI/SQLJOBS objects accordingly
• Check and adapt relevant DB queries used in external tools/programs

Release Packaging

The Release Packaging (zip files, folder structure and its content) have been changed.

The following files and folders have been removed:

• Folder ./11/ (the content, e.g. UserInterface, AutomationEngine) are now directly in the base directory and no longer in the sub folder.
• The uc4setup files and the CD image files have been removed.
• There is a new installer available for single box installation in the product bundles.

Possibly automated installation routines using release images will not work.

If there are scripts relying on the old package structure, you have to adapt the paths.

CP/WP routing 

FORMS request routing to RA agents has been changed.

When you edit RA connection objects, a corresponding RA agent has to be active in the same client to retrieve data.

Assign an RA agent for relevant clients.

Third party monitoring via External Monitoring Interface

The new state-of-the-art JMX interface EMI (External Monitoring Interface) has been introduced.

The existing SNMP interface is still available, but the third party integration with BMC Patrol, HP OpenView, and Tivoli has been de-supported.

There is no incompatibility, but the integration with BMC Patrol, HP OpenView, and Tivoli is not available anymore.

Integrate third party monitoring systems via the new EMI instead.

Doubled MQ tables

MQ tables have been doubled and renamed.

MQ tables might not be considered when you use DB Reorg and existing scripts might fail due to changed table names.

Adapt DB Reorg scripts.

Proxy INI file extensions

Parameters and the format of some parameter values have been changed in the INI file due to functionality improvements.

You will not be able to reuse INI files of Proxy versions prior to v2 without adaptions.

Adapt INI file parameters according to the Proxy user documentation.

The table below lists new features that might lead to compatibility issues or should be taken care of when upgrading - it does not list all new features of this AE version.
New features are described in full in the release notes.

Description of the table columns:

• Topic - Name of the general topic or new feature
• Changed behavior - What has been changed
• Possible incompatibilities - Impact the change may have
• Actions/Countermeasures - What can be done to identify and/or remove possible incompatibilities

Message numbers

(For detailed information on individual possible impact see also the topic Message number changes)

Message numbers are now eight digits long, instead of seven.

Old format: Unnnnnn
New format Unnnnnnn

Filtering on Unnnnnn numbers won't work with

• PREP_PROCESS_REPORT
• PREP_PROCESS_FILE
• Output scan filter
• Log scan scripts or custom tools
• Job reports of old java-based agents contain 7 and 8 digit messages
• Agents of older versions send/log 7 digit message numbers

• Find out script functions PREP_PROCESS_REPORT and PREP_PROCESS_FILE in scripts via search function e.g. change PREP_PROCESS_REPORT(,,’ACT’,’*U1234567*’) to PREP_PROCESS_REPORT(,,’ACT’,’*U01234567*’) 
• Check filter objects manually or use a SQL statement such as
  e.g. SELECT OH_Name FROM OH,OFC WHERE OH_Idnr=OFC_OH_Idnr AND OFC_Filtertext LIKE 'U001%' AND OH_DeleteFlag = 0

Text variable values 

• 1024 character limit for text based fields/variables has been removed
• Data type of DB fields has been changed from VARCHAR (1024) to CLOB (unlimited)

• Oracle DBMS_LOB Package must be installed to handle CLOB  
• Results on string based operations can exceed 1024 characters instead of string truncation or output error messages
• Queries of SQL/SQLI objects might operate on tables with text fields of changed data types -> refer to chngdb.sql

Check queries on tables with changed fields (VARCHAR -> CLOB)

e.g. search for "OVW_Value1": SELECT OH_Name FROM OH WHERE OH_Idnr IN (SELECT OVD_OH_Idnr FROM OVD WHERE OVD_SQL like '%OVW_Value1%' OR  OVD_SQLOra like '%OVW_Value1%' OR OVD_SQLDB2 like '%OVW_Value1%')

Script function GET_OH_IDNR

The second parameter client was removed from this Automic script function due to security reasons.

Objects using this script function with the specified parameter client will abort.

Check all script tabs for this function and remove the parameter / correct the function call.

Login Objects for File Events on Windows, Unix, zOS

A new privilege has been introduced for user objects

New privilege is checked and used per default after AE version upgrade.

• On upgrade the new default is used and no actions are necessary.
• But if user objects are loaded via Transport Case or XML data is imported for example, this privilege may not be set. It must be set manually, if necessary.

zOS: Replacing Job line in JCL

The value of attribute MVS_JOBTYPE has been deprecated.

Currently no incompatibility, the old value is still compatible.

No checks/adaptions needed

XML Variables (VARA.XML)

• The "Oracle Instant Client" is no longer supported.
• "Oracle XML DB Package" has to be installed.

• The "Oracle Full Client" must be installed (instead of "Oracle Instant Client").
• The "Oracle XML DB Package" must be installed.

No checks/adaptions needed, part of the AE DB installation

FAULT_OTHER tasks

Deactivation behavior of tasks with status  FAULT_OTHER has been changed.

• In previous AE versions, tasks have been deactivated in activities list by default. As of version 11 their deactivation is bound to the deactivation options of the task.
• Consequently more tasks will remain in the activities window, than in previous AE versions.

Check deactivation options accordingly.

(If needed, they can be set to archive the behavior of previous AE version)

Script function GET_ATT_SUBSTR(call_text)

In previous AE versions, at least a blank was returned. As of version 11, no value in case of missing call text is returned.

Error occurs on empty call text.

RA agents

Shared third party libraries were removed from the RA agent core.

You have to obtain all needed shared third party libraries and ensure that they are installed properly.

Download Oracle JDBC driver and copy it to the lib directory of the RA core.

This is only required for an RA agent that needs an Oracle DB connection (via JDBC) - e.g. RA Banner, RA Oracle EBS, RA JDE or RA Oracle Retail.

From version v9 SP4 to v10 SP4, GET_VAR resolves recursively. This means, if a value contains an "&", the script processor tries to resolve this. This has been changed.
Example:
Static variable 'MYVAR', Key = 1 Value = 'Schmidt&Partners'
Script
:set &partners = 'no'
:set &value# = get_var('MYVAR','1')

The variable &value# will contain therefore:
v9 SP4 to v10: Schmidtno
V11 : Schmidt&Partners

With version v10 SP4 HF1 the behavior was configurable (key RESOLVE_GET_VAR in UC_SYSTEM_SETTINGS).

In v11 the configurationA set of constituent components that make up a system. This includes information on how the components are connected including the settings applied. option was removed. As a conclusion all variable values, that should be resolved recursively, must use the Script function RESOLVE_VAR.

Potentially all scripts that use GET_VAR and a variable value that contains "&" characters, are affected.
Affected scripts/variables are not easy to determine, especially with dynamic variables.
Automic suggests that you apply comprehensive tests.

To determine affected Variable objects, you may use this SQL query:

select oh_client, oh_name from oh where oh_idnr in
(select ovw_oh_idnr from ovw where
ovw_value1 like '%&%'
or OVW_Value2 like '%&%'
or ovw_value3 like '%&%'
or ovw_value4 like '%&%'
or ovw_value5 like '%&%'
) order by OH_Client, OH_Name asc

The Script function RESOLVE_VAR may be used to recursively resolve a static Variable object.

Script variables have been changed to unlimited length.

You can use GET_VAR to check for a non-existing key, which returns a blank, as shown in this example:
:SET &test# = GET_VAR(‚VARA.TEST‘,‘KEY_DOES_NOT_EXIST‘)

If you use STR_MATCH() following on this, with an empty string for the script variable &test#, like this:
:SET &true# = STR_MATCH(&test#,"")

in v10 of the Automation Engine the return value for &true# will be "Y"
In V11 it will be "N".

Explanation:
In STR_MATCH() a variable of length 0 is expected (""). Since the variable &test# contains a blank (" "), the return value for &true# is "N".

The v11 behavior thus is correct.

The result of the following script is different in v10 and v11.1:

:SET&TEST#=' 0000000050'
:IFFORMAT(&TEST#, "0") =50
:P"valid"
:ENDIF

In v10 it will result in the output "valid" in the report.

In v11.1 you will receive an error message about not allowed non-numeric line input.

Upgrade Utilities

Upgrade AE and ServiceManager

Upgrade Framework Integration (optional)

Upgrade UserInterfaces (if they have not yet been upgraded)

Upgrade the Enterprise Control Center (if it has not yet been upgraded)

Stop all clients

Stop the Automation Engine

Back up the AE database

Upgrade the AE database

• Automic strongly recommends reading the notes concerning database modifications below before starting the upgrading process.
• Upgrade your AE database using the utility AE DB Load. The individual steps you find below.
• If you have created a duplicate, Automic recommends upgrading it now in order to make sure that the original instance remains operable. Keep in mind to enter the correct database connection in the INI files of the utilities.

Notes on Upgrading the AE Database   Important Notes:   Automic recommends that a database administrator execute the following steps.  Direct modifications that are made in database contents without using AE programs will result in an inconsistent database. Always make a database backup before you process any work steps. This process can take a while depending on the database size and upgrading complexity. Ensure there is sufficient disk space for the \AUTOMIC\DB directory and that the database's LOG section can store all this data.  Schemes that provide access to the database should be removed after the updating process in order to avoid unintended database modifications. DB2: After updating the AE database, check the size of the tablespaces and if required, run the following SQL statement: alter tablespace <UC4 tablespace name> reduce max;

Database Upgrade with AE DB Load Utility Notes: The DB directory of the installation CD can include several versions of SQL scripts and database files. They are required if you upgrade several versions. Be sure to use the correct version if <vers> is indicated in the document. Automic strongly recommends you also read the notes for upgrading the AE database above. Modifications that should be made are available in the special_rt.sql file. Also execute the new_mq.sql file. Search the UC_UPD.TXT file for the lines shown below and remove the comment "message" at the beginning. Extract of the adjusted UC_UPD.TXT file: process_sql_file new_mq.sql process_sql_file special_rt.sql The special_rt.sql file converts the stored reports in the database. Depending on the number of report data records that are affected by the conversion it is important to ensure that there is sufficient memory and a transaction log of the appropriate size. Because the table is copied, it exists twice for a while which also prolongs the conversion process. Automic recommends reorganizing the reports before you execute the UC_UPD.TXT file in order to minimize the number of report data records. Procedure: 1. Changing the database scheme and loading new initial data to the database Server computer All server processes must be stopped. Pay special attention if server processes run on several computers. The following steps must only be processed when all server processes have been stopped. Admin computer The directory containing the database files must be stored at the location that has been specified in the INI-file parameter INPUT of the utility AE DB Load. The folder that includes the BIN directory of the utilities is the default folder.  Windows example: Utility in C:\AUTOMIC\UTILITY\BIN Database files in C:\AUTOMIC\UTILITY\DB Utilities - Windows: The files for the AE database are provided in IMAGE:DB. Copy the complete DB folder to the above directory Utilities - UNIX: The database files are included in the archive db.tar.gz which is provided in the folder IMAGE:DB. To unpack the archive, use the following commands: gzip -d db.tar.gz bzw. gunzip db.tar.gz tar xvfo db.tar (Linux: tar -zxvf db.tar.gz) Copy the unpacked files to the defined directory. Start the program AE DB Load to upgrade the database.Select the file <DB directory>\GENERAL\<vers>\UC_UPD.TXT The current database version is identified and the database upgraded. As this happens, the database structure and data are changed. Objects of client 0000 are automatically replaced or amended. 2. Selecting the authentication method Admin computer The utility AE.DB Load displays a mask in which you must select an authentication method. This mask is only displayed if the database is upgraded to a new Automation Engine version. 3. Installing partitioning with ILM (optional) Admin computer A mask opens in which you can select the settings for partitioning with ILM. This step is optional.  You cannot undo an AE database partitioning process.

Configure and start the Java Work Process (JWP)

• As of version 11.2 of the Automation Engine the function Export with references and the full text search in the ECC require a JWP.

General

The JWP is a component of the Automation Engine which is required for the following functions:

• Single Sign-on (via KDC)
• Adaptive ERT calculation
• Function Export with references
• Application Release Automation Integration
• ECC Global Search

Files Provided

The JWP is provided in the same directory as all the other Automation Engine files.

File / Directory	Description
ucsrvjp.jar	File for starting the JWP.
/lib/	Directory with OSGI implementation and JDBC driver.
/plugins/com.automic.database.jar	File for database access.
/plugins/com.automic.ara.jar	Opens the ARA WebService.
/plugins/com.automic.ert.jar	(Adaptive) ERT calculation.
/plugins/com.automic.kernel.jar	JWP kernel.
/plugins/com.automic.network.jar	TCP/IP connections.
/plugins/com.automic.sso.jar	Single Sign-on.
/plugins/com.automic.util.logging.jar	Logging / Trace.
/plugins/org.apache* /plugins/org.eclipse*	Additional OSGI bundles for console and services.

The directory /configuration/ is created automatically when the JWP is first started and contains the OSGI bundle's cache.

Installation

Unpack the files

In Windows, the JWP files are automatically copied from the SETUP.EXE program to the BIN directory. In UNIX, the files are located in the respective TAR archive.

Copy the provided "plugin" and "lib" directories into the BIN directory of the Automation Engine.

The subsequent installation steps depend on the database type used.

MS SQL Server

	1)	Install JDBC driver

Download Microsoft JDBC Driver 4.0 for SQL servers.

After downloading, copy the file "sqljdbc4.jar" into the lib directory of the Automation Engine.

 

	2)	Activate TCP/IP in the MS SQL server

Now check whether the MS SQL server instance used allows access via TCP/IP.

Open the SQL Server Configuration Manager and select Protocols for MSSQLSERVER under SQL Server Network Configuration. The item "TCP/IP" must be set to "Enabled" in the right-hand section.

 

	3)	Determine the MS SQL server port

The default port of the MS SQL server port is 1433.

If you are not sure of the port of your MS SQL server instance, you can find this out from its log file. The message "Server is listening on [ 'any'<ipv4> port number]" should be found in the current log file, which contains the port.

 

	4)	Modify the Automation Engine configuration file

The JWP uses the same configuration file (ucsrv.ini) as the other work processes of the Automation Engine system.

The database connection must be modified in the configuration file for the JWP. There are 2 different options for this:

1) DSN-less ODBC Connection

Please note that with this option, the same database connection string that is also used by all the other WPs in the Automation Engine system must be changed in the configuration file. When installing a JWP for an existing system, all WPs must be subsequently restarted.

A connection string is required in the [ODBC] section of the configuration file, the syntax of which does not require DSN. The server and database name must be specified directly in this case.

SQLDRIVERCONNECT=ODBCVAR=SNNNNNRN,Driver={SQL Server Native Client VERSION};Server=tcp:SRVNAME,PORT;Database=DBNAME;Uid=DBUSER;Pwd=DBPWD

• VERSION- Version of SQL Server Native Client. Is displayed in the SQL Server Configuration Manager.
• SRVNAME - Name of the database computer.
• PORT - port of the MS SQL server instance.
• DBNAME - Database name.
• DBUSER - Database user.
• DBPWD - Database password.

Example:

[ODBC]
SQLDRIVERCONNECT=ODBCVAR=SNNNNNRN,Driver={SQL Server Native Client 11.0};Server=tcp:dbsrv01,1433;Database=AEV10;Uid=user;Pwd=password

The entry should be on one line (no break).

 

2) Separate connection string for the JWP

With this option, a separate database connection string for the JWP is defined in the [JDBC] section.

Example:

[JDBC]
SQLDRIVERCONNECT=jdbc:sqlserver://dbsrv01;databaseName=AEV10

The name and password of the database user are used by the [ODBC] item.

The advantage of this method is that the connection string of the other WPs ([ODBC] section) does not need to be changed or restarted.

Oracle

	1)	Install JDBC driver

Copy the JDBC driver "ojdbc6.jar" from the Oracle database client installation to the "lib" folder of the JWP.
The file is located here: ORACLE_HOME/jdbc/lib/ojdbc6.jar

 

	2)	Configuring the database connection

There are 2 options:

1) Connection via OCI

Modification of the INI file "ucsrv.ini" is not necessary in this option. However, the JWP requires access to the Oracle database libraries in the same way as a WP. In UNIX, the environment variables LD_LIBRARY_PATH or SHLIB_PATH must therefore be selected accordingly, depending on the platform.

More information on installing the JDBC driver can be found in the JDBC installation instructions from Oracle.

 

2) Direct connection to the database

You can connect directly to the database using the Oracle JDBC Thin driver.

The [JDBC] section in the ucsrv.ini file must be configured accordingly. Example:

[JDBC]
SQLDRIVERCONNECT=jdbc:oracle:thin:@dbserver:1521/service_name

The name and password of the database user can be found in the [ODBC] item.

DB2

	1)	Install JDBC driver

Copy the file "db2jcc4.jar" (JDBC driver) into the "lib" directory of the JWP.

This file is part of the DB2 client and is located in the sub-directory "SQLLIB/java".

	2)	Configuring the database connection

Modification of the ucsrv.ini file is not necessary.

However, if required, the database connect string can be defined in the [JDBC] section of the INI file.

Example:

[JDBC]
SQLDRIVERCONNECT=jdbc:db2://server:<port>/database

The user name and password for database access can be found in the [ODBC] item.

Add certificates for SSL

In order to use SSL, the certificate(s) of the LDAP server must be available to the Java Work Process.
The JWP uses the default keystore file "cacerts" in the lib/security directory of the JRE.

Two options are available to add the certificates:

1	Add certificates with the keytool

• Go to the jre\lib\security folder of the Java installation and import the certificate with the keytool command:
  
  keytool -keystore cacerts -importcert -alias ldapServer -file certficate.cer
  
  
• When prompted to trust this certificate respond by typing "Y".

 

2	Add certificates via download

Another option to install the certificate is the command line parameter -installcert of the Java Work Process.

java -jar ucsrvjp.jar -installcert <host>:<sslport>

• This assumes that the Java Work Process has write access to the cacerts file of the Java installation.
• This command detects the path of cacerts, connects to the specified host and port and tries to create an SSL connection.

If a certificate is missing, the message "unable to find valid certification path to requested target" is printed and the missing certificate is downloaded and stored in the cacerts file.

Start the JWP

Use this kind of command to start the JWP via the command line:

java -Xmx512M -jar ucsrvjp.jar -IC:\temp\ucsrv.ini

The file "ucsrvjp.jar" is provided in the same directory as the other Automation Engine files. It is used exclusively to start the JWP.

The JWP can also be started via ServiceManager.

java -Xmx512M -jar ucsrvjp.jar -svc%port% -IC:\temp\ucsrv.ini

The -svc parameter should be omitted when starting directly via the command line.

The parameter -I to specify the INI file is optional. If the parameter is missing, the JWP attempts to find the file "ucsrv.ini" in the current working directory (= directory in which the file "ucsrvjp.jar" is located).

Start the Automation Engine and clients

Monitor the AE system

Install the Agents

Install the remaining components

Use the new functions