Upgrade Process

As of version 11.2 of the Automation Engine you can use the Zero Downtime Upgrade to upgrade your AE system with no downtime.

A wizard is provided to guide you through the whole process. There is no need for scripting or use of AE internals. You only need to follow the steps that are visible in the Automic Web Interface at each step.

Notes:

Using Zero Downtime Upgrade from v11.2 to v12 requires usind the classic UserInterface. In your client, open the System menu and open the Upgrade Wizard. You can download the classic UserInterface from the https://downloads.automic.com/ and use the Automation Engine v11.2 documentation. We recommend that you switch to the AWI as soon as the first new CP is turned on because users and agents disconnect automatically in the new AWI. In the classic UserInterface, you would do this in the System Overview where you can manually disconnect active users and agents.
Using Zero Downtime Upgrade from v12 onwards: Available from client 0000 exclusively. Here you can use the new Automic Web Interface.
The Automic Web Interface and the AE server must have the same release version. To check other compatibility issues, see Automic Compatibility Matrix.

Zero Downtime Upgrade is possible when the alternative MS SQL Server schema is already in use. However, you cannot use it to switch to this schema. For more information, see Integrating an Alternative MS SQL Server Schema.

Our consultants are experts in upgrading AE systems. Contact CA Automic Support, our experts will be pleased to assist you whenever it is necessary.

This page includes the following:

Overview

Zero Downtime Upgrade is a special function in the Automation Engine meant to enable system upgrades between major or minor versions without closing down the AE system.

You can use this function to upgrade between all maintained versions by CA Automic. Depending on changes between your current version and later versions, you may need to upgrade to a version between your current version and the desired target version first.

The Zero Downtime Upgrade needs certain prerequisites in order to work properly. Among others you should install two separate installations, either on separate hosts or on the same host. Refer to the detailed description below to prepare your setup accordingly.

Two special modes exist for this upgrade function, the compatibility mode and the parallel mode.

Compatibility mode

This mode is started as soon as you choose the option Next after you reached step 2 in the wizard and upgraded the database. In this mode, in effect until you choose the option Next in the final seventh step, certain system optimization functions are not available. System performance will be reduced noticeably.
Parallel mode

A mode during the upgrade when base and target version processes are active at the same time.

You will start the process by choosing the option Run System Upgrade Wizard in the System Upgrade page. That way the system will be set into the compatibility mode.

This mode will persist during the whole time, even if you should conduct a rollback, until you choose the option Next in the seventh final step of the wizard to use the new target version.
In this mode the system will use base version and target version CPs and WPs as the upgrade process advances, but base version WPs may slow down until the upgrade is complete.
While you are in compatibility mode, you are not allowed to switch a WP to a DWP.

In the course of the upgrade you will be asked to test (even for a few days), if all things work smoothly before finalizing the upgrade. If you encounter errors, you will be able to perform a rollback by using the Rollback option. The Rollback option will only be visible in step 4 if the wizard has been successfully processed.

Note: Although a rollback option has been implemented, database changes persist. The rollback is a temporary measure only.

It is recommended to start an upgrade only if the necessary preparations for setup and testing have been taken care of.

When an Automation Engine process running the old version stops, it cannot be restarted. When all Automation Engine processes running the old version have stopped (e.g. because of a power failure), the Zero Downtime Upgrade finalizes the upgrade process. No rollback is possible in this case. When the finalize action is done, the old work processes and communication processes will end automatically.

Once you have finished the whole upgrade process by confirming Next in the final (seventh) step of the wizard, a rollback is no longer possible.

When the upgrade process finishes, the compatibility mode ends and the system runs normally again.

Important!

It is strongly recommended to carry out the following upgrade steps in the order they are listed in the CAPKI documentation. For more information, see CAPKI - Securing the ServiceManager.
It is possible to set the privilege for this kind of upgrade for any User object, but it is recommended that only the system administrators coordinate and conduct upgrades.
You have to conduct the upgrade from start to finish in the same client.
Utilities cannot be rolled back once the database has been upgraded (step 2 in the upgrade wizard). You will have to use the target version utilities from then on.
System performance will be reduced considerably once the upgrade has been started and the compatibility mode is active. As long as the compatibility mode is active, the performance will be about two thirds of ordinary runs.
Automation Engine utilities are external applications. As administrator ensure that no utilities run during the Zero Downtime Upgrade process.
As two processing lines have been introduced, the following script elements might fail and throw a script error message (no. 20909), when you switch to parallel mode or initiate a Zero Downtime Upgrade rollback:
- CHANGE_LOGGING() for agent
- SYS_INFO(MQ*,*)
- MODIFY_SYSTEM(SMGR_LOOKUP)
To prevent this error, take care not to have these elements in use, when switching modes and restart jobs as necessary.
You can log into a numbered client in the Automic Web Interface during the DB load. You cannot log into client 0, because initial data is inserted/updated until the commit is done at the end of DB Load.

Upgrade and rollback steps:

Requirements

System Setup

In order to be able to conduct the Zero Downtime Upgrade properly, you have to use a distributed installation . These installations can either be situated on two separate hosts or two separate installations on the same host, but in different bin directories. For more information, see ZDU - Distributed Installation .

The upgrade process makes reference to either the base or the target system, which also represent the base and target version in the upgrade scenario.

Should you use a PROXY and a firewall in the systems used for the upgrade, ports will have to be configured accordingly.

Information security services, such as the TLS implementation of the ServiceManager, require features provided by the CAPKI package. You need to install CAPKI on all computers in which the AE processes or the ServiceManager and any of its clients will run.

Important! In a UNIX environment do not overwrite the existing installation, which is possible without warning on UNIX, even if the binaries are currently in use.

More information:

Set User Privilege

In order to upgrade your system you have to make sure that the respective user or users (usually the AE's administrators) have the necessary permissions.

The user (administrator) conducting the upgrade needs the privilege Execute system upgrades to be set in the corresponding User object.

For more information about how to set privileges in User objects, see Granting Automation Engine Privileges

Upgrade the Automic Web Interface

You have to install a new AWI instance every time you upgrade to a major or minor version of the Automation Engine.

Upgrading an existing installation to the new release upgrades the AWI framework and all its plug‑ins at the same time. Use the same steps to install hot fix packages between releases.

Stop the Apache Tomcat Service.
1. Open your Windows Services manager.
2. Stop the Apache Tomcat service.
Back up current files.

These steps are optional. You can make a safety copy of your entire AWI installation, or just the configuration files. Although, you can use very little of the previous configurations in the new version, you might find it helpful to have a copy for later reference when you configure the upgrade.
- To backup your whole AWI installation, go to your Tomcat installation, open the webapps folder and make a backup copy of the entire awi folder.
- To backup only the configuration files, go to the folder where you have the current AWI version installed, and make a backup copy of the folder webapps/awi/config.
Upgrade the application server.

Important: AWI runs only on Tomcat. If you want to use another application server for the Automic Web Interface, see Installing the Automic Web Interface.

Before upgrading the application server go to Automic Compatibility Matrix to see which version is compatible with AWI.

You will find the upgrade instructions and other relevant information on the homepage of your application server provider.
Restart the application server.

You will find the relevant information on the homepage of your application server provider.
Deploy the new AWI version (WAR file) on your application server.

If you are using Tomcat, copy the awi.war file to the webapps folder of the Tomcat installation. If the Tomcat is running, the awi.war file will be deployed automatically, creating a folder with the same name.

Important! Deleting the awi.war file removes the corresponding subfolder, which also undeploys the AWI.

If you are using WebSphere or WebLogic, please refer to their official documentation for more information.
Configure the new AWI version.

For more information on how to configure your upgraded AWI, see Configuring Automic Web Interface.

If you made a backup before you started the upgrade, then you can retrieve the following files and settings from the backup files from the previous webapps/<ECC>/config folder to the new webapps/<AWI>/config folder:
- uc4config.xml:
  
  You can replace the new file with your backup.
  
  If you have more than one AE system connection defined, you can allow your users to choose their connection at login, or you can specify any of the defined connections as the only valid connection. You defined this in the automationengine.index setting of the configuration.properties file.
  
  Previously, users could not choose a connection and you had to be sure that your one valid connection was defined first in uc4config.xml.
- configuration.properties:
  
  Important: Do not replace the file in your new installation with your backup. Instead, define the file as described in this guide. However, you can refer to your backup files for the following two parameters, which may have optionally been defined in your AWI environment:
  - defaultHomeDashboard (This would be the dashboard attribute in the navigation.xml file.)
  - customHomeDashboardsFolder (This would be from the previous configuration.properties file.)
- logback.xml:
  
  Do not restore your backed up version.
  
  However, if you changed the default setting for the severity level of event messages (DEBUG) for the root element, then you might want to make the same change for AWI.
- Theme folder:
  
  If you had customized the color or banner logo in your previous environment, then copy the contents of your backed up webapps/awi/config/theme folder to the folder of the same name in your current AWI version.
  
  For more information, see colors.properties.

The Upgrading Steps

While upgrading, you can leave the process at any time and return to it later at the step you left off by calling the System Upgrade page. The Automation Engine system will store the information of the step where you left off and open the page at that step automatically as soon as you return.

Once you have finished the whole upgrade process by confirming Next in the final (seventh) step of the wizard, a rollback is no longer possible.

Upgrade Procedure

Log on to the Automic Web Interfacein system client 0000.

If you granted the privilege Execute system upgrades to a user for the first time in the preliminary step, that user will have to log off and restart and then log on to the AWI of the AE with those credentials.
On the Automation Engine Management > System Upgrade page of the Administration perspective, select Run System Upgrade Wizard to start the Zero Downtime Upgrade. Follow the wizard's instructions.

This starts the Compatibility Mode and enables upgrading the database with the target version's schema and initial data.

Note: Leave the page open and use the external DB Load Utility to upgrade the database. For more information, see AE DB Load.

While this step is processed, the AE system loads the target version database schema and initial data.

When you have finished upgrading the database with DB Load, confirm the step in the wizard by clicking Next. This starts a check in the Automation Engine, if the initial data for base and target version are present and the target version data has been loaded correctly.

The DB Load Utility carries out the following actions:

Running System Check

Should ZERO_DOWNTIME_UPGRADE be set to N and the system be running, DB Load will end the loading process with a corresponding message.
Version Check

DB Load checks, if the correct version for the Zero Downtime Upgrade function is installed and used. If the version is not correct, a message to that effect will be returned.
New DB Schema and Initial Data - Check and Load
- If the key ZERO_DOWNTIME_UPGRADE is set to N and the system is not running, DB Load will load the new DB schema and initial data. For future reference records are written into the appropriate database table.
- If the key UC_SYSTEM_SETTINGS / ZERO_DOWNTIME_UPGRADE = Y and the system is running, DB Load will also load the new DB schema and initial data. Otherwise DB Load aborts and returns the corresponding message.
- In addition DB Load checks, if CPs and WPs of the target version are already active. In case active CPs/WPs of the target version are found, DB Load will end and return the corresponding message.

Install and start communication and work processes from the target version parallel to the already running ones.

Note: Use the second installation (see preliminary steps) for installing the new processes from the target version. Make sure not to use recurring names for processes you install. Best practice is the setup of two separate installations. For more information, see ZDU - Distributed Installation .

The new processes are not yet in use at this point.
If you haven't upgraded your AWI as part of the preliminary steps, do so now. The wizard provides a short overview of the upgrading procedure for the AWI.
In the next step the base and target version CPs will change places: The running tasks will finish on base version CPs, new tasks will be started on target version CPs, until all base version CPs are idle.
- Therefore it is necessary for you to disconnect users still connected to base version AWIs, or to inform them to log off.
- Disconnect agents connected with base version CPs manually so that they reconnect automatically with the new CP generation. Make sure that agents can be disconnected without interruption of running executions.
Note: The wizard offers an interactive page with two panes, where you can disconnect users and agents yourself. The buttons for disconnecting users and restarting agents change the side of the pane and show up over the table for new or old version, depending on the function you are using: Upgrade or Rollback.

To disconnect a user or several users or all users at once:
1. Click Users, find the user or users you want to disconnect.
2. You can multi-select user entries by using the following key combinations:
  - Use Strg+click to select only certain users.
  - Use Shift+click to select all users between first and last selection.
3. Disconnect all or only certain users.
  - If only a certain number of users shall be disconnected, use the Disconnect User, which disconnects all users you selected in the previous step.
  - To disconnect all users in the list at once, use Disconnect all Users. In this case no selection is necessary.
4. Users you disconnected will disappear from the lists.
To reconnect an agent or several agents or all agents at once:
1. Click Agents, find the agent or agents you want to reconnect.
2. You can multi-select agent entries by using the following key combinations:
  - Use Strg+click to select only certain agents.
  - Use Shift+click to select all agents between first and last selection.
3. Reconnect all or only certain agents.
  - If only a certain number of users shall be reconnected, use the Reconnect Agent, which reconnects all agents you selected in the previous step.
  - To reconnect all agents in the list at once, use Reconnect all Agents. In this case no selection is necessary.
4. Agents you reconnected will appear either in the new version or the old version list, depending on the version you reconnected them from.
The system runs now with the target version initial data and processes, while the base version processes will be subsequently phased out.

Important!
- This parallel mode should last as long as needed for proper testing. Use it for testing the system to determine, if the upgrade has been successful and all processes or tasks run smoothly. This is necessary in order for you to decide, if you want to finish the upgrade by using the final step. For more information, see FAQ - Zero Downtime Upgrade.
- At this point the option Rollback will be available as well, which is not visible in the previous steps.
  
  Important! This is the last opportunity to Rollback before finalizing the upgrade process.
If testing was successful, click Next. Analytics, CDA, and other plug-ins and additional components can be upgraded here.
Check the memory entries in the database and delete the ones referring to the old version.
If you want to upgrade your system's agents, please use the Centralized Agent Upgrade. For more information, see CAU - Upgrade Procedure.

Clicking Next finalizes the upgrade process. This is only possible, if no base version processes are active anymore.

Notes:
- This step ends the parallel and compatibility modes and you are no longer able to roll back to the old version.
- If the error message ZERO_DOWNTIME_UPGRADE - 'FINALIZE' not possible. Work processes to shut down are not idle yet./Please try again later. is displayed, schedule tasks may be active. Wait for the scheduled tasks to change their execution period (usually the next day), then return to the wizard and try executing the option again.
- Once you have finished the whole upgrade process by confirming Next in the final (seventh) step of the wizard, a rollback is no longer possible.

Rollback Option

This rollback function is meant for a possible troubleshooting. Certain tasks or processes may not be running quite as expected while testing the new installation. Choosing this option allows you to check on those tasks or processes.
The option will be visible in the wizard in step 4, as soon as the new version CPs and WPs have been started and are running.
Using this option will roll back the processing to the base version.

Once you have finished the whole upgrade process by confirming Next in the final (seventh) step of the wizard, a rollback is no longer possible.

Rollback Procedure

In order to be able to conduct a rollback you have to be at step 4 of the System Upgrade wizard, Upgrade Mode. As always, if you left off at a certain step, you can return to it calling the System Upgrade sub-page.

In step four of the upgrade process, select Rollback and follow the wizard's instructions.
The system switches back using CPs/WPs from the base version you started from, while the new processes are subsequently phased out.

Exception! The primary role (PWP) remains with a new WP until Restart has been executed.
Analyze the reason which caused you to execute the Rollback. Two main scenarios may cause a rollback:
- Something in AE object definitions or environment settings has to be adapted in order to work correctly with the target version. In this case, make sure to solve the issue and continue with step one in the wizard.
  
  To return to step one of the wizard: In Rollback Mode of step 4 of the wizard (see above) click Next.
- The new processes / initial data from the target version have not been working as expected. In this case contact CA Automic Support for assistance in finding out what has to be done.
  - If new components have to be installed to solve the issue, execute the option Restart in step four of the wizard (Rollback Mode).
    
    This will end the parallel mode and switch back the PWP role to base version and shut down running components of the target version so that they can be replaced.
  - Install components you may have received from CA Support and continue with the Upgrade procedure step one, clicking Next to start the check for initial data, as described above.
  Note: When you click Restart, the error message ZERO_DOWNTIME_UPGRADE - 'Restart' not possible. Work processes to shut down are not idle yet./Please try Restart again later. may be displayed. Wait for the scheduled tasks to change their execution period (usually the next day), then return to the wizard and try executing the option again.