Universal Language Support

As of version 24, Automic Automation supports Unicode / UTF-8. Automic Automation can now easily handle operations in which different character sets are involved, as can be the case with File Transfers, OS Agent Jobs, importing/exporting objects, loading data, and so forth. This enhancement has an impact on the entire Automic Automation system. For example, some fields in the database schema have been extended, the AE processes now incorporate a database UTF-8 check at startup, various script statements and functions have been extended too. For a detailed description of how Automic Automation supports Unicode / UTF-8 and how the various Automic Automation components are affected, read Universal Language Support (Unicode)

Exceptions:

The following Automic Automation components DO NOT support Unicode / UTF-8 operations:

  • Service Manager

  • Call APIs

  • Archive Browser AE DB utility

  • Client Copy (only when both Clients reside in Unicode-compliant systems

This page outlines the functions implemented for Unicode support:

Upgrade Requirement

The Universal Language Support implementation required significant changes that affect the database. For this reason, if the database that you used with versions prior to version 24 did not support UTF-8, then you must migrate it to a new one that is configured for UTF-8. Upgrading from previous versions of Automic Automation to version 24 is only possible after you have migrated the database. This is also the reason why you cannot use Zero Downtime Upgrade (ZDU) for a first upgrade to an Automic Automation version that supports Unicode / UTF-8.

To help you with the database migration, an Action Pack is released with version 24 that helps you automate this process. For more information, see Migrating AE Databases to UTF-8.

Please note that, if you use an MSSQL AE DB, make sure that the source database (to perform the migration) is on version MSSQL Server 2017 or higher before migrating it. The Migration Action Pack does not support MSSQL Server versions lower than 2017. The destination AE DB has to be on version 2019 or higher to support UTF-8.

Handling Obfuscated Passwords

As of this version, obfuscated passwords start with two leading dashes ( ––10) instead of two leading soft hyphens (--10). For compatibility reasons, the Automic Automation components that process obfuscated password can handle both formats.

For passwords obfuscated in previous versions, the system checks if the clear text version behind the obfuscated password can be read with UTF-8 or not. If not, the password is de-obfuscated into clear text, obfuscated again and written back to the AE database, overwriting the previous password. The process is automatic and marked with a corresponding conversion flag in the database. For more information, see Obfuscating Passwords.

DB Schema Enhancements

Specific fields of the AE DB schema have been extended to be able to store multi-byte UTF-8 data and to fulfill customer requests for longer values.

When you load initial data for the first time into an existing AE database, its schema is also updated and, since the size of many text fields is now larger, the loading action might take longer than usual.

New Standard Character Conversion Sets in Code Tables (CODE)

With the introduction of the Unicode support we have included additional standard character encoding sets in Automic Automation. Standard character encoding sets are NOT objects. They are conversion tables that are embedded in Automic Automation's code and that cannot be changed or deleted.

When configuring File Transfers or Jobs, you have now the possibility to select one of these sets to perform the character conversion. Standard character encoding sets are easy to identify: They are displayed in angle brackets, which differentiates them from the Code Table objects.

For more information, see Code Tables (CODE)

Impact in the Automic Web Interface

The implementation of Universal Language Support (unicode/UTF-8) has an impact on AWI too.

Input Fields

AWI consists of many text input fields where you can now enter UTF-8 characters. The layout and size of these fields has not been modified. Some characters that are now supported require more space. This means that you can enter more UTF-8 characters in a text input field than it can display. In that case, a string might not be displayed at its full length. However, AWI provides resizing mechanisms to help you in these situations.

Sorting Order

By supporting the new encoding UTF-8 and its configuration, the sorting order of data might be different than in previous versions with other/legacy encodings.

Text Editors

Script objects, Pre Process/Process/Post Process pages and the Script Editor are some of the AWI components that consist of or contain a text editor. You can enter UTF-8 characters in text editors now. If you do so, you can observe the following behavior, which is usual with state-of-the-art text editors in the context of UTF-8:

If you place the cursor within a UTF-8 character (neither at its beginning nor at its end) and then enter another character, the original character is split in two parts that are not valid UTF-8 characters anymore.

Language Support

As in previous versions, Automic Automation still supports the three languages, English, German, and French. The introduction of Unicode has no impact on that.

Importing/Exporting Objects

If you import objects from previous versions that do not support Unicode, Automic Automation reads the legacy encoding and converts it to UTF-8. This means that the imported objects will display any special characters correctly. For more information, see Exporting/Importing Objects.

Enhancement in the Automation Engine scripting language

To be able to handle UTF-8 characters, all script functions have been extended and/or tested for UTF-8 support. Also, a new script function called HEX_2_STRING has been added. For more information, see Automation Engine scripting language .