Installing Analytics Manually

Analytics is an optional component of Automic Automation and it is fully integrated with the Automation Engine. Analytics lets you create data-rich charts and dashboards from both Automation Engine and external data. There are specific Analytics widgets that incorporate many productive features. For example, the VARA to Grid widget can help you analyze data related to KPIs and general objectives. The VARA to Chart can help you create charts based on data accessed via VARA objects. For more information, see Analytics and Reporting.

Important! This topic describes how to install Analytics manually in an on-premises Automic Automation system. This topic does not apply for Automic Automation Kubernetes Edition (AAKE) systems because Analytics is already included in the AAKE offering.

For more information about Analytics in AAKE installations, see:

This page includes the following:

Prerequisites

Analytics requires PostgreSQL for extracting the data to be used in the charts.

Important! Use the compatibility matrix to find the relevant information on supported versions, setup, or prerequisites. For more information, see compatibility matrix.

System Overview

The graphic below is a high-level overview of the installation process. The highlighted area shows where you must do something to install Analytics.

Click the image to expand it.

Analytics Infrastructure

Analytics consists of three main components. This graphic illustrates the Analytics infrastructure and highlights the main components in yellow:

  • The Analytics backend is responsible for data exchanges between the Automation Engine and Analytics.

    The Analytics backend is bundled into the IA Agent that is delivered with the Analytics package. It connects to the Analytics datastore and to the Automation Engine database through the application.properties file, and to the Automation Engine through the ucxedda.ini file. Its core is the analytics-backend.jar file.

    To start the Analytics backend, you start the IA Agent.

  • The PostgreSQLAnalytics datastore, the Analytics database that extracts the data from the Automation Engine to be used for reporting purposes.

  • The Web UI plugin that contains the plugin.properties configuration file with the REST API encryption key for security and connection details to the Analytics database.

Installation Steps - Overview

This list summarizes the procedure to install Analytics. The following sections in this topic describe each step in detail.

  1. Copy the Analytics files.

    The Automic Automation install package that you have downloaded contains all the Analytics files that you need.

    Copy the Analytics folder available under Automic.Automation_<version number>/Automation.Platform in the installation package to your Automic Automation directory where you want to run Analytics.

  2. Install and configure the Analytics datastore.

    1. Install PostgreSQL if you have not already done so.

    2. Configure the database.

    3. Deploy the Analytics data structures.

    For more information, see Installing and Configuring the Datastore.

  3. Install and configure the backend

    1. Configure the backend using the application.properties file so that it can interact with the Analytics datastore and the Automation Engine database.

    2. Copy the JDBC driver to enable data connections.

    3. Update ucxdda.ini to allow connection between the backend and the Automation Engine.

    4. Configure the Service Manager for Analytics.

    For more information, see Installing and Configuring the Backend.

  4. Install and configure the Web UI plugin

    1. Copy the Web UI plugin over to the autoinstall folder.

    2. Update the plugin.properties file to enable the communication between the plugin and the backend over the REST API.

    For more information, see Install and configure the Analytics Web UI plug in. .

Installing and Configuring the Datastore

PostgreSQL must be installed and configured for Analytics to run. The PostgreSQLAnalytics datastore extracts the data to be used for reporting purposes. If you have not already done so, install the appropriate PostgreSQL version now. Download it from www.postgresql.org/download/

When installing PostgreSQL, you will have to set the super user password. Keep this password at hand, you will need it later.

This topic does not cover the PostgreSQL installation procedure. It only touches on aspects of the installation that are relevant for Analytics and describes how to configure it. For information about the PostgreSQL installation, please refer to the official PostgreSQL documentation.

Important! Use the compatibility matrix to find the relevant information on supported versions, setup, or prerequisites. For more information, see compatibility matrix.

Note: To optimize database performance and efficiency, see: Maximizing Efficiency with the Datastoreat Automic Automation System Requirements and Sizing.

PostgreSQL on Windows

  • The PostgreSQL installer for Windows can be run in graphical or in silent install mode.

  • It is not necessary to install the Stack builder.

PostgreSQL on Linux

Make sure that the tcp/ip connection is enabled (localhost).

To Configure PostgreSQL

  1. Configure PostgreSQL so that it can accept connections from any user and host. You do this in the pg_hba.confPostgreSQL file.

    1. Go to PostgreSQL/<version number>/data and open pg_hba.conf.

      Note: By default, the PostgreSQL files are stored in Program Files.

    2. In pg_hba.conf, add the following:

      TYPE: host

      DATABASE: all

      USER: all

      ADDRESS: 0.0.0.0/0

      METHOD: md5

  2. To upload the Analytics data structures to PostgreSQL, execute a script called setup.psql that is provided with the Analytics installation package. The script is located under <installation directory>\Automation.Platform\Analytics\Datastore\setup.psql.

    Execute the setup.psql script from the Analytics datastore directory using the following PostgreSQL psql command:


    [path to postgres bin]\psql -h vmwin -p 5432 -U postgres -W -v username="analytics" -v password="'analytics'" -v db_name="analytics" -f setup.psql

    Where:

    • -h is the host where the database is

    • -p is the port (by default, this is 5432)

    • -U is the super user

    • -W means no super user password

    • -v are the script parameters for the database username, the password and the name of the database

    • -f designates the script that will be executed.

    Tip: Keep the Analytics defaults, all in lowercase

    When executing the script you will be prompted to enter the PostgreSQL super user password.

    The script creates the following:

    • A role

    • A database, including the following tables: ah, ah_cleaned, an_enricher, api_key, ara-execution, ara-execution_cleaned, ara-execution_enricher, configuration, lasim, lasim_cleaned, schema_version, scoped_api_key, shared_dashboard, status.

      The setup.psql script creates the initial api_key table, and the rest are created when the sync starts.

    • The rest of the datastore

    • The backend API key

      This is the key that secures the communication between the different components and the Analytics backend.

Important! Copy the contents of the command window to a text file. You will need the API key as well as the datasource URL, username and password later.

The datastore is installed and configured. You must install and configure the Analytics backend now.

Installing and Configuring the Backend

The Analytics backend pulls the data from the Automation Engine and makes it available to the web interface. You need to configure the backend so that the data exchange is possible.

The Automation Engine and the Windows, UNIX, and Java Agents communicate using TLS/SSL. These agents establish a connection with the Java communication process (JCP), which uses trusted certificates to prove their identity to other communication partners.

Important! Make sure you are familiar with the TLS/SSL and certificate implementation before installing and/or upgrading the respective component. For more information, see:

When you used certificates signed by a CA, the certificates are stored in the respective Java or OS store by default; that is the Java trust store for Java components and Java Agents, the Windows OS store for Windows Agents, or the TLS/SSL store for UNIX Agents. In this case, you only have to check that the root certificates already are in the respective store.

If the relevant certificates are not there and you want to import them, you can use OS or Java specific tools for that purpose, such as Keytool, cert-manager, OpenSSL and such. For more information on how to use those tools, please refer to the respective product documentation.

If you do not want to use the default locations for the components and Agents listed above, make sure you use the trustedCertFolder=, agentSecurityFolder=, and keyPassword= parameters (if applicable) in the respective configuration (INI) file to define the path to the folder where the trusted certificates are stored.

Important! TLS/SSL Agents (in containers and on-premises) as well as the TLS Gateway, when used for the Automic Automation Kubernetes Edition, establish a connection to an ingress / HTTPS load balancer, which requires a certificate for authentication.

Make sure that address of the load balancer is defined on both sides: the Automation Engine and the Agent / TLS Gateway and that your HTTPS load balancer has the required certificates in place. For more information, see Connecting to AWI, the JCP and REST Processes Using an Ingress.

To Install the Analytics Backend

  1. Unzip the content of the deployment package (Analytics.On.Premise_Analytics.Backend_ .zip) into the folder where you want to run the backend.

    Package content

    • analytics-backend.jar

    • application.properties.sample

    • logging.xml

    • README.txt

    • THIRD-PARTY-LICENSES.txt

    • UC.msl

    • UCXedda.ini

  2. Configure the backend to connect to the Analytics datastore and the Automation Engine database.

    The backend has a configuration file called application.properties.sample in the Analytics/backend folder. You configure the connection here.

    1. Copy application.properties.sample and rename it to application.properties to keep the original file and do the configuration in the renamed one.

    2. To specify the connection to the Analytics datastore, open application.properties and enter the datastore URL, username and password. You find these values in the text file where you stored the content of the command window after executing the setup.psql script.

      Example:

    3. To specify the JDBC connection to the Automation Engine database, in application.properties enter the database URL, username and password.

      1. In the text file, write the following:

        ALTER SESSION SET CURRENT_SCHEMA=automic

        This will set the schema to automic while retaining the already configured database username and password.

      2. In application.properties enable the following setting:

        datasource.AE.schema=file:AE_schema.sql

      Important! In some Oracle versions the user name and password are case-sensitive.

      DB2

      1. Specify a read-only user for the DB2 schema. For this purpose, create a text file in the Analytics backend directory. You can call it AE schema.sql, for example.

      2. In the text file, write the following:

        SET SCHEMA=automic

        This will set the schema to automic while retaining the already configured database username and password.

      3. In application.properties enable the following setting:

        datasource.AE.schema=file:AE_schema.sql

      PostgreSQL

      You find the database URL in the JDBC section, sqlDriverConnect= parameter in the Automation Engine's UCSrv.ini file. The username and the password were set when the Automation Engine database was created.

    4. Save and close application.properties.

    5. To enable the connection to the Automation Engine database, create a jdbc folder and copy the correct JDBC driver from /bin/lib in your Automic Automation installation directory to the new jdbc folder.

      The JDBC drivers are:

      • MSSQL:./jdbc/sqljdbc4.jar

      • DB2: ./jdbc/db2jcc-4.jar

  3. To connect the Analytics backend to the Automation Engine, enter the name of the Automation Engine system, the host and the port in the ucxedda.ini file that is available in the Analytics Backend directory.

    Example:

    ; system: AE system name

    ; The name can consist of a maximum of 8 characters. Allowed are the upper-case letters A to Z, numbers and "_".

    system=<name of your AE system>

    ...

    ; jcp: Address of the JCP communication process in the AE system to which the component should connect.

    ; Allowed formats:

    ; DNS Name:Port number

    ; TCP/IP Address:Port number

    jcp=<host>:8443

  4. To configure the Service Manager to be able to control Analytics and to set it to start automatically at boot, update the UC4.smd and UC4.smc files. They are located in the ServiceManager/bin folder in your Automic Automation installation directory.

    1. Stop the Service Manager.

    2. Open the UC4.smd file and enter the following:

      DEFINE ANALYTICS-BACKEND;C:\Program Files\jdk-11.0.14\bin\java -Xmx2g -Xms512m -XX:+UseG1GC -jar analytics-backend.jar;C:\<Analytics installation directory>\Analytics\backend

      Where:

      • DEFINE ANALYTICS-BACKEND is an AE instruction to create the Analytics service

      • C:\Program Files\jdk-11.0.14\bin\java -Xmx2g -Xms512m -XX:+UseG1GC -jar analytics-backend.jar is a Java command from the Java bin directory with some standard arguments. Keep them as they are.

      • C:\<Analytics installation directory>\Analytics\backend is the directory from which analytics-backend.jar is executed

    3. Save and close UC4.smd.

    4. Open the UC4.smc file. It contains the autostart and timing instructions.

      Enter the following:

      WAIT 3

      CREATE ANALYTICS-DATASTORE

      WAIT 14

      CREATE ANALYTICS-BACKEND

      For information about how to secure the backend, see Securing Connections to the AE (TLS/SSL).

    5. Save and close UC4.cmd.

  5. Start the IA Agent from the Service Manager. This starts the Analytics backend.

    To confirm that the IA Agent is online, log in to Automic Automation and go to the Administration perspective > Agents list.

    Troubleshooting: If the IA Agent is not online, check the Agent (ucxedda.log) and the backend (analytics-backend.log) log files.

  6. Install and configure the Analytics Web UI plug in.

    1. Copy the webui-analytics-plugin.jar file from the Analytics/plugin folder in your Analytics installation directory to the autoinstall folder in your Automic Automation installation directory.

      For installations with the Jetty Launcher, this is /plugins/autoinstall. For more information, see Content of the Delivery Artifact.

      For Tomcat installations, this is \webapps\AWI\WEB-INF\autoinstall.

    2. To configure the plugin so that it can access the backend, open the plugin.properties file and enter the following:

      • backend.api_key=

        This key secures access to the backend REST API. It was generated by the setup.psql script when you set up the datastore. When setting up the datastore, you copied this key to a text file.

      • backend.endpoint=

        This is the hostname of the Analytics Backend host

      The plugin.properties file is located in the autoinstall folder in your Automic Automation installation directory. For more information, see Content of the Delivery Artifact.

Analytics is installed and configured now.

Configuring the Analytics Cache

The Analytics cache can speed up repetitive database queries by keeping results in memory until a configured expiration time is reached. The default values can be found in the application.properties file.

Default Cache Values:

#########################

## Cache configuration ##

#########################

# Enable caching of charts

service-cache.enabled=true

# Request cache expiration

service-cache.expiration_minutes=120

# Request cache refresh

service-cache.refresh_minutes=2

# Specify how many requests should stay cached at maximum

service-cache.size=10000

# Specify the mode of the expiration process

# Possible values: after_access, after_write

service-cache.expiration_mode=after_write

Where:

  • service-cache.enabled turns the cache on and off

  • service-cache.expiration_minutes lets you to set the expiration minutes value.

    This parameter controls the deletion time for cached values. If your database queries take more than two minutes, you should change the expiration time to a value between 30 and 120 minutes. These settings depend on your data freshness requirements.

  • service-cache.refresh_minutes determines when the cache should be refreshed in minutes

  • service-cache.size defines the number of items you would like the cache to hold

    If you are using more than 10 different dashboard widgets, you should set your cache size to 10000 items.

  • service-cache.expiration_mode determine how the expiration timer is started

    If you set the parameter to after_write the timer starts after the value has been written to the cache. If you set the parameter to after_access the timer starts after the cache value has been accessed. An example when the timer starts is when you open a dashboard and a chart is displayed.

Securing the Backend

As a Backend Administrator you can secure communications to restrict access to authorized users.

The UI plug-in communicates with the Analytics backend using HTTPs. A pre-shared API key is used to protect the data in the Analytics backend and restrict requests to authorized API users only. This key is created during the datastore setup.

Diagram showing how backend is secured using HTTPs

To Secure the Backend

  1. Enable TLS/SSL on the backend by setting the following in the application.properties file:

    server.port=8443

    # next 3 properties must be deactivated (commented out) if you want to use the Backend with http

    server.ssl.key-store=classpath:localhost.p12

    server.ssl.key-store-password=analytics

    server.ssl.key-password=analytics

    A valid TLS/SSL certificate (localhost.p12 in this example) must be within the root classpath of the backend application.

  2. Set the front end to call the Backend with HTTPS.

    The URL to call the Backend is set in the webui-plugin-reporting/plugin.properties file.

    backend.endpoint=https://localhost:8443

    backend.endpoint.verifyCertificate=true

    Where:

    • backend.endpoint.verifyCertificate=true means that the certificate is properly checked (it must be from a certified, known provider). If you want to use a self signed certificate, you must add it to the local java cacert file.
    • backend.endpoint.verifyCertificate=false means that the certificate is not validated.

To Set the Analytics Action Pack to Use HTTPs

  1. Click Process Assembly perspective.

  2. Open the PACKAGES folder and select the PCK.AUTOMIC_ANALYTICS Action Pack.

    If this package is not available, you can download it from our marketplace at https://marketplace.automic.com/

  3. Double-click ACTIONS and again ANALYTICS. The Analytics Actions are displayed in the list.

  4. Execute the PCK.AUTOMIC_ANALYTICS.PUB.ACTION.ANALYTICS_CONFIGURATION Action to configure the Analytics Action Pack.

  5. Select Requests in the menu bar.

    The content of the request contains the fields for setting the Analytics Action Pack to use HTTPs.

  6. Enter the following Analytics backend URL: https://<your.analytics.host>:8443

  7. Add the self-signed certificate to the Java installation on the Agent you decide to execute the Analytics Actions.

    Notes:

    • The self signed certificate is located in the local CAcert trustStore of the java version that is used to execute the analytics-groovy-runner.jar file.

    • You can also ignore TLS/SSL issues arising from untrusted certificates and host name mismatches. To do so, select the Yes radio button.

    • Once you have configured your agent to use the HTTPs, all services that run on the agent are switched to HTTPs. The HTTP interface will not longer work.

    • When you use a certificate to secure your interface, the certificate is applied to all services running on the IA Agent.

  8. Click Submit

See also: