Setting up the Streaming Platform
On Windows
Manual setup instructions for the Streaming Platform on Windows are currently not provided.
It is recommended that you use the Linux installation in a production environment.
Note: If you would like to evaluate the Streaming Platform on Windows, use ONE Installer to setup a single-node system.
For more information, see: ONE Installer - Single-Box Installation
On Linux
Follow these steps to set up the Streaming Platform.
Step 1: Download the required zip file from ca.com
-
Log in to the Support page and click on Download Management .
- Using the search bar, enter CA Automic followed by package you want to download.
Example: Automic Automation -
After downloading the file, unzip it to a temporary directory.
- Locate the kafka tgz file in the following directory: <Automic>/External.Resources/kafka.
- Untar the kafka tgz file. We will copy this file to the target installation machine in Step 2 below.
Step 2: Start the Streaming Platform
- On the target installation machine, create a new directory that looks like this: <automic>/External.Resources/kafka.
- Copy the untarred kafka tgz file content into: <automic>/External.Resources/kafka directory. The directory should look like this:
-
Copy the following script into Notepad (or a text editor) and save it as start.sh in the following directory: <Automic>/External.Resources/kafka/bin.
Important!
- Note that in the following script you must replace the %JDK% variable with your actual jdk path.
- Replace %ZOOKEEPER_CONNECT% variable with the actual [host]:[port] of Zookeeper (for example the default is localhost:2181).
#!/usr/bin/env bash echo Starting up Zookeeper stop_kafka() { echo Stopping kafka > kafka_stop.out ./kafka-server-stop.sh echo Stopping zookeeper >> kafka_stop.out ./zookeeper-server-stop.sh echo Shutdown finished >> kafka_stop.out exit } trap stop_kafka SIGHUP SIGINT SIGTERM SIGUSR1 SIGKILL export JAVA_HOME=%JDK% export PATH=${JAVA_HOME}/bin:$PATH ./zookeeper-server-start.sh ../config/zookeeper.properties > zookeeper.out 2>&1 & while true do cmd=$(ps aux | grep "$JAVA_HOME/bin/java.*zookeeper-gc\.log.*" | grep -v "grep" | wc -l) echo $cmd if [ $cmd -ge 1 ] then echo Zookeeper is running break else echo NOT RUNNING sleep 1 fi done # sleep a few extra seconds to make sure that the port is actually available sleep 4 ./zookeeper-shell.sh %ZOOKEEPER_CONNECT% rmr /brokers ./kafka-server-start.sh ../config/server.properties > kafka.out 2>&1 & while true do sleep 1 done
- In the Automic/External.Resources/kafka directory, create a new sub-directory called zk-tmp. This directory will hold all your zookeeper data files.
-
Open the zookeeper.properties file located here: <automic>/External.Resources/kafka/config. Find the dataDir parameter and edit it to reflect the newly created zk-tmp directory.
Example: dataDir=/automic/External.Resources/kafka/kafka/zk-tmp
- In the <automic>/External.Resources/kafka/bin directory under root user make the file executable for example: run the command chmod +x ./start.sh.
bin
config
libs
site-docs
NOTICE
Step 3: Add the Streaming Platform to the Service Manager SMD file
Note: For steps 3 and 4 you must have Service Manager installed. If you do not have Service Manager installed, see the following links: ServiceManager, Start Parameters - ServiceManager.
- If you have installed Service Manager open the uc4.smdfile found here : <automic>/Automation.Platform/ServiceManager/bin/uc4.smd. Using your defined directory path, add a DEFINE STREAMING-PLATFORM line as shown in the example below:
DEFINE STREAMING-PLATFORM;<automic>/External.Resources/kafka/kafka/bin/start.sh;
<automic>/External.Resources/kafka/kafka/bin
Step 4: Add the Streaming Platform to the Service Manager SMC file
- Open the uc4.smcfile found here:<automic>/Automation.Platform/ServiceManager/bin/uc4.smc andinsert the following two lines (in bold) and close and restart the service manager or simply start the STREAMING-PLATFORM service:
WAIT 10
CREATE RULE-ENGINE
WAIT 3
CREATE ANALYTICS-DATASTORE
WAIT 14
CREATE ANALYTICS-BACKEND
WAIT 3
CREATE STREAMING-PLATFORM
Step 5: Adapt the Configuration file
Note: This step can not be completed until you install the Analytics Backend. If you have not done this, take note of the following host and port configuration.
- Some adaptions must be made in the application.properties configuration file. This file can be found here: <automic>/Automation.Platform/Analytics/backend/application.properties:
Set the communication parameters for the Backend to communicate with Kafka:
kafka.bootstrap_servers=hostIP:9092
Note: The hostIP and port see the machine that Kafka is installed on.
Set the communication parameters for the Backend to communicate with Zookeeper:
zookeeper.bootstrap_servers=hostIP:2181
Notes:
- The hostIP and port, check the machine that Zookeeper is installed on.
- If you are using the Event Engine feature and the Streaming Platform becomes unavailable, Rules that are running on the Rule Engine will fail after a timeout. The default timeout is 5 minutes (305000 milliseconds) and it can be adapted if needed. To change the timeout, configure the following:
kafka.consumer_configs.edda_events[request.timeout.ms]=305000
Step 6: Secure the Streaming Platform (optional)
For further information see: TLS/SSL Setup between Backend and Streaming Platform