Install Cruise Control

Process

On the cloud9 instance open a terminal window:

  1. Install Java8 on the cloud9 instance
sudo yum -y install java-1.8.0-openjdk java-1.8.0-openjdk-devel
sudo alternatives --config java 

You will be asked to select a version. It should be #2, /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java.

example:

mitchadmin:/usr/lib/jvm $ sudo alternatives --config java                                                                                                                                       

There are 2 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
*+ 1           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
   2           /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java

Enter to keep the current selection[+], or type selection number: 2
mitchadmin:/usr/lib/jvm $ java -version
openjdk version "1.8.0_265"
OpenJDK Runtime Environment (build 1.8.0_265-b01)
OpenJDK 64-Bit Server VM (build 25.265-b01, mixed mode)

Repeat the process for javac.

mitchadmin:~/cruise-control-2.4.25 (master) $ sudo update-alternatives --config javac                                                                                                           

There are 2 programs which provide 'javac'.

  Selection    Command
-----------------------------------------------
*+ 1           /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/javac
   2           /usr/lib/jvm/java-1.8.0-openjdk.x86_64/bin/javac

Enter to keep the current selection[+], or type selection number: 2
  1. Download a cruise-control release from https://github.com/linkedin/cruise-control/releases.

    cd ~/
    wget https://github.com/linkedin/cruise-control/archive/2.5.22.tar.gz
    
  2. You will need to extract the source code from the file you just downloaded and go into its directory

tar -xvzf 2.4.25.tar.gz
cd cruise-control-2.4.25/
  1. To build Cruise Control you must initialize it as a git repo,
git init && git add . && git commit -m "Init local repo." && git tag -a 2.0.130 -m "Init local version."
  1. Build cruise control with
./gradlew jar copyDependantLibs
  1. Get the Amazon MSK ClusterArn. Note that if you have more then one active Amazon MSK cluster you will need to set MSKARN manually.
export MSKARN=`aws kafka list-clusters|grep ClusterArn|cut -d ':' -f 2-|cut -d ',' -f 1 | sed -e 's/\"//g'`
  1. Set the bootstrap.servers to be used for configuring Cruise Control.
export MYBROKERS=`aws kafka get-bootstrap-brokers --cluster-arn $MSKARN|grep BootstrapBrokerString|grep 9092| cut -d ':' -f 2- | sed -e 's/\"//g' | sed -e 's/,$//'`
  1. Set the Zookeeper Connect string for configuring Cruise Control.
export ZOOKEEPER=`aws kafka describe-cluster --cluster-arn $MSKARN|grep ZookeeperConnectString|grep -v Tls|cut -d ':' -f 2-|sed 's/,$//g'|sed -e 's/\"//g'`

  1. Edit config/cruisecontrol.properties to fit your environment.
sed -i "s/localhost:9092/${MYBROKERS}/g" config/cruisecontrol.properties
sed -i "s/localhost:2181/${ZOOKEEPER}/g" config/cruisecontrol.properties
sed -i "s/com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler/com.linkedin.kafka.cruisecontrol.monitor.sampling.prometheus.PrometheusMetricSampler/g" config/cruisecontrol.properties
sed -i "s/webserver.http.port=9090/webserver.http.port=9091/g" config/cruisecontrol.properties 
sed -i "s/capacity.config.file=config\/capacityJBOD.json/capacity.config.file=config\/capacityCores.json/g" config/cruisecontrol.properties
echo "prometheus.server.endpoint=localhost:9090" >> config/cruisecontrol.properties
mkdir logs; touch logs/kafka-cruise-control.out

Note: that prometheus.server.endpoint= is the host that is currently running prometheus as built in the Open Monitoring Lab.

  1. Modify config/capacityCores.json. For m5.large instance type:
{
  "brokerCapacities":[
    {
      "brokerId": "-1",
      "capacity": {
        "DISK": "10737412742445",
        "CPU": {"num.cores": "2"},
        "NW_IN": "1073741824",
        "NW_OUT": "1073741824"
      },
      "doc": "This is the default capacity. Capacity unit used for disk is in MB, cpu is in number of cores, network throughput is in KB."
    }
  ]
}
  1. Run Cruise Control from inside the decompressed Cruise Control directory.
./kafka-cruise-control-start.sh -daemon config/cruisecontrol.properties
  1. After a few minutes, in tail logs/kafkacruisecontrol.log you should see something like:
[2020-11-09 16:01:30,060] INFO Kicking off metric sampling for time range [1604937570060, 1604937690060], duration 120000 ms with timeout 120000 ms. (com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager)
[2020-11-09 16:01:30,141] INFO Generated 279 partition metric samples and 6 broker metric samples for timestamp 1604937690000. (com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsProcessor)
[2020-11-09 16:01:30,142] INFO Collected 279 partition metric samples for 279 partitions. Total partition assigned: 279. (com.linkedin.kafka.cruisecontrol.monitor.sampling.SamplingFetcher)

This verifies that Cruise Control is collecting metrics from Prometheus.