In order to use Open Monitoring with Prometheus, you need to have completed the steps in this preparation guide.
The critical step to using Open Monitoring is that you need to enable it. It’s not enabled by default on all clusters, so if you didn’t enable it when you created your cluster, you’ll have to turn it on.
Any clusters created as part of this workshop will have Open Monitoring enabled, but if you’re using this against an existing cluster, this section will help ensure you’re ready.
Sign in to the account where the Amazon MSK cluster is you want to monitor
Open the Amazon MSK Cluster Console
Click on the Amazon MSK cluster you want to enable monitoring on
Scroll to the Monitoring section
If you have already enabled Open Monitoring, you will see a screen similar to this - you can continue to the next preparation step if so:
If you haven’t enabled Open Monitoring, you will see a screen similar to this:
Enable Open Monitoring
Since it’s not enabled, lets turn it on!
Enable open monitoring with Prometheuscheckbox
JMX Exporterand the
This will trigger a configuration update on the cluster hat will take a couple minutes to apply. You will see a blue bar indicated the operation is in progress. When it is complete, the bar will turn green to indicate the operation is complete.
You are set! Now you can move on to Step 2 and make the monitoring ports accessible.
Create a new SG called
MSK_Monitoring with no rules
Security Groupsin the left navigation pane
Create Security Groupat the top of the screen
Access to MSK monitoring from monitoring services
Create, having added no rules
MSK Workshop ServiceSG to include 2 new rules
Open the EC2 service, then go to Security Groups
Click on the
MSKWorkshop-KafkaService then in the bottom pane click
Edit and add the following rule
Note: In the
Source field, simply start typing
MSK_monitoring and it will populate with the security group ID.
Attach the MSK_Monitoring SG to your KafkaClientInstance instance and your Cloud9 instance
Actionsfrom the top bar
Change Security Groups
Assign Security Groupsin the bottom right corner
This will be used shortly