The following steps will prepare you for the creation of the Amazon MSK cluster in this lab.
Note that running this module will launch AWS resources that have an associated cost. If you are not running this lab as part of an Amazon MSK workshop using provided accounts, remember to clean up when you are done to keep from incurring ongoing charges for resources left running.
If you want to keep your lab work separate or you’re using an existing account and don’t want to overlap with others, you can create a VPC and Subnets to work in. Follow these easy steps to get started:
Ensure that you’ve created an SSH key in your account
Use CloudFormation to launch an Amazon MSK focused VPC
The stack creates:
If you wish, you can download the CloudFormation template here
You can now proceed with the prep and lab. Ensure that you use the new VPC when you’re completing the lab, it won’t be the default so make sure you pick it from the drop downs presented.
By default, the cluster will be attached to the ‘default’ security group, which allows all ports between all members of the group. This is fine for testing, but it’s not a best practice in production.
We need two security groups - one to attach to producers, consumers, and admin hosts, and the other to attach to the Amazon Amazon MSK cluster that references the first.
The CloudFormation template already created one of them - the “client” security group. Look at the Outputs tab of the CloudFormation stack created in the previous step and copy the value for the key KafkaClientEC2InstanceSecurityGroupId.
Create the security group for the Amazon MSK cluster.
Click on Services
in the top left corner of the console, and select EC2
Select Security Groups in the left pane
Click Create Security Group
Fill out the form as follows:
Security group name: MSKWorkshop-KafkaService
Description: Access to the Kafka service on the MSK cluster
VPC: [select the VPC you are using for your lab (AWSKafkaTutorialVPC)]
Create rules
a. Click Add rule
Use:
KafkaClientEC2InstanceSecurityGroupId
SG you copied in the previous step, from Cloudformation Outputs (msklab-KafkaClientInstance…)]
Source2: Add the Cloud9 security group by typing cloud9
into the custom field - you’ll see something like aws-cloud9-msklab...
)c. Click Add Rule
Use:
cloud9
into the custom field - you’ll see something like aws-cloud9-msklab...
)e. Click Add Rule
Use:
cloud9
into the custom field - you’ll see something like aws-cloud9-msklab...
)Click Create
Example:
In the security group list, select the Group ID
for the MSKWorkshopKafkaService
SG, and copy it to your notepad/texteditor
You are done! Now you can move on to the lab.