Go back to the AWS Cloud9 console. Click on MSKClient-Cloud9EC2Bastion.
Click on Open IDE (or go back to the browser tab if already open).
You should be connected to the KafkaClientEC2Instance instance. If not, ssh to the KafkaClientEC2Instance instance by pasting the ssh command you copied in the last section.
cd /tmp/kafka git clone https://github.com/aws-samples/integration-sample-lambda-msk.git cd integration-sample-lambda-msk
Run the deploy.sh script. The script does the following. (You can cat the script to look at its contents.)
Note: Change the values of MSK_STACK and KAFKA_CLIENT_STACK if you changed them from the default values.
Note: This Lambda function uses the AWS Glue Schema Registry by default to deserialize messages. The CSR parameter indicates use of a 3rd party Schema Registry instead of the default AWS Glue Schema Registry. Since, in this lab we’re using the default AWS Glue Schema Registry, the CSR parameter is set to false. The SECONDARY_DESERIALIZER parameter allows the AWSKafkaAvroDeserializer that integrates with the AWS Glue Schema Registry to use a specified secondary deserializer that points to a 3rd party Schema Registry. This enables the AWSKafkaAvroDeserializer to deserialize records that were not serialized using the AWS Glue Schema Registry. This is primarily useful when migrating from a 3rd party Schema Registry to the AWS Glue Schema Registry. If the SECONDARY_DESERIALIZER parameter is set to true, the Lambda function can seamlessly deserialize records using both a 3rd party Schema Registry and the AWS Glue Schema Registry. However, in this lab we will only use the AWS Glue Schema Registry.
export MSK_STACK=MSK export KAFKA_CLIENT_STACK=MSKClient export VPC_STACK=$(aws cloudformation describe-stacks --stack-name $MSK_STACK --query 'Stacks.Outputs[?OutputKey==`VPCStackName`].OutputValue' --output text) export SECONDARY_DESERIALIZER=false export CSR=false ./deploy.sh $VPC_STACK $KAFKA_CLIENT_STACK $MSK_STACK $SECONDARY_DESERIALIZER $CSR
At this point, everything is setup and the Lambda function is already processing records from Amazon MSK. We will look at the environment in the next section.