kafkacat-buildpack

by trevorscott

GitHub Readme.md

kafkacat-buildpack

Heroku buildpack for kafkacat, a command line based Apache Kafka producer and consumer.

Makes an auto-configured kafkacat command available on the dyno.

Installs kafkacat version 1.3.1 with librdkafka 0.11.0.

Set Up

Your app must have the Apache Kafka on Heroku add-on attached to use this buildpack.

Add this buildpack as the first one:

heroku buildpacks:add \
  --app $APP_NAME \
  --index 1 \
  https://github.com/trevorscott/kafkacat-buildpack

✏️ Replace $APP_NAME with the name for your specific Heroku app.

If your app didn't already have a buildpack explicitly set, then it was relying on autodetect of an officially supported buildpack. In that case, add the required language buildpack as the last one. Example: heroku buildpacks:add heroku/nodejs

Default Config

The Kafka add-on creates four config vars:

  • KAFKA_TRUSTED_CERT
  • KAFKA_CLIENT_CERT
  • KAFKA_CLIENT_CERT_KEY
  • KAFKA_URL

This buildpack automatically configures kafkacat with these config vars, so it connects to a broker in the Kafka cluster with SSL by default. The configuration uses the first broker URL of the KAFKA_URL config var. (See the .profile.d script and the configured kafkacat command for more details.)

Usage

Since our version of kafkacat comes preconfigured, you should omit SSL & broker configuration.

For example, to read messages from the topic your-topic:

$ kafkacat -t your-topic

Or to write a message to your-topic:

echo "This is a good message." | kafkacat -P -t your-topic

See the kafkacat README for more examples.

Development

Save build to s3

git clone https://github.com/trevorscott/kafkacat-buildpack.git && cd kafkacat-buildpack
heroku create kafkacat-buildpack-appname
heroku buildpacks:set -i 1 https://github.com/trevorscott/heroku-buildpack-inline
heroku config:set AWS_SECRET_ACCESS_KEY=<your-secret>
heroku config:set AWS_ACCESS_KEY_ID=<your-access-key>
git push heroku master
heroku run bash

...and then once in your dyno's bash session...

$ tar -zcvf kafkacat.tar.gz kafkacat-binary/ 
$ source bin/s3
$ putS3 kafkacat.tar.gz