dreamhouse-kafka-postgres

by srikanthannadi

GitHub Readme.md

Dreamhouse Kafka Postgres

Consumes events from Kafka and inserts them into Postgres.

Run on Heroku:

  1. Follow the instructions to deploy the Dreamhouse Web App (Kafka Branch) on Heroku

  2. Deploy on Heroku

  3. Attach the Dreamhouse Web App's Kafka to this app:

     # Get the Kafka Addon ID (e.g. kafka-something-12345)
     heroku addons:info heroku-kafka -a DREAMHOUSE_WEB_APP_HEROKU_APP_NAME
    
     # Attach the addon to the newly deployed app
     heroku addons:attach KAFKA_ADDON_ID -a DREAMHOUSE_KAFKA_POSTGRES_HEROKU_APP_NAME
    
  4. View some properties in your Dreamhouse Web App

  5. Check that the view events were consumed and then added to Postgres:

     heroku pg:psql -c "SELECT * FROM interactions" -a DREAMHOUSE_KAFKA_POSTGRES_HEROKU_APP_NAME
    

Run Locally:

  1. Install Node.js

  2. Fetch the NPM dependencies: npm install

  3. Get the Kafka environment variables from a Heroku app:

     heroku config -s -a DREAMHOUSE_KAFKA_POSTGRES_HEROKU_APP_NAME > .env
     set -o allexport
     source .env
     set +o allexport
    
  4. Start the app: npm run dev