A specialized lightweight CI server for building Salesforce projects from Github repositories using CumulusCI flowsLicense:BSD
MetaCI started as an extension of the CumulusCI 2 (https://github.com/SFDO-Tooling/CumulusCI/tree/feature/2.0) project. After spending almost a year trying to find a cloud hosted CI service that could handle our needs for Salesforce managed package builds, the crazy idea was born: why not just write our own CI server specific to our needs?
A few key things to point out that made this compelling:
So, MetaCI was started as a prototype of a crazy idea mid-November 2016, got prototype buy in mid-December 2016, and is going into production to replace Bamboo Cloud before January 31, 2017.
So, what does this thing actually do?
You can also fork the CumulusCI-Test repository and use that as a demo since it is already configured for CumulusCI.
The recommended way to deploy and configure MetaCI is via the MetaCI-CLI project which can access your local CumulusCI project configurations including repo info and org configs to quickly get you up and running with MetaCI.
Full documentation on launching and configuring MetaCI on Heroku using the MetaCI-CLI: http://metaci-cli.readthedocs.io/
You can also reference the following session recordings showing live demos of how to configure CumulusCI and MetaCI for projects: https://www.youtube.com/watch?v=CgvYKDqb6Ng https://www.youtube.com/watch?v=TN-e785MqBA
Or, if you feel like deploying manually and configuring through the Admin interface:
To create a normal user account, just go to Sign Up and fill out the form. Once you submit it, you'll see a "Verify Your E-mail Address" page. Go to your console to see a simulated email verification message. Copy the link into your browser. Now the user's email should be verified and ready to go.
To create an superuser account, use this command:
$ python manage.py createsuperuser
For convenience, you can keep your normal user logged in on Chrome and your superuser logged in on Firefox (or similar), so that you can see how the site behaves for both kinds of users.
This app comes with RQ, a Redis message queue library for Python.
Check the Procfile to see the commands used to run the workers on Heroku. You can run a single local worker that watches all queues with:
python manage.py metaci_rqworker high medium default short --worker-class metaci.build.worker.RequeueingWorker
The first step is to add your repositories. Go to https://<your_app_name>.herokuapp.com/admin and log in using the admin user you created earlier. Go to Repository and click Add.
Enter the repo name, owner name, and the url. Currently only repositories on github.com are supported. The github id will be automatically looked up for you so you can leave it blank.
Any org you connect to your local CumulusCI keychain can be added to MetaCI as a build org. Go to CUMULUSCI -> Orgs -> Add and give the org a name, select the repo, and paste in the results of cumulusci2 org info <org_name> on your local system. Remember that org names are already namespaced by their repository so rather than package_name_feature, just call the org feature.
For a few flows, you need to have the github service configured in CumulusCI. On your local system, run cumulusci12 project show_github to get the json to load add the github service under Service -> Add. If you get an error, run cumulusci2 project connect_github to configure the github service in your local system then run show_github again.
Plans are what ties together a repository, org, and CumulusCI flows. Plans can have the following trigger types:
When you create Commit or Tag plans, the webhook should be automatically created in the repository to listen on the Github push event. Creating the webhook requires that the GITHUB_USERNAME you used in the Heroku config for the app is an admin on the repository.
Additionally, you can define a Plan Repository Trigger that will trigger a plan based on another plan. For example, you could create a trigger such that when Plan X for Repository A completes successfully, Plan Y for Repository B is queued. This is especially helpful when building against upstream dependencies.
You can set Plans and Repositories and Private. When a Plan or Repository is private, the Plan or Repository and its builds will not show up in the public view. They will show up for any user with the is_staff permission.
To set up user logins using Github, go to /admin and create a new Social App. Create a new OAuth Application in your Github Settings on github.com to get the client id and secret info. Once created, have your users go to https://<your_app_name>.herokuapp.com/accounts/github/login to login via Github. Once they log in you can go to Users under admin and check the is_staff field for your staff users.
Click the bell icon at the top to view the My Notifications page (/notifications) where you can view and add your notifications.
MetaCI can be configured to monitor its own build queue and scale its own Heroku dynos based on load in multiple Heroku Apps. It will check the queue once a minute and add worker dynos when needed. Once all builds are complete, all worker dynos will be shut down. Heroku only bills for the dyno seconds used, so this scaling can save money while allowing for greater concurrency when desired.
To configure autoscaling:
heroku authorizations:create. Set the HEROKU_TOKEN setting to this authorization token.
metaci.build.autoscaling.autoscale, Queue =
short, and Interval =
In development, it is often nice to be able to see emails that are being sent from your application. If you choose to use MailHog when generating the project a local SMTP server with a web interface will be available.
To start the service, make sure you have nodejs installed, and then type the following:
$ npm install $ grunt serve
(After the first run you only need to type
grunt serve) This will start an email server that listens on
127.0.0.1:1025 in addition to starting your Django project and a watch task for live reload.
To view messages that are sent by your application, open your browser and go to
The email server will exit when you exit the Grunt task on the CLI with Ctrl+C.
In Production, set up Mailgun as a Heroku addon.
Sentry is an error logging aggregator service. You can sign up for a free account at https://getsentry.com/signup/?code=cookiecutter or download and host it yourself. The system is setup with reasonable defaults, including 404 logging and integration with the WSGI application.
Setting the Sentry DSN in production is optional but highly recommended. Having good error management for your CI app is really nice!