Continuous Integration and Unit Testing are two critical processes in the development life cycle of every software project. Therefore, the question is not if it should be done, but how it should be done. Luckily there are plenty of options available for free (for open source projects). This post was written to help you decide which of the four Continuous Integration systems (CircleCI, TravisCI, Codeship and Jenkins) to use depending on your priorities, using a scaffold application and created with a generator. The application uses
gulp and connects to
mongod. However, this post will not cover the delivery process.
In this first post in my two-part series, I will cover CircleCI and TravisCI. In my next post, I will cover Codeship and Jenkins and give my final evaluation. This comparison shows how easy/hard it is to make each option work and what considerations you should have when you deploy a similar application.
The following aspects were evaluated on each system:
- Initial Setup: Complexity for creating the project and running basic tests
- Services: Complexity for connecting services (mongod in this case)
- Performance: Time it takes to run the operations
- UI/UX: Satisfaction in the opinion of the author
- Pricing (For evaluation purposes, I used free offerings only)
Each system is configured so that builds are triggered when changes are committed. Then, performance and UI/UX is evaluated. Ten iterations were considered to average each system’s performance without changing the code.
These operations are considered as standard to build a MEAN application:
- rm -rf node_modules
- npm install –no-optional
- npm test
- gulp build
Some systems require additional operations like installing underlying software, that time is considered as part of their performance evaluation.
We will begin here with CircleCI:
Sign up requires only an authorization of CircleCI with Github. You will then see a list of repositories, so you can create the project with one click. The system looks for a file called circle.yml to determine what steps conform the build. The documentation is very helpful, providing many examples for the most common languages and features. Setting the node version only requires a statement.
CircleCI provides many services out of the box and it’s bounded to localhost. The list includes:
Postgres, MySQL, MongoDB, Cassandra, redis, elasticsearch, rabbit-mq, etc, so for this application nothing is required.
The main page is informative, showing a list of the latest builds with the commit message, time ago and time it took.
This page is responsive, but, in my opinion, it doesn’t adapt well enough when the width is reduced. This is because the repository list is kept at the same width occupying empty space below the repo name.
Each command is captured in a panel that is collapsed once the command has completed. Scrolling to the bottom of the page will keep it autoscrolling.
CircleCI correctly supports ANSI codes for different colors.
The same responsive view is shown on mobile devices, which doesn’t use space appropriately.
UI/UX Rating: 8
- Free for 1 job, 1 container
- $150/m for 4 containers
- $350/m for 8 containers
The first thing to learn about TravisCI is that there are two sites: Travis-CI.com for enterprise clients and Travis-CI.org for open source projects. Immediately, I discovered a big delay, as I found out that Travis comes with a really old C++ compiler that doesn’t support C++11. This is important because
nodejs and many other languages compile libraries/gems using C++, so you will get strange errors that don’t seem to relate to your code. Finally, you must add a file called .travis.yml to your repo and Travis will start triggering builds when code is committed to Github. TravisCI does have good documentation and community base.
Services in Travis work similar to CircleCI, with the exception that in Travis these are not started by default. So, you need to specify which services you want load from this list:
MySQL, Postgres, MariaDB, MongoDB, CouchDB, RabbitMQ, memcached, Redis, Cassandra, Neo4j, ElasticSearch, RethinkDB, etc. For each of these Travis supports a number of versions.
The time shown here includes installing a new compiler as noted above.
The main page shows the latest build which is very useful. As in CircleCI, it shows time ago, commit message and commit id, but also a link to compare the last changes, which is also very useful too.
The same view adapts correctly when reduced in width:
It has autoscrolling and refreshes nicely.
Travis CI does support ANSI codes.
The responsive view adapts correctly on mobile devices.
UI/UX Rating: 10
- Free for open source projects
- $129/m for 2 concurrent builds
- $249/m for 5 concurrent builds
- $489/m for 10 concurrent builds
If you have any questions about continuous integration systems please leave a comment below. Part two, where I declare the winner, can be found here.