Continuous Integration Systems I: CircleCI, TravisCI, Codeship, Jenkins

Continuous Integration and Unit Testing are two critical processes in the development life cycle of every software project. Therefore, the question is not if it should be done, but how it should be done. Luckily there are plenty of options available for free (for open source projects). This post was written to help you decide which of the four Continuous Integration systems (CircleCI, TravisCI, Codeship and Jenkins) to use depending on your priorities, using a scaffold application and created with a generator. The application uses nodejs 4.4.6, npm, gulp and connects to mongod. However, this post will not cover the delivery process.

In this first post in my two-part series, I will cover CircleCI and TravisCI. In my next post, I will cover Codeship and Jenkins and give my final evaluation. This comparison shows how easy/hard it is to make each option work and what considerations you should have when you deploy a similar application.

Evaluation Method

The following aspects were evaluated on each system:

  1. Initial Setup: Complexity for creating the project and running basic tests
  2. Services: Complexity for connecting services (mongod in this case)
  3. Performance: Time it takes to run the operations
  4. UI/UX: Satisfaction in the opinion of the author
  5. Pricing (For evaluation purposes, I used free offerings only)

Each system is configured so that builds are triggered when changes are committed. Then, performance and UI/UX is evaluated. Ten iterations were considered to average each system’s performance without changing the code.

Test Program

These operations are considered as standard to build a MEAN application:

Preparation

  1. rm -rf node_modules
  2. npm install –no-optional

Unit Testing

  1. npm test

Build

  1. gulp build

Some systems require additional operations like installing underlying software, that time is considered as part of their performance evaluation.

We will begin here with CircleCI:

CircleCI

Initial Setup

Sign up requires only an authorization of CircleCI with Github. You will then see a list of repositories, so you can create the project with one click. The system looks for a file called circle.yml to determine what steps conform the build. The documentation is very helpful, providing many examples for the most common languages and features. Setting the node version only requires a statement.

Rating: 10

Services

CircleCI provides many services out of the box and it’s bounded to localhost. The list includes: Postgres, MySQL, MongoDB, Cassandra, redis, elasticsearch, rabbit-mq, etc, so for this application nothing is required.

Rating: 10

Performance

Main WindowUI/UX

The main page is informative, showing a list of the latest builds with the commit message, time ago and time it took.

This page is responsive, but, in my opinion, it doesn’t adapt well enough when the width is reduced. This is because the repository list is kept at the same width occupying empty space below the repo name.

Build Window

Each command is captured in a panel that is collapsed once the command has completed. Scrolling to the bottom of the page will keep it autoscrolling.

ANSI Support

CircleCI correctly supports ANSI codes for different colors.

Mobile

The same responsive view is shown on mobile devices, which doesn’t use space appropriately.

UI/UX Rating: 8

Pricing

  1. Free for 1 job, 1 container
  2. $150/m for 4 containers
  3. $350/m for 8 containers

Travis CI

Initial Setup

The first thing to learn about TravisCI is that there are two sites: Travis-CI.com for enterprise clients and Travis-CI.org for open source projects. Immediately, I discovered a big delay, as I found out that Travis comes with a really old C++ compiler that doesn’t support C++11. This is important because nodejs and many other languages compile libraries/gems using C++, so you will get strange errors that don’t seem to relate to your code. Finally, you must add a file called .travis.yml to your repo and Travis will start triggering builds when code is committed to Github. TravisCI does have good documentation and community base.

Rating: 7

Services

Services in Travis work similar to CircleCI, with the exception that in Travis these are not started by default. So, you need to specify which services you want load from this list:  MySQL, Postgres, MariaDB, MongoDB, CouchDB, RabbitMQ, memcached, Redis, Cassandra, Neo4j, ElasticSearch, RethinkDB, etc. For each of these Travis supports a number of versions.

Rating: 10

Performance

The time shown here includes installing a new compiler as noted above.


Main PageUI/UX

The main page shows the latest build which is very useful. As in CircleCI, it shows time ago, commit message and commit id, but also a link to compare the last changes, which is also very useful too.

The same view adapts correctly when reduced in width:

It has autoscrolling and refreshes nicely.

ANSI Support

Travis CI does support ANSI codes.

Mobile

The responsive view adapts correctly on mobile devices.

UI/UX Rating: 10

Pricing

  1. Free for open source projects
  2. $129/m for 2 concurrent builds
  3. $249/m for 5 concurrent builds
  4. $489/m for 10 concurrent builds

 —

If you have any questions about continuous integration systems please leave a comment below. Part two, where I declare the winner, can be found here.

 
Jose Hidalgo

When I was a kid I learned Logo (turtle) at school. We didn't have a computer at home. So, for me, the time I could spend "programming" was gold. That started the fire. Later, I got a computer with a modem and I discovered linux. I have been using the terminal ever since. Now it's called "DevOps," but for me, it's about learning powerful tools.

Related Articles

Ready to be Unstoppable?

Partner with Gorilla Logic, and you can be.