Pre-tested “Commits” using Git
The ability to use pre-tested commits is a feature of certain Continuous Integration servers like Teamcity. The whole concept is not easily transfered to a Git infrastructure. There are several approaches. Teamcity 6.5 now newly supports an approach called “Personal builds on branches” (see here). In our project we needed the feature a long time ago and created a Git workflow that fulfills the intention of pre-tested commits and does not need any special features of the CI server.
But first of all what is the problem that is solved by testing a commit before it is committed? I think Geek and Poke gets it perfectly. You want to be able to get the latest checkin from VCS an be sure that it builds. Classic CI is in the wrong sequence. You commit first and let the CI server test afterwards. Of course you recognize a problem fast but by then your fellow coders will already get your mess. The responsible coder will run the test on his machine first but what do you have a CI server for?
The solution is to let the CI server test your changes before they are available to other developers. The classic approach used by Teamcity is pretty complex. They use an IDE Plugin or a so called Command Line Runner. It works like this:
- Before you commit you will start the remote run using one of the tools. The tool will create a patch of your changes and send them to Teamcity.
- Teamcity will now checkout a fresh version from the VCS and apply the patch to it.
- Now your CI run will compile and test everything including your changes.
- If successful the plugin will proceed to really check in your changes into Version Control.
Aside from it’s complexity this solution did not work well with distributed VCS like Git or Mercurial.
With Version 6.5 there is another option “Personal builds on branches”. You commit your changes into special branches that follow a certain name pattern. If Teamcity recognizes this commit it will start a personal build and push to the master branch if the build was successful. A very similar approach has been described for Jenkins.
We are using Git as our VCS since more than a year but the new solution by Teamcity has only been added very recently. This situation forced us to create our own solution and to our surprise this was much easier than expected. The main reason was the distributed nature of Git. Some of the points are:
- A commit is something that only effects your local repository as long as you don’t push it. This allows to test after a commit and before push. So you don’t need a special mechanism to create patches and send them somewhere.
- The central repository is not the only one. Beside your local copy you can create as many repositories as you like. So you can push wherever you like to in order to e.g. run tests.
- The central repository that you pull from doesn’t need to be the same that you push to.
- Commits have a globally unique identifier. You can try to push a commit twice. Git will recognize it and don’t apply it the second time.
You can probably already guess what I am up to, so lets show a picture of the overall setup.
There is one central Git-repository that only contains pre-tested changes. I call this “Green Repository” because it should only contain changes that lead to green builds. Every developer pulls from this repository but nobody is allowed to push to it. Instead everybody has a personal repository (think fork if you were on GitHub). The CI Server watches those personal repositories. After a commit it starts the compile and test. If that was successful it pushes the changes to the Green Repository. Now everybody can pull your changes and can be sure they have been tested.
You might wonder about some details of this setup so I try to answer some questions here:
- The remote server that contains the Green Repository and all personal remote repositories could be a simple git ssh setup. We are using a local installation of the Gitorious software instead. It is very easy to clone a central repository and to set access rights this way. There is also a commercial solution by the creators of GitHub called GitHub:FI.
- The remote-run on the CI Server needs to include 2 steps. Running the build (e.g. maven) and pushing the changes after successful build. We are using build dependencies in Teamcity in order to do these two steps. The first step is a simple maven goal and the second step is a simple command line call “git push”.
- In order to make the push to your personal remote repository easier you should create an alias. “git remote add remote-run [your remote repo url]” then it is easy to push your changes using “git push remote-run”.
- This solution works best with small teams that do not change often. The setup procedure for a new team member requires good knowledge of the solution. The setup itself can be done in less than 15 Minutes.
The question is whether we will continue using our own solution or switch to the new feature of Teamcity. So far I can’t see any advantage of the Teamcity feature. With our solution we are even more flexible in regard of branch design. So I think our layout will live on for a while.
If your CI-Server has no support for pre-tested commits you might give our setup a try. I am eager to hear your experiences. I am also happy to answer any specific questions about the setup in comments.