Automating deployment of a containerised Play 2.2.X App with Git + Docker

Introduction

The idea here is to harness the power of Git hooks and Docker to automatically deploy your Play app in a new Docker container upon a simple git push to your production server

 Requirements

  1. Git and SSH on your deployment server (easy setup instructions can be found here)
  2. Docker installed on your deployment server, as well as a a ‘deploy’ user that will have the Git repository and should be able to access docker (either through sudo or not. Also, I fancy my sudoers file with a bit more organization, so this is what my sudoers looks like [the relevant part at least]).
  3. A template Docker container with Play installed. Taig’s container is based on Play 2.2.3 out of the box, but his Dockerfile can serve as inspiration for your own version of Play (yep, you only have to change the version number)

Why

After playing a bit around with Dokku and googling for a dokku-like framework to do this deployment, I realised that fiddling with Dokku to do just what I needed was too cumbersome for the simple objective that I had in mind.

I did went through other frameworks, but everything seemed too complicated. If you actually have a suggestion on a system/framework to do this, please let me know (it can even be dokku, and actually be more simple than I expected [e.g. a custom buildpack that I don’t know about, whatever, let me know]).

The magic

So, all of the magic involved is done through the use of a Git hook. I got the inspiration of the usage of the lock on another post-commit hook that I found on Github (sorry, this was a bit ago and I didn’t took note of the link) and I also got the idea of making sure that the play compilation occurs before we try to build the new container from this great article with tips and lessons on automating deployment with docker (do read, some other lessons might apply in your case), otherwise we might end up with lots of faulty containers just because we had some fortuitous compilation error.

So, the Git hook follows:
As you can see, I’m compiling the code that I receive to the

workspace="/home/deploy/sandbox"

directory, replacing the production conf files that are found at

conf_files="/home/deploy/fb/conf"

with the ones that are on Git (so that you can easily leave your passwords/API keys outside of Git [yep, you should never do that :)]) The rest of the file is pretty much straightforward. I add the JAVA_OPTS when compiling the code with play stage because the arguments -JXmx=512 and -Xmx=512m do not work to lower the memory usage of the compilation itself, that actually does not need 1Gb of memory to run :) Also, I have set up on the root of the Git repository a Dockerfile that is called into action  with this command:

sudo docker build --tag="joantune/feedburner-prod:$last_commit" .

The Dockerfile that I use is based on Taig’s one, and as you can see, it is pretty basic:

Next steps

I will probably in the future change the build to accommodate a loadbalancer (like Nginx or other) for ‘hotdeploy’ (although with this solution we have only a few seconds of downtime) and more importantly for caching, or eventually even using a CDN for the static files (see this for a great article on how to optimize the serving of static files for play)  .

Anyway, any thoughts on these matters are appreciated. Hope you have found this useful.

Cheers!

GitPlay LogoDocker

Play 2.2.X + JPA + Hibernate – Bootstrapping the Database migration

Play Logo

Hibernate logo

Introduction

After reading about the issue of migrating databases on apps based on Play 2.2.X with JPA + Hibernate here: http://stackoverflow.com/questions/221379/hibernate-hbm2ddl-auto-update-in-production and here: http://stackoverflow.com/questions/6681445/play-hibernate-and-evolutions

I actually ended up using Flyway and it’s Play extension/plugin to do the DB migrations.

But inspired on other technologies that I used before, and on a comment I saw somewhere on StackOverflow (sorry, no link, I cannot remember where I saw it), I decided to make my life easier by using Hibernate’s automatic schema migration features to bootstrap my migration script.

After a bit of googling I knew that it was possible to do it programmatically or with an SBT/Maven/whatever plugin because of this: http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/toolsetguide.html#toolsetguide-s1-6

After briefly considering, and giving up on the idea of making a build ‘target’ (call it target, goal or whatever depending on how you build your app). I gave up on the idea because I knew very little about SBT, and I didn’t want to migrate the Play app to Maven, just for the sake of doing this.

So, inspired by this page I wrote the following Plugin instead:

As you can see, it writes the migration script to the “conf/db/migration” + <persistenceUnitName> + “.sql”  file, that can then be used as a base to write your own migration script.

After writing that plugin, I simply had to enable it by adding it to the play.plugins file, after the flyway one (to allow the migration to occur):

1000:com.github.tototoshi.play2.flyway.Plugin
1001:util.plugins.DBMigrationMR

I also commented the

<property name="hibernate.hbm2ddl.auto" value="validate" />

line at the persistence.xml file, as the ‘validation’ is now done with the plugin.

If you have any suggestions/comments please feel free to share

MSRP .NET port released!

Thanks to ContactMakers effort, another port of the MSRP library sees the light of day.

The original official release info:

Greetings fellow developers,

Since the 10th of January, that the MSRP C# (.NET) version is available!

Much thanks to ContactMakers for the effort applied on the port

For more details, visit the project’s webpage here

As always: happy coding!

The MSRP team.

MSRP new version release!

So, here it is, thanks to Tom, the new final version of the MSRP Peer Java library has been released!!

This is the initial announcement:

Greetings Java developer,

After a flying start within the brilliant Google Summer of Code, having it simmer for a while and finally polishing it up with some actual use-cases, the MSRP project team is proud to announce version 1.0 of its’ magnificent library!

This is an open source library, implementing the Message Session Relay Protocol (MSRP: RFC 4975), for you to deploy in any application.

Functionalities include

– establishing MSRP sessions

– sending and receiving instant messages (chat) using MSRP

– sending and receiving files using MSRP

– message/cpim wrapping to interface with other chat systems

– nicknames (draft-ietf-simple-chat)

– message composition indication (RFC 3994)

More information can be obtained at http://msrp.java.net

For a quick introduction, read the tutorial: http://msrp.java.net/tutorial.html

The entire project is hosted here: http://java.net/projects/msrp

Build versions can be found in the central Maven repository, just include the dependency in your projects’ pom.xml buildfile and you’re good to go.

So be our guest, use it, abuse it and send us your patches, comments, issues, wishlists and what have you.

Happy coding!

The MSRP team.

Special thanks go to:

– The Jitsi project for initiating this

– Google summer of code for making it possible

– Nlnet for early development sponsoring

– ContactMakers for sponsoring the polishing.

Feedback is welcomed :)

Jenkins – Tying SVN and Maven with some Groovy script and SVN Hooks – autodiscovering Maven modules on the SCM and automaticly creating Jenkins Jobs

Jenkins CI

(I took this post out from the ‘shelf’, it was lying here as a the draft for more than two years, anyway, in case it might still be useful, I publish it today :))

Introduction

So, due to the recent build structure change of the main project I work with (Bennu), we ended up with having to deal with ~61 Maven modules, and trying to minimize the changes to the workflow process of the developer. That meant that we wanted to trigger a deploy on Maven of the module’s JAR/WAR, when its code changed, with a simple SCM commit.

Architecture

So, we are still using SVN  for the SCM. (I’m lobbying for Git, and git-flow, which actually is great to tackle some issues we have with the maintenance tasks, and I have been actually profiting from the use of git-svn for quite some time now on my current workspace [yeah, we ended up using it :)])

We have two SCM repositories, a public and a private;

We used an old version of Hudson as a continuous integration (CI) tool (updated to latest version of Jenkins), to make sure the committed code was minimally ok;

We use Sonatype’s Nexus as a Maven repository [although we could have used jenkins itself, I later discovered];

The SCMs, Maven rep, and CI tool are all on the same server (although Jenkins offloads all the work to a more capable server);

We now have ~61 Maven modules to manage all of a sudden (we changed from the old, ant and black voodoo based, module system, to the popular and very handy maven);

Solution

So, we figured that the best would be to have Jenkins deploy to the Maven repository all of the commits that were made, being the committer responsible to update the module’s pom, and in case something went wrong with the compilation of that module, or with any of its dependencies, an email would have been sent warning everybody.

The challenge was to make this automatic i.e.: Having Jenkins sensing the Maven modules amongst other un-mavenized source code, and recognize changes to them on each commit, so that Jenkins is actually able to deploy to the Maven repository the existing modules, and also new ones.

For that job, the solution I came up with was a somehow elaborate SVN hook, that relies on  a couple of conventions, and that creates a new Jenkins job, in case a job doesn’t yet exist for that Maven module, and triggers a build of it upon changes in the SCM of that project’s code.

So, without delay, to the SVN Hook in question:

If you have any comments/feedback, let me know

Jenkins – allowing the ‘build trigger remotely’ functionality with the CAS plugin

Jenkins CI

Summary

So, right down to it, I gave the feature of allowing ‘build triggers remotely’ to the Jenkins(/Hudson) CAS plugin, thus allowing HTTP GET requests to bypass CAS authentication on ‘build triggers remotely’ links (i.e. http://JENKINS_URL/build?token=SMTHNG). It was one of my ways to give something back to this awesome tool. You can see the code here,  whose pull request has been made , alongside with its issue. Not so interesting reasons of why I did it, are detailed below, comments of code/reasons/whatever are welcome :)

Context

At work, we are starting to use hudson/jenkins to build and deploy each module of the new Bennu structure (~61 modules, all of them a Maven project and deployable on their own, a big change from what we had).

We already used a pre-forked version of Hudson, and seen that I had to make some changes to Hudson, I decided to upgrade. So, the question was Jenkins or Hudson? I Chose Jenkins, why?! no real good reason, I wanted to upgrade, I tried to google the choice, and found something in favor of Jenkins, plus the fact that the main community moved to Jenkins and the fact that I know that corporations can be tough on O.S., and I’m all in favor of open source, I chose Jenkins :).

Anywho, before I realized that Jenkins has SSH access (!!) I decided to tackle an old problem that I had solved before with the use of Apache, that was authentication on Jenkins through CAS. We had chosen Apache before to handle CAS authentication, instead of the Hudson CAS plugin, because it was the simplest way to allow CAS authentication while making sure that the ‘build trigger remotely’ urls (something like http://JENKINS_URL/build?token=SMTHNG) were accessible without CAS authentication for some hosts, so that my SVN hooks could have trigger the build when something had changed. We weren’t actually very happy with the Apache solution, as it sometimes the connection fails with a segfault on the Apache side (?! yeah, really!)!! not sure why, not planning on pulling out the GDB, after trying to search a bit for that bug, I just resigned to use it as it was, after all, we didn’t used the web interface all that much. Something that is bound to change with the new infrastructure used for Bennu.