{"id":1028,"date":"2015-01-02T11:40:17","date_gmt":"2015-01-02T09:40:17","guid":{"rendered":"http:\/\/laxstrom.name\/blag\/?p=1028"},"modified":"2015-01-02T16:08:53","modified_gmt":"2015-01-02T14:08:53","slug":"oregano-deployment-tool","status":"publish","type":"post","link":"https:\/\/laxstrom.name\/blag\/2015\/01\/02\/oregano-deployment-tool\/","title":{"rendered":"Oregano deployment tool"},"content":{"rendered":"

This blog post introduces oregano<\/em>, a non-complex, non-distributed, non-realtime deployment tool. It currently consists of less than 100 lines of shell script and is licensed under the MIT license.<\/p>\n

The problem.<\/strong>\u00a0For a very long time, we have run translatewiki.net<\/a> straight from a git clone, or svn checkout before that. For years, we have been the one wiki which systematically run latest master, with few hours of delay. That was not a problem while we were young and wild. But nowadays, due to the fact that we carry dozens of local patches and thanks to the introduction of composer<\/a>, it is quite likely that git pull --rebase<\/code> will stop in a merge conflict. As a consequence, updates have become less frequent, but have semi-regularly brought the site down for many minutes until the\u00a0merge conflicts were manually resolved. This\u00a0had to change.<\/p>\n

The solution.<\/strong> I wrote a simple tool, probably re-inventing the wheel for the hundredth time, which separates the current deployment in two stages: preparation and pushing out new code. Since I have been learning a lot about Salt<\/a> and its quirks<\/a>, I named my tool “oregano”.<\/p>\n

How it works.<\/strong> Basically, oregano is a simple wrapper for symbolic links<\/a> and rsync. The idea is that you prepare your code in a directory named workdir<\/code>. To deploy the current state in workdir<\/code>, you must first create a read-only copy by running oregano tag<\/code>. After that, you can run oregano deploy<\/code>, which will update symbolic links so that your web server sees the new code. You can give the name of the tag with both commands, but by default oregano will name a new tag after the current timestamp, and deploy the most recently created tag. If, after deploying, you find out that the new tag is broken, you can quickly go back to the previously deployed code by running oregano rollback<\/code>. Below this is shown as a command line tutorial.<\/p>\n

mkdir \/srv\/mediawiki\/ # the path does not matter, pick whatever you want\r\n\r\ncd \/srv\/mediawiki\r\n\r\n# Get MediaWiki. Everything we want to deploy must be inside workdir\r\ngit clone https:\/\/github.com\/wikimedia\/mediawiki workdir\r\n\r\noregano tag\r\noregano deploy\r\n\r\n# Now we can use \/srv\/mediawiki\/targets\/deployment where we want to deploy\r\nln -s \/srv\/mediawiki\/targets\/deployment \/www\/example.com\/docroot\/mediawiki\r\n\r\n# To update and deploy a new version\r\ncd workdir\r\ngit pull\r\n# You can run maintenance scripts, change configuration etc. here\r\nnano LocalSettings.php\r\n\r\ncd .. # Must be in the directory where workdir is located\r\noregano tag\r\noregano deploy\r\n\r\n# Whoops, we accidentally introduced a syntax error in LocalSettings.php\r\noregano rollback\r\n<\/pre>\n
As you can see from above, it is still possible to break the site if you don’t check what you are deploying. For this purpose I might add support for hooks, so that one could run syntax checks whose failure would prevent deploying that code. Hooks would also be handy for sending IRC notifications, which is something our existing scripts do when code is updated: as pushing out code is now a separate step, they are currently incorrect.<\/p>\n
By default oregano will keep the 4 newest tags, so make sure you have enough disk space. For translatewiki.net, which has MediaWiki and dozens of extensions, each tag takes about 200M. If you store MediaWiki localisation cache, pre-generated for all languages, inside workdir<\/code>, then you would need 1.2G for each tag. Currently, at translatewiki.net, we store localisation cache outside workdir<\/code>, which means it is out of sync with the code. We will see if that causes any issues; we will move it inside workdir<\/code> if needed. Do note that oregano creates a tag with rsync --cvs-exclude<\/code> to save space. That also has the caveat that you should not name files or directories as core<\/code>. Be warned; patches welcome.<\/p>\n
The code is in the translatewiki repo<\/a> but, if there is interest, I can move it to a separate repository in GitHub. Oregano is currently used in translatewiki.net and in a pet project of mine nicknamed InTense<\/em>. If things go well, expect to hear more about this mysterious pet project in the future.<\/p>\n","protected":false},"excerpt":{"rendered":"
This blog post introduces oregano, a non-complex, non-distributed, non-realtime deployment tool. It currently consists of less than 100 lines of shell script and is licensed under the MIT license. The problem.\u00a0For a very long time, we have run translatewiki.net straight from a git clone, or svn checkout before that. For years, we have been the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,2,9,11],"tags":[116,114,115,80,117,124],"class_list":["post-1028","post","type-post","status-publish","format-standard","hentry","category-english","category-mediawiki","category-translatewiki-net","category-vapaasuomi","tag-deployment","tag-intense","tag-oregano","tag-pet-projects","tag-salt","tag-translatewiki-net"],"_links":{"self":[{"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/posts\/1028","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/comments?post=1028"}],"version-history":[{"count":6,"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/posts\/1028\/revisions"}],"predecessor-version":[{"id":1035,"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/posts\/1028\/revisions\/1035"}],"wp:attachment":[{"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/media?parent=1028"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/categories?post=1028"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/laxstrom.name\/blag\/wp-json\/wp\/v2\/tags?post=1028"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}