Category → productivity
A big thanks to Atlassian for allowing me to post this series!!
There is NO reason, not to use a version control system while developing puppet manifest/modules. Stating that should be an open door. It allows you to go back in time, share things more easily and track your changes. There is a lot of information out there on how to work with git or any other system. But here a few tips that might help you developing modules:
Tip 1 : give each module it's own repo and use a superproject to join
In a lot of blogposts and even in the excellent Pro Puppet book I see people checking in their entire environment directory into version control.
I'm all for version control but if you manage your modules dir as one flat repository, you loose the way to easily update and share modules from the forge. In essence you are doing a copy that starts living it's own life.
The idea goes like this:
/etc/puppet/environments/development/modules (super-project repo) -- puppet-apache (sub-project repo) -- puppet-mysql (sub-project repo) ...
The super-project repo will contain links to the submodules it uses. This allows the reuse of the sub-project repos in different super-projects. F.i. puppet-modules-team1, puppet-modules-team2 could be superprojects and use different sub-modules.
Git has the concept of submodules that allows you to link a parent repository with subprojects. Further detailed documentation can be found at http://book.git-scm.com/5_submodules.html
Using this approach with puppet is nicely described at https://we.riseup.net/riseup+tech/puppet-git-submodules, with some tips at https://labs.riseup.net/code/documents/7
I was always scared away of using submodules, because things like checking in the superproject first and forgetting the submodule, makes part of it unusable. Let alone adding the submodule directoy with a 'slash' has bitten me a few times.
It's a good approach but it requires being awake :)
To create a development repo with a specific apache submodule
$ cd /etc/puppet/environments/ $ hg init development $ cd development/ $ mkdir modules $ echo 'modules/apache = [git]git://github.com/puppet-modules/puppet-apache.git' > .hgsub $ hg add .hgsub $ git clone git://github.com/puppet-modules/puppet-apache.git modules/apache $ hg ci -mapache
Now if we are checking this out to our test environment, we'll see that it automatically checks out the submodules.
$ cd /etc/puppet/environments $ hg clone development/ test updating to branch default resolving manifests getting .hgsub getting .hgsubstate cloning subrepo modules/apache from git://github.com/puppet-modules/puppet-apache.git remote: Counting objects: 177, done. remote: Compressing objects: 100% (94/94), done. remote: Total 177 (delta 59), reused 168 (delta 52) Receiving objects: 100% (177/177), 22.97 KiB, done. Resolving deltas: 100% (59/59), done. 2 files updated, 0 files merged, 0 files removed, 0 files unresolved
I found this workflow:
- less scary: no way to wrongly add directories
- it's handy that you can add both svn, git and hg subrepositories
- submodules are checked out by default (no git submodules init, update)
- by default hg wants to commit to all subrepos when commiting to the super-project repo. This Recursive commits can be disabled with the local ui.commitsubrepos configuration setting introduced in Mercurial 1.8.
- More information on Mercurial subrepositories can be found at: http://mercurial.selenic.com/wiki/Subrepository?action=show&redirect=subrepos#Synchronizing_in_subrepositories
- Or for a complete run through : http://www.accidentalhacker.com/using-mercurial-subrepositories/
Tip 2: Think about your git workflow
In many of the examples you'll see people just commit to the 'master' branch. This works of course but if you are working on different modules/features, it's best to think about your git workflow. There are a lot of blogpost describing how to work in feature branches. Still I found it hard to remember the commands for branching, deleting old branches and so on.
It's basicly git helpers that take away a lot of the pain for working on hotfixes, features, releases. The usage is pretty easy:
# Create the master/developer/release/hotfix branches $ git flow init # Start working on a feature (branched from develop) $ git flow feature start feature1 ... do some work $ git add ...somework... $ git commit -m "somework feature1" # This will merge feature1 back to develop $ git flow feature finish feature1 # Now lets start a release $ git flow release start release1 ... do some work $ git add ...somework... $ git commit -m "release1" # This will merge release1 into master $ git flow release finish release1
The whole idea is described in details at http://nvie.com/posts/a-successful-git-branching-model/ . And the following video will show you how it works.
More information can also be found at:
Tip 3: Use pre/post-commit hooks
Even with the awesome editor support we previously described, it's still easy to miss a semi-column, or have an incorrect puppet syntax.
It's good practice to verify the syntax of your puppet manifests before comitting them to version control. This is well described on the puppetlabs version control page.
In essence before you commit, it will execute
puppet --parseonlyto check if the syntax is correct.
A lesser used technique is to run the same check on post-commit:
Instead of running your master repository out of /etc/puppet/environment [PROD], you checkin to an intermediate repository [CHECK].
developer -> LOCAL REPO (pre-commit) -> push -> CHECK REPO -> (post-commit) -> PROD REPO
In the post commit you can:
- also verify the syntax in case someone didn't check before commiting.
- if all successfull push the repo to the PROD directories
This helps overcome the problem that your puppetmaster does a run while the repo is an incorrect state.
More details are described at Git workflow and Puppet Environments
More to come
The next post will be about different ways to test your puppet manifests
I've spent some time recently on setting up my environment to work more productively on writing puppet manifests. This blogpost highlights some of the findings to get me more productive on editing puppet files and modules. Some older information can be found at Editor Tips on the puppetlabs website.
Tip 1: Syntax highlighting,snippet completion
Puppet syntax is very specific, it's important to get clues about missing curly braces, semi-colums, etc .. as fast as possible. There is support for this in the most common editors:
@Masterzen has created a textmate bundle for use with puppet. You can find it at https://github.com/masterzen/puppet-textmate-bundle.
Michael Halligan describes how to install it from the commandline
mkdir -p /Library/Application\ Support/TextMate/Bundles cd /Library/Application\ Support/TextMate/Bundles git clone git://gitorious.org/git-tmbundle/mainline.git Git.tmbundle git clone http://git.gitorious.org/git-tmbundle/mainline.git Git.tmbundle git clone https://github.com/masterzen/puppet-textmate-bundle.git Puppet.tmbundle git clone https://github.com/drnic/Chef.tmbundle.git Chef.tmbundle osascript -e 'tell app "TextMate" to reload bundles'
If textmate is not your thing, here is how you can pimp up your vim:
When you look around for puppet/vim integration there seem to have been some re-incarnations:
- The first option is just setting the syntax of any .pp file to ruby syntax
- The second option as Garett Honeycutt describes:
- is a more elaborate version of highlighting pp files (orginally written by Luke Kanies).
- this file is distributed with puppet itself.
- Stick gaves us even more advanced tips at :
- R.I. Pienaar showed us how to use Snipmate with vim and puppet :
- His snippets can be found at http://www.devco.net/code/puppet.snippets
To use the vim-puppet plugin, you're best to use pathogen written by Tim Pope. I've followed the instructions at http://tammersaleh.com/posts/the-modern-vim-config-with-pathogen.
I've enabled the following plugins in my update_bundles script
Most notable plugins:
- Tabular gives you automatic => alignment
- Syntastic gives you syntax feedback while you edit files
- Snipmate gives you the snippets on tab expansion
- Specky gives you functionality for rspec files
- vim-ruby gives you extra functionality for ruby files
- vim-cucumber gives you functionality for cucumber files
For more information on the vim-puppet project go to:
The snippets that are expanded in the vim-puppet plugin can be found at:
Tip 2: don't create modules structure by hand
I keep forgetting the correct structure, files etc.. when I create a new module. Luckily there is an easy way to generate a puppet module structure using the puppet-module gem
$ gem install puppet-module $ puppet-module Tasks: puppet-module build [PATH_TO_MODULE] # Build a module for release puppet-module changelog # Display the changelog for this tool puppet-module changes [PATH_TO_MODULE] # Show modified files in an installed module puppet-module clean # Clears module cache for all repositories puppet-module generate USERNAME-MODNAME # Generate boilerplate for a new module puppet-module help [TASK] # Describe available tasks or one specific task puppet-module install MODULE_NAME_OR_FILE [OPTIONS] # Install a module (eg, 'user-modname') from a repositor... puppet-module repository # Show currently configured repository puppet-module search TERM # Search the module repository for a module matching TERM puppet-module usage # Display detailed usage documentation for this tool puppet-module version # Show the version information for this tool Options: -c, [--config=CONFIG] # Configuration file $ puppet-module generate puppetmodule-apache ========================================================================================= Generating module at /Users/patrick/demo-puppet/modules/puppetmodule-apache ----------------------------------------------------------------------------------------- puppetmodule-apache puppetmodule-apache/tests puppetmodule-apache/tests/init.pp puppetmodule-apache/spec puppetmodule-apache/spec/spec_helper.rb puppetmodule-apache/spec/spec.opts puppetmodule-apache/README puppetmodule-apache/Modulefile puppetmodule-apache/metadata.json puppetmodule-apache/manifests puppetmodule-apache/manifests/init.pp
Tip 3 - Geppetto: a Puppet IDE
Note: this NOT related with the Gepetto (one P) project by Alban Peignier
James Turnbull was so kind to make a quick screencast on how it works:
But remember it's Java based, so it might take a while to fire it up :)