Puppet Faces are an extendable API for tricking out your Puppet instances. (“Faces” is just short for “Interfaces”.) Just a couple days ago I wrote about my survey of puppet + ec2 provisioning tools.

The problem I’m trying to solve, which I don’t feel like I’ve solved well, is how to give a type to a new system at bootstrap time, without using DNS. The type variable maps to a node manifest group, and determines the personality of a host – is it a database, webserver or development instance?

What I’d like to do is pass a type to puppet at install time and have the puppetmaster and the agent remember that mapping between host and type.

I did it with a really simple Facter plugin, install scripts named by type (passed in to puppet node install), and a file created by the install script in /etc/puppet.

Then, I wanted to be able to see which hosts were configured with which install type. Facter was aware of the type, so this seemed like it should be pretty easy…

I wrote a quick and dirty Face that pulls information out of $varlib/nodes/*.yaml on the puppet master. I imagine there are better ways to do this, but in the absence of documentation or someone to tell me not to do this, I forged ahead!

There were two things that I spent quite a bit of time chewing on before figuring it out:

1. If you want to make an :action in your Face the default, you just add default in the body of your :action block. I had to dig through a few cloudpack files before I found it!
2. If you are creating and terminating hosts frequently, you may end up with a bunch of certs and other annoying metadata laying around. To clean it up, the Puppet Node Face has a command you can run:
   
# puppet node clean [hostname]


   You’ll probably need to be the user that’s running puppet for this to work — it affects things that the puppetmaster owns in $varlib.

   If you’re doing this with code, it’s:
   
Puppet::Face[:node, :current].clean('hostname')


I put a little patch into a recent version of cloudprovisioner that invokes clean during a terminate. It’s quick and dirty, and only for AWS.

The resources I’ve found useful are:

• Kelsey’s talk at PuppetConf about Faces
• Kelsey Hightower’s Git repo for Puppet Dashboard Face
• Kartar’s Github face

And to a lesser extent, these blog posts were helpful for filling in a few gaps:

• What the heck are Faces?
• Creating a simple “backup” Face

I thought this would be easy.

I started using Vagrant, and was productive with it in about a day. Really a couple hours. Most of my time was spent downloading the correct version of VirtualBox, looking for starter images and then a small amount of time experimenting with the Vagrantfile scripting language (for multiple VMs).

And we made some Puppet configs.

Then I wanted to use those same Puppet configs with EC2.

So my ultimate goals were:

1. Reuse my existing puppet configs as much as possible
2. Have a completely automated deploy of a server system (including checkouts of code from a private github repo)
3. Have a puppetmaster in EC2
4. Be able to provision systems from EC2 or my laptop
5. Make the whole process easy for my coworkers

This is mostly a list of what I failed at using, and the thing I succeed with at the end.

Short aside:

Pro tip to people writing documentation: Most tutorials and sites that make recommendations for tools leave out the part where you run into all kinds of insane problems. Create a wiki page or a place where you collect the problems. Please.

For example: My Cloud Formation to Ubuntu AMI deploy was failing with an error in cfn.rb that said: “Unexpected return.” Um. Ok. *facepalm*

The problem was that a AWS-image specific JSON file wasn’t present (and couldn’t be created) on the target machine. So instead of noting (raise an exception, anyone?) that the file wasn’t present, the module just executed a bare return.

Because I don’t know much about Puppet internals, this was a very annoying problem to solve. (like, what gets installed in /var/lib/puppet/lib vs. in the gem install vs. the cloudpack library I was told to install in /etc/puppet/modules?)

Stepping back a bit – a useful note from the Cloud Formation folks would have been: “Hey – this probably won’t work if you try to deploy to non-Amazon Linux AMI distros of Linux.” It’s not obvious that’s the case! You’re supposed to be able to completely control the classes being installed on the target system, right? Bad assumption, apparently.

And we’re back!

Let me know in the comments if you’ve successfully navigated any of the tools I didn’t pick. Juju, in particular, I don’t think I gave a fair chance (since I didn’t try it at all).

Here’s my list:

1. Juju

   I just wasn’t sure this was a reasonable thing to install/use. No one I knew had ever heard of it. Didn’t try it.

2. Mcollective + tools ported to PHP

   I’m interested in Mcollective, but the configs looked overly complex, and I didn’t have anyone close by that was actively using it.

   The examples scared me away because of the PHP. I already had three languages at play in the deployment, and I didn’t need another language dependency. So, I didn’t bother trying it.

3. Custom scripts based on the ec2-tools packages

   This approach works, but is fragile and a PITA to keep updated. I tried it as a “getting oriented” exercise, and abandoned it.

4. Mccloud

   This looked awesome! I could reuse all my Vagrant configs and not really have to change anything… Except I had to maintain duplicate configs, just sub ‘Mccloud’. Eh.

   I may revisit this tool in the future, but it seemed to require pretty much the same things as the tool I ultimately decided to use, and didn’t seem as flexible. I also had a weird restriction where it wouldn’t allow me to spin up the correct type of image (I wanted m1.small in my testing). Could have been PEBKAC — I didn’t take good enough notes to say for sure.

5. cloud-init

   This looked very promising! We were already using Ubuntu so seemed like a good fit.

   Pros: easy – pass in a shell script when starting an EC2 instance from the web. Cons: required yet-another-configuration style. But there were command-line tools and it was looking very promising.

   In the end, using a supported package would have required me to be running a Linux desktop to start my puppetmaster. I didn’t search much harder than brew install cloud-init for a Mac-equivalent (that doesn’t exist). So, I moved on to the next thing.

6. AWS Cloud Formation

   I launched a puppetmaster pre-configured instance! I sort of got puppetmaster running! Then I tried to deploy an Ubuntu AMI from it… This does not work.

   So, I will save you a ton of time: Avoid trying to mix the pre-specified Cloud Formation images with other systems.

   Someone showed me the chunk of the config you can rip out and probably get it to work. I was frustrated at that point, and moved on. Too much tweaking was required, for what was uncertain gain at that point.

7. PuppetLab’s Cloud Provisioner

   This is what I am currently using! I’m running HEAD pulled directly from github. Older versions are not recommended. (I tried three versions.)

   The configuration is pretty straightforward and documented. The one thing (a very important thing) is that you have to amend your $RUBYLIB if you don’t install the code in your version of ruby’s default libdir. There’s no gem. Yet.

   I customized the deploy script to my liking – there is an unsupported option called --install-script you can pass in that will execute whatever .erb (a shell script!) you’d like if you put it in ~/.puppet/scripts. You can also pass in your puppetmaster hostname with --server.

   Totally sweet.

   The command-line is ok, but there’s also a programmatic interface in Ruby. Dan Bode showed me a short code snippet that worked (hostnames & keys sanitized):

   
irb(main):012:0> require 'puppet'
irb(main):013:0> require 'puppet/face'
irb(main):014:0> Puppet::Face[:node, :current].install('myserver.compute-1.amazonaws.com', :keyfile => 'mykey.pem', :login => 'ubuntu', :install_script => 'custom-puppetmaster', :server=>'myserver.compute-1.amazonaws.com')


   I so appreciate this! Faces is awesome.

I’ve got some additional tweaking to do yet, but I’m planning to commit a few amendments to the provisioner scripts included by default and the README. And I filed a couple bugs.

Overall, I’d bet that cloud-provisioner (if you use the version currently on github) will work for most people.

Zombie puppet teaser…

Just a quick note that I’m helping start Puppet PDX and I posted some details over here. Configuration management is awesome stuff, and if you live in the Portland area, you should come check out the new group. We’re meeting at Paddy’s, 6pm on 2/27/09.