The Cloud Coach

CloudOps Educational Resources

Golden Images

Posted at — Jan 15, 2020

When automating the delivery of software it doesn’t take long before you realise you can speed things up, and make some things more secure, by pre-baking some machine images. After these images have been incorporated into the deployment process, automation becomes easier and more stable.

Packer

HashiCorp’s Packer is a perfect fit. Literally designed for making machine images such as AMIs, it supports a wide range of providers (AWS, Azure, etc), builders, provisioners, and more.

It’s the very definition of Images As Code (IAC.)

Packer uses a simple JSON configuration language to describe what image you want building and how to provision it. It’s also easy to install, being a single binary (written in Go.)

More information can be found at packer.io.

Development Workflow

Developing a new image is relatively straight forward but can get more complex depending on requirements.

A simple EBS backed AMI will look something like this:

{
	"variables": {
		"aws_region": "ap-southeast-2",
		"aws_source_ami": "ami-0328aad0f6218c429"
	},
	"builders": [
		{ 
			"type": "amazon-ebs",
			"region": "{{ user `aws_region`}}",
			"source_ami": "{{ user `aws_source_ami`}}",
			"instance_type": "t2.micro",
			"ssh_username": "ubuntu",
			"ami_name": "my-ami-{{timestamp}}",
			"associate_public_ip_address": true
		}
	],
	"provisioners": [
		{
			"type": "shell",
			"inline": [ "while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done" ]
		},
		{
			"type": "shell",
			"script": "provision.sh"
		}
	]
}

Notice how we reference a provision.sh file? That’s the shell script that is pushed and executed on the remote EC2 Instance (which Packer stands up for us) as whatever user you’re SSHing in as. In the above example we SSH in as ubuntu, which means the provision.sh shell script will have to use sudo whenever a privileged command is required.

Here’s an example provision.sh script that will update the package repository and the system’s packages (including the kernel), then install a few packages:

#!/bin/bash

sudo apt clean
sudo apt update -y
sudo apt upgrade -y
sudo apt install unzip python-pip -y

Again notice the use of sudo here.

After these files have been written the engineer can create a new AMI very easily:

packer build ami.json

Packer will now read the file and build a new AMI in AWS for us.

AMI Names

Inside of our ami.json file, above, we have the following code:

"ami_name": "my-ami-{{timestamp}}"

This determines the name of the AMI. The use of timestamp is a convenience provided by Packer and it will append a timestamp to the end of the AMI’s name. This prevents overlapping names that cause Packer’s procesess to fail.

Fetching the AMI

If you’re writing Terraform code and want to fetch this AMI, you might be wondering how you do this given the timestamp at the end of the name? How do you calculate that? Or do you hard code AMIs and update them when a new AMI is created?

In Terraform we can use a data{} block to fetch the latest AMI for us. Like this:

data "aws_ami" "agent" {
	most_recent = true
	owners      = ["self"]
  	filter {
		name   = "name"
		values = ["my-ami-*"]
	}
}

Here we’re telling Terraform to fetch the latest AMI based on a pattern that conforms to the ami_name we defined in our ami.json file.


Baking AMIs, or Golden Images, is a great way to speed up the deployment of autoscaling resources or simply manage the stability of any deployments by always building from a known-to-be-good, well maintained base.