Introducing Packer: building Vagrant base boxes hands-free

I have referred to Packer in some of my cloud-related presentations as an example of a tool for creating immutable infrastructure. In addition to the cloud, Packer supports a great many other build targets as well. Since I work with VirtualBox and Vagrant a lot, Packer’s ability to create Vagrant base boxes is super awesome. Combined with local box versioning I can build new Vagrant systems in almost no time. More importantly though, I can simply kick the process off, grab a coffee, and when I’m back, enjoy a new build of my Oracle Linux Vagrant base box.

The task at hand

I would like to build a new Vagrant base box for Oracle Linux 7.8, completely hands-off. All my processes and workflows therefore need to be defined in software (Infrastructure as Code).

Since I’m building private Vagrant boxes I don’t intend to share I can ignore the requirements about passwords as documented in the Vagrant documentation, section “Default User Settings”. Instead of the insecure key pair I’m using my own keys as well.

The build environment

I’m using Ubuntu 20.04.1 LTS as my host operating system. Packer 1.5 does all the hard work provisioning VMs for Virtualbox 6.1.12. Ansible 2.9 helps me configure my systems. Vagrant 2.2.7 will power my VMs after they are created.

Except for VirtualBox and Packer I’m using the stock packages supplied by Ubuntu.

How to get there

Packer works by reading a template file and performs the tasks defined therein. If you are new to Packer, I suggest you visit the website for more information and some really great guides.

As of Packer 1.5 you can also use HCL2. Which is nice, as it allows me to reuse (or rather add to) my Terraform skills. However, at the time of writing the documentation warns that HCL2 support is still in beta, which is why I went with the JSON template language.

High-level steps

From a bird’s eye view, using my code Packer template…

  • Creates a Virtualbox VM from an ISO image
  • Feeds a kickstart file to it for an unattended installation
  • After the VM is up it applies an Ansible playbook to it to install VirtualBox guest additions

The end result should be a fully working Vagrant base box.

Provisioning the VM

The first thing to do is to create the Packer template for my VM image. Properties of a VM are defined in the so-called builders section. As I said before, there are lots of builders available… I would like to create a Virtualbox VM from an ISO image, so I went with virtualbox-iso, which is really super easy to use and well documented. So after a little bit of trial and error I ended up with this:

{
  "builders": [
    {
      "type": "virtualbox-iso",
      "boot_command": [
        "<esc>",
        "<wait>",
        "linux text inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ol7.ks",
        "<enter>"
      ],
      "disk_size": "12288",
      "guest_additions_path": "/home/vagrant/VBoxGuestAdditions.iso",
      "guest_os_type": "Oracle_64",
      "hard_drive_interface": "sata",
      "hard_drive_nonrotational": "true",
      "http_directory": "http",
      "iso_checksum": "1c1471c49025ffc1105d0aa975f7c8e3",
      "iso_checksum_type": "md5",
      "iso_url": "file:///m/stage/V995537-01-ol78.iso",
      "sata_port_count": "5",
      "shutdown_command": "echo 'packer' | sudo -S shutdown -P now",
      "ssh_timeout": "600s",
      "ssh_username": "vagrant",
      "ssh_agent_auth": true,
      "vboxmanage": [
        [
          "modifyvm",
          "{{.Name}}",
          "--memory",
          "2048"
        ],
        [
          "modifyvm",
          "{{.Name}}",
          "--cpus",
          "2"
        ]
      ],
      "vm_name": "packertest"
    }
  ],
  "provisioners": [
    {
      "type": "ansible",
      "playbook_file": "ansible/guest_additions.yml",
      "user": "vagrant"
    }
  ],
  "post-processors": [
    {
      "keep_input_artifact": true,
      "output": "/home/martin/vagrant/boxes/ol7_7.8.2.box",
      "type": "vagrant"
    }
  ]
} 

If you are new to Packer this probably looks quite odd, but it’s actually very intuitive after a little while. I haven’t used Packer much before coming up with this example, which is a great testimony to the quality of the documentation.

Note this template has 3 main sections:

  • builders: the virtualbox-iso builder allows me to create a VirtualBox VM based on an (Oracle Linux 7.8) ISO image
  • provisioners: once the VM has been created to my specification I can run Ansible playbooks against it
  • post-processors: this line is important as it creates the Vagrant base box after the provisioner finished its work

Contrary to most examples I found I’m using SSH keys for communicating with the VM, rather than a more insecure username/password combination. All you need to do is add the SSH key to the agent via ssh-add before you kick the build off.

While testing the best approach to building the VM and guest additions I ran into a few issues prompting me to upload the guest additions ISO to the vagrant user’s home directory. This way it wasn’t too hard to refer to it in the Ansible playbook (see below).

Kickstarting Oracle Linux 7

The http_directory directive in the first (builder) block is crucial for automating the build. As soon as Packer starts its work, it will create a HTTP server in the directory indicated by the variable. This directory must obviously exist.

Red-Hat-based distributions allow admins to install the operating system in a fully automated way using the Kickstart format. You provide the Kickstart file to the system when you boot it for the first time. A common way to do so is via HTTP, which is why I’m so pleased about the HTTP server started by Packer. It couldn’t be easier: thanks to my http_directory a web server is already started, and using the HTTPIP and HTTPort variables I can refer to files inside the directory from within the template.

As soon as Packer boots the VM the Kickstart file is passed as specified in boot_command. I had to look the syntax up using a search engine. It essentially comes down to simulating a bunch of keystrokes as if you were typing them interactively.

Long story short, I don’t need to worry about the installation, at least as long as my Kickstart file is ok. One way to get the Kickstart file right is to use the one that’s created after a manual operating system installation. I usually end up using /root/anaconda-ks.cfg and customise it.

There are 3 essentials tasks to complete in the Kickstart file if you want to create a Vagrant base box:

  1. Create the vagrant user account
  2. Allow password-less authentication to the vagrant account via SSH
  3. Add vagrant to the list of sudoers

First I have to include a directive to create a vagrant user:

user --name=vagrant

The sshkey keyword allows me to inject my SSH key into the user’s authorized_keys file.

sshkey --username=vagrant "very-long-ssh-key"

I also have to add the vagrant account to the list of sudoers. Using the %post directive, I inject the necessary line into /etc/sudoers:

%post --log=/root/ks-post.log

/bin/echo "vagrant        ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers

%end 

Calling the Ansible provisioner

So far I have defined a VM to be created (within the builders section). The installation is completely hands-off thanks for the Kickstart file I provided. However, I’m not done yet: I have yet to install the Virtualbox guest additions. This is done via the Ansible provisioner. It connects as vagrant to the VM and executes the instructions from ansible/guest_additions.yml.

This is a rather simple file:

- hosts: all
  become: yes
  tasks:
  - name: upgrade all packages
    yum:
      name: '*'
      state: latest

  - name: install kernel-uek-devel
    yum:
      name: kernel-uek-devel
      state: present

  - name: reboot to the latest kernel
    reboot:

  # Guest additions are located as per guest_additions_path in 
  # Packer's configuration file
  - name: Mount guest additions ISO read-only
    mount:
      path: /mnt/
      src: /home/vagrant/VBoxGuestAdditions.iso
      fstype: iso9660
      opts: ro
      state: mounted

  - name: execute guest additions
    shell: /mnt/VBoxLinuxAdditions.run 

In plain English, I’m becoming root before updating all software packages. One of the pre-rquisites for compiling Virtualbox’s guest additions is to install the kernel-uek-devel package.

After that operation completed theVM reboot into the new kernel before mounting the guest addition ISO I asked to be copied to /home/vagrant/VBoxGuestAdditions.iso in the builder section of the template.

Once the ISO file is mounted, I call VBoxLinuxAdditions.run to build the guest additions.

Building the Vagrant base box

Putting it all together, this is the output created by Packer:

$ ANSIBLE_STDOUT_CALLBACK=debug ./packer build oracle-linux-7.8.json 
virtualbox-iso: output will be in this color.

==> virtualbox-iso: Retrieving Guest additions
==> virtualbox-iso: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: /usr/share/virtualbox/VBoxGuestAdditions.iso => /usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: Retrieving ISO
==> virtualbox-iso: Trying file:///m/stage/V995537-01-ol78.iso
==> virtualbox-iso: Trying file:///m/stage/V995537-01-ol78.iso?checksum=md5%3A1c1471c49025ffc1105d0aa975f7c8e3
==> virtualbox-iso: file:///m/stage/V995537-01-ol78.iso?checksum=md5%3A1c1471c49025ffc1105d0aa975f7c8e3 => /m/stage/V995537-01-ol78.iso
==> virtualbox-iso: Starting HTTP server on port 8232
==> virtualbox-iso: Using local SSH Agent to authenticate connections for the communicator...
==> virtualbox-iso: Creating virtual machine...
==> virtualbox-iso: Creating hard drive...
==> virtualbox-iso: Creating forwarded port mapping for communicator (SSH, WinRM, etc) (host port 2641)
==> virtualbox-iso: Executing custom VBoxManage commands...
    virtualbox-iso: Executing: modifyvm packertest --memory 2048
    virtualbox-iso: Executing: modifyvm packertest --cpus 2
==> virtualbox-iso: Starting the virtual machine...
==> virtualbox-iso: Waiting 10s for boot...
==> virtualbox-iso: Typing the boot command...
==> virtualbox-iso: Using ssh communicator to connect: 127.0.0.1
==> virtualbox-iso: Waiting for SSH to become available...
==> virtualbox-iso: Connected to SSH!
==> virtualbox-iso: Uploading VirtualBox version info (6.1.12)
==> virtualbox-iso: Uploading VirtualBox guest additions ISO...
==> virtualbox-iso: Provisioning with Ansible...
    virtualbox-iso: Setting up proxy adapter for Ansible....
==> virtualbox-iso: Executing Ansible: ansible-playbook -e ...
    virtualbox-iso:
    virtualbox-iso: PLAY [all] *********************************************************************
    virtualbox-iso:
    virtualbox-iso: TASK [Gathering Facts] *********************************************************
    virtualbox-iso: ok: [default]
    virtualbox-iso: [WARNING]: Platform linux on host default is using the discovered Python
    virtualbox-iso: interpreter at /usr/bin/python, but future installation of another Python
    virtualbox-iso: interpreter could change this. See https://docs.ansible.com/ansible/2.9/referen
    virtualbox-iso: ce_appendices/interpreter_discovery.html for more information.
    virtualbox-iso:
    virtualbox-iso: TASK [upgrade all packages] ****************************************************
    virtualbox-iso: changed: [default]
    virtualbox-iso:
    virtualbox-iso: TASK [install kernel-uek-devel] ************************************************
    virtualbox-iso: changed: [default]
    virtualbox-iso:
    virtualbox-iso: TASK [reboot to enable latest kernel] ******************************************
==> virtualbox-iso: EOF
    virtualbox-iso: changed: [default]
    virtualbox-iso:
    virtualbox-iso: TASK [Mount guest additions ISO read-only] *************************************
    virtualbox-iso: changed: [default]
    virtualbox-iso:
    virtualbox-iso: TASK [execute guest additions] *************************************************
    virtualbox-iso: changed: [default]
    virtualbox-iso:
    virtualbox-iso: PLAY RECAP *********************************************************************
    virtualbox-iso: default                    : ok=6    changed=5    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
    virtualbox-iso:
==> virtualbox-iso: Gracefully halting virtual machine...
==> virtualbox-iso: Preparing to export machine...
    virtualbox-iso: Deleting forwarded port mapping for the communicator (SSH, WinRM, etc) (host port 2641)
==> virtualbox-iso: Exporting virtual machine...
    virtualbox-iso: Executing: export packertest --output output-virtualbox-iso/packertest.ovf
==> virtualbox-iso: Deregistering and deleting VM...
==> virtualbox-iso: Running post-processor: vagrant
==> virtualbox-iso (vagrant): Creating a dummy Vagrant box to ensure the host system can create one correctly
==> virtualbox-iso (vagrant): Creating Vagrant box for 'virtualbox' provider
    virtualbox-iso (vagrant): Copying from artifact: output-virtualbox-iso/packertest-disk001.vmdk
    virtualbox-iso (vagrant): Copying from artifact: output-virtualbox-iso/packertest.ovf
    virtualbox-iso (vagrant): Renaming the OVF to box.ovf...
    virtualbox-iso (vagrant): Compressing: Vagrantfile
    virtualbox-iso (vagrant): Compressing: box.ovf
    virtualbox-iso (vagrant): Compressing: metadata.json
    virtualbox-iso (vagrant): Compressing: packertest-disk001.vmdk
Build 'virtualbox-iso' finished.

==> Builds finished. The artifacts of successful builds are:
--> virtualbox-iso: VM files in directory: output-virtualbox-iso
--> virtualbox-iso: 'virtualbox' provider box: /home/martin/vagrant/boxes/ol7_7.8.2.box

This concludes the build of the base box.

Using the newly created base box

I’m not quite done yet though: as you may recall I’m using (local) box versioning. A quick change of the metadata file ~/vagrant/boxes/ol7.json and a call to vagrant init later, I can use the box:

$ vagrant box outdated
Checking if box 'ol7' version '7.8.1' is up to date...
A newer version of the box 'ol7' for provider 'virtualbox' is
available! You currently have version '7.8.1'. The latest is version
'7.8.2'. Run `vagrant box update` to update. 

That looks pretty straight-forward, so let’s do it:

$ vagrant box update
==> server: Checking for updates to 'ol7'
    server: Latest installed version: 7.8.1
    server: Version constraints: 
    server: Provider: virtualbox
==> server: Updating 'ol7' with provider 'virtualbox' from version
==> server: '7.8.1' to '7.8.2'...
==> server: Loading metadata for box 'file:///home/martin/vagrant/boxes/ol7.json'
==> server: Adding box 'ol7' (v7.8.2) for provider: virtualbox
    server: Unpacking necessary files from: file:///home/martin/vagrant/boxes/ol7_7.8.2.box
    server: Calculating and comparing box checksum...
==> server: Successfully added box 'ol7' (v7.8.2) for 'virtualbox'! 

Let’s start the environment:

$ vagrant up
Bringing machine 'server' up with 'virtualbox' provider...
==> server: Importing base box 'ol7'...
==> server: Matching MAC address for NAT networking...
==> server: Checking if box 'ol7' version '7.8.2' is up to date...
==> server: Setting the name of the VM: packertest_server_1598013258821_67878
==> server: Clearing any previously set network interfaces...
==> server: Preparing network interfaces based on configuration...
    server: Adapter 1: nat
==> server: Forwarding ports...
    server: 22 (guest) => 2222 (host) (adapter 1)
==> server: Running 'pre-boot' VM customizations...
==> server: Booting VM...
==> server: Waiting for machine to boot. This may take a few minutes...
    server: SSH address: 127.0.0.1:2222
    server: SSH username: vagrant
    server: SSH auth method: private key
==> server: Machine booted and ready!
==> server: Checking for guest additions in VM...
==> server: Setting hostname...
==> server: Mounting shared folders...
    server: /vagrant => /home/martin/vagrant/packertest 

Happy Automating!