The longer I work in IT the more I dislike repetitive processes. For example, when updating my Oracle Linux 8 Vagrant Base Box I repeat the same process over and over:
- Boot the VirtualBox (source) VM
- Enable port forwarding for SSH
- SSH to the VM to initiate the update via
dnf update -y && reboot - Run
vagrant package, calculate the SHA256 sum, modify the metadata file - Use
vagrant box updateto make it known tovagrant
There has to be a better way to do that, and in fact there is. A little bit of shell scripting later all I need to do is run my “update base box” script, and grab a coffee while it’s all done behind the scenes. The most part of the exercise laid out above is quite boring, but I thought I’d share how I’m modifying the metadata file in the hope to save you a little bit of time and effort. If you would like a more thorough explanation of the process please head over to my previous post.
Updating the Metadata File
If you would like to version-control your vagrant boxes locally, you need a metadata file, maybe something similar to ol8.json shown below. It defines my Oracle Linux 8 boxes (at the moment there is only one):
$ cat ol8.json
{
"name": "ol8",
"description": "Martins Oracle Linux 8",
"versions": [
{
"version": "8.4.0",
"providers": [
{
"name": "virtualbox",
"url": "file:///vagrant/boxes/ol8_8.4.0.box",
"checksum": "b28a3413d33d4917bc3b8321464c54f22a12dadd612161b36ab20754488f4867",
"checksum_type": "sha256"
}
]
}
]
}
For the sake of argument, let’s assume I want to upgrade my Oracle Linux 8.4.0 box to the latest and greatest packages that were available at the time of writing. As it’s a minor update I’ll call the new version 8.4.1. To keep the post short and (hopefully) entertaining I’m skipping the upgrade of the VM.
Option (1): jq
Fast forward to the metadata update: I need to add a new element to the versions array. I could have used jq for that purpose and it would have been quite easy:
$ jq '.versions += [{
> "version": "8.4.1",
> "providers": [
> {
> "name": "virtualbox",
> "url": "file:///vagrant/boxes/ol8_8.4.1.box",
> "checksum": "ecb3134d7337a9ae32c303e2dee4fa6e5b9fbbea5a38084097a6b5bde2a56671",
> "checksum_type": "sha256"
> }
> ]
> }]' ol8.json
{
"name": "ol8",
"description": "Martins Oracle Linux 8",
"versions": [
{
"version": "8.4.0",
"providers": [
{
"name": "virtualbox",
"url": "file:///vagrant/boxes/ol8_8.4.0.box",
"checksum": "b28a3413d33d4917bc3b8321464c54f22a12dadd612161b36ab20754488f4867",
"checksum_type": "sha256"
}
]
},
{
"version": "8.4.1",
"providers": [
{
"name": "virtualbox",
"url": "file:///vagrant/boxes/ol8_8.4.1.box",
"checksum": "ecb3134d7337a9ae32c303e2dee4fa6e5b9fbbea5a38084097a6b5bde2a56671",
"checksum_type": "sha256"
}
]
}
]
}
That would be too easy ;) Sadly I don’t have jq available on all the systems I’d like to run this script on. But wait, I have Python available.
Option (2): Python
Although I’m certainly late to to the party I truly enjoy working with Python. Below you’ll find a (shortened) version of a Python script to take care of the metadata addition.
Admittedly it does a few additional things compared to the very basic jq example. For instance, it takes a backup of the metadata file, takes and parses command line arguments etc. It’s a bit longer than a one-liner though ;)
#!/usr/bin/env python3
# PURPOSE
# add metadata about a new box version to the metadata file
# should also work with python2
import json
import argparse
import os
import sys
from time import strftime
import shutil
# Parsing the command line. Use -h to print help
parser = argparse.ArgumentParser()
parser.add_argument("version", help="the new version of the vagrant box to be added. Must be unique")
parser.add_argument("sha256sum", help="the sha256 sum of the newly created package.box")
parser.add_argument("box_file", help="full path to the package.box, eg /vagrant/boxes/ol8_8.4.1.box")
parser.add_argument("metadata_file", help="full path to the metadata file, eg /vagrant/boxes/ol8.json")
args = parser.parse_args()
# this is the JSON element to add
new_box_version = {
"version": args.version,
"providers": [
{
"name": "virtualbox",
"url": "file://" + args.box_file,
"checksum": args.sha256sum,
"checksum_type": "sha256"
}
]
}
...
# check if the box_file exists
if (not os.path.isfile(args.box_file)):
sys.exit("FATAL: Vagrant box file {} does not exist".format(args.box_file))
# read the existing metadata file
try:
with open(args.metadata_file, 'r+') as f:
metadata = json.load(f)
except OSError as err:
sys.exit ("FATAL: Cannot open the metadata file {} for reading: {}".format(args.metadata_file, err))
# check if the version to be added exists already.
all_versions = metadata["versions"]
if args.version in all_versions.__str__():
sys.exit ("FATAL: new version {} to be added is a duplicate".format(args.version))
# if the new box doesn't exist already, it's ok to add it
metadata['versions'].append(new_box_version)
# create a backup of the existing file before writing
try:
bkpfile = args.metadata_file + "_" + strftime("%y%m%d_%H%M%S")
shutil.copy(args.metadata_file, bkpfile)
except OSError as err:
sys.exit ("FATAL: cannot create a backup of the metadata file {}".format(err))
# ... and write changes to disk
try:
with open(args.metadata_file, 'w') as f:
json.dump(metadata, f, indent=2)
except OSError as err:
sys.exit ("FATAL: cannot save metadata to {}: {}".format(args.metadata_file, err))
print("INFO: process completed successfully")
That’s it! Next time I need to upgrade my Vagrant boxes I can rely on a fully automated process, saving me quite a bit of time when I’m instantiating a new Vagrant-based environment.
