Building vagrant images with mkosi

Posted on za 27 juli 2024 in systemd

Last FOSDEM, there where some talks around mkosi using it for kernel hacking and systemd integration tests. These talks got me interested in mkosi, a systemd project for building OS images. After chatting some more with the maintainers, I considered the idea of moving the arch-boxes project to mkosi. (note: it has not been decided yet on whether to move the project to mkosi yet) Arch-boxes is an Arch Linux project which creates official vagrant and cloud images for Arch Linux, using bash scripts to create these images and requiring root privileges. This made me wonder, can I create Vagrant images for Arch Linux with mkosi.

In short mkosi does not require root privileges for building, has configuration file support with profiles and does most of the heavy lifting.

What is Vagrant?

Vagrant was a popular way to setup a development virtual machine for projects, all you had to do is provide a Vagrantfile and vagrant up would setup a development VM and for example mount your code into the VM via sshfs allowing for a "reproducible" development environment.

The image you use in your Vagrantfile is called a box in Vagrant's terminology and are hosted by the Vagrant project.

Vagrant also supports the concept of providers, these are the backends on which the box would run, the default being VirtualBox, alternatives are Hyper-V, libvirt and more.

Format

Usually the Vagrant boxes are made using packer, but other tools such as kiwi, osbuild (only libvirt provider) also support creating these boxes. The alternative tools usually only support a subset of the providers packer supports. In the Linux community VirtualBox and libvirt are usually the most popular ones. At least this is what Arch Linux and Fedora supports.

The base box format is described in a very short summary what needs to be in the image: * Package manager * Enough disk space to do interesting things * A vagrant user with SSH pubkey and password-less sudo * SSH server enabled and running * Provider specific tools (for example VirtualBox guest tools)

The image format is provider dependant and is described for VirtualBox and libvirt.

mkosi

We start things off by creating a base mkosi.conf, which defines the packages we require for both the libvirt and VirtualBox provider:

[Distribution]
Distribution=arch

[Content]
Packages=
    systemd
    udev
    linux
    base
    grub
    openssh
    sudo
    btrfs-progs
    reflector

Hostname=archlinux
Locale=C.UTF-8
Timezone=UTC
Keymap=us

Bootable=yes
Bootloader=grub
BiosBootloader=grub
KernelCommandLine=net.ifnames=0

The image that mkosi creates runs systemd-firstboot by default, to avoid the first boot to prompt interactively for details we pre-configure the hostname, locale, etc.

By default only an UEFI bootable image is created, as BIOS boot is required we inform mkosi to set this up for us.

Partitioning

As described in the vagrant box format, a user should have enough disk space to do interesting things, by default mkosi makes a small ext4 image. We want to provide a 20G disk and use btrfs by default, if we want to change the partitioning in mkosi we have to define everything in a directory called mkosi.repart:

00-esp.conf

[Partition]
Type=esp
Format=vfat
CopyFiles=/boot:/
CopyFiles=/efi:/
SizeMinBytes=512M
SizeMaxBytes=512M

05-bios.conf

[Partition]
# UUID of the grub BIOS boot partition which grubs needs on GPT to
# embed itself into.
Type=21686148-6449-6e6f-744e-656564454649
SizeMinBytes=1M
SizeMaxBytes=1M

10-root.conf

[Partition]
Type=root
Format=btrfs
CopyFiles=/
SizeMinBytes=20G
SizeMaxBytes=20G

User creation / ssh

Vagrant images require a vagrant user with sudo (password-less) and a vagrant specific pubkey to be able to log in to the box. There are no user creation helpers, but there are various steps of the build process where you can execute your own scripts. For user creation we use the post package installation hook:

mkosi.postinst.chroot

useradd --create-home --user-group vagrant --password "$(openssl passwd -6 vagrant)"

install --directory --owner=vagrant --group=vagrant --mode=0700 $DESTDIR/home/vagrant/.ssh
install --owner=vagrant --group=vagrant --mode=600 authorized_keys ${DESTDIR}/home/vagrant/.ssh/authorized_keys

Creating a sudo config file for the vagrant user is as easy as dropping a file into mkosi.extra:

mkosi.extra/etc/sudoers.d/vagrant

Defaults:vagrant !requiretty
vagrant ALL=(ALL) NOPASSWD: ALL

Services

By default the vagrant box needs to run sshd, on Arch Linux services are not started automatically but this can be easily done with a systemd-preset file:

mkosi.extra/etc/systemd/system-preset/10-base-image.preset

enable sshd.service

Profiles

We have now full-filled all the steps required for a vagrant box except the provider specific tooling, for this we will use mkosi profile support. For every profile we create a directory in mkosi.profiles.

VirtualBox

For the VirtualBox image the guest utils needs to be installed and a special service running for more advanced (for example shared clipboard, folders). We can append the base packages in the profile specific mkosi.conf:

mkosi.conf

[Output]
ImageId=Arch-Linux-x86_64-virtualbox

[Content]
Packages=
        virtualbox-guest-utils-nox

Enabling the vboxservice is adding another systemd-preset file:

mkosi.extra/etc/systemd/system-preset/15-virtualbox.preset

enable vboxservice.service

Libvirt

Libvirt does not require any specific packages to be installed or services enabled, we only configure the ImageId to differ from VirtualBox:

mkosi.conf

[Output]
Format=disk
ImageId=Arch-Linux-x86_64

Image

By default mkosi outputs a raw image, Vagrant boxes require something different depending on the provider. To allow a user to easily customize the build image mkosi offers a mkosi.postoutput script to be run.

Libvirt

The box format is described in the official provider docs. We need to provide a tarball containing:

  • Vagrantfile
  • box.img (as qcow2)
  • metadata.json

mkosi.postoutput

#!/bin/bash

IMAGE_NAME="Arch-Linux-x86_64-libvirt-latest.box"

cp "${SRCDIR}/Vagrantfile.libvirt" "${OUTPUTDIR}/Vagrantfile"

qemu-img convert -f raw "${OUTPUTDIR}/${IMAGE_ID}.raw" -O qcow2 "${OUTPUTDIR}/box.img"
# Calculate the disk size for metadata.json, this size needs to be in Gb and without a fraction so round this up.
DISK_SIZE=$(qemu-img info --output=json "${OUTPUTDIR}/box.img" | jq '."virtual-size" / 1024 / 1024 / 1024 + 0.5 | round')

echo '{"format":"qcow2","provider":"libvirt","virtual_size":'"${DISK_SIZE}"'}' > "${OUTPUTDIR}/metadata.json"
tar -C "${OUTPUTDIR}" -czf "${OUTPUTDIR}/${IMAGE_NAME}" Vagrantfile metadata.json box.img

In this script the OUTPUTDIR is the staging directory used to store artifacts generated during the build, from the SRCDIR we copy the static libivrt specific Vagrantfile, convert the created image to qcow2 and name it box.img. The most "tricky" part is generating the metadata.json file which needs to know the size of the image, specifying the wrong size can lead booting without a root partition or a non-bootable image.

The last step creates the tar archive which can be imported using vagrant box add archvirt /path/to/file.box.

VirtualBox

The VirtualBox format is similar, it requires a tar archive with:

  • Vagrantfile
  • metadata.json
  • packer-virtualbox.vmdk (vm image as vmdk)
  • box.ovf file (Open Virtualization Format)

mkosi.postout

#!/bin/bash

IMAGE_NAME="Arch-Linux-x86_64-virtualbox-latest.box"

cp "${SRCDIR}/Vagrantfile.virtualbox" "${OUTPUTDIR}/Vagrantfile"
echo '{"provider":"virtualbox"}' > "${OUTPUTDIR}/metadata.json"
qemu-img convert -f raw -O vmdk "${OUTPUTDIR}/${IMAGE_ID}.raw" "${OUTPUTDIR}/packer-virtualbox.vmdk"
sed -e "s/MACHINE_UUID/$(uuidgen)/" \
  -e "s/DISK_UUID/$(uuidgen)/" \
  -e "s/DISK_CAPACITY/$(qemu-img info --output=json "${OUTPUTDIR}/packer-virtualbox.vmdk" | jq '."virtual-size"')/" \
  -e "s/UNIX/$(date +%s)/" \
  -e "s/MAC_ADDRESS/080027AF9290/" \
  box.ovf > "${OUTPUTDIR}/box.ovf"
tar -C "${OUTPUTDIR}" -czf "${OUTPUTDIR}/${IMAGE_NAME}" Vagrantfile metadata.json packer-virtualbox.vmdk box.ovf

Similar to the libvirt script, we copy the Vagrantfile, create a metadata.json file and convert the created image to a vmdk image. The last step is rather hackily creating the required ovf file from a known template.

The xml part can use some improvements, although other projects also simply replace a XML template, it is worth investigating if xmlstarlet for example is a better way to format and validate the created xml file.

Conclusion

It was quite easy to create a Vagrant image with mkosi, most of the discovery work was already done in arch-boxes so all that was left was moving it over to mkosi configuration and figuring out the post installation and image conversion scripts.

Next up will be converting the cloud images to mkosi. You can find all the files mentioned in this post in the following GitHub repository.

Arch Linux mkosi systemd