Reproducing Arch [core] repository packages

As Arch Linux we are working on reproducible builds for a while and have a continuous test framework rebuilding package updated in our repositories. This test does an asp checkout of a package and builds it twice in a schroot, we do not try to reproduce actual repository packages yet. In the end this is however what we want to achieve, giving users the ability to verify a repository package by rebuilding it on their own hardware.

repro was created to achieve this goal, it creates a build chroot with the packages installed during build (from the .BUILDINFO file), sets SOURCE_DATE_EPOCH accordingly, fetches the correct PKGBUILD and then builds the package. This tool however does not run in a CI environment yet, so a bash script was hacked together to build all our [core] (232) packages one by one leading to 0% reproducibility with the following issues:

• makepkg options differed, these options are recorded in BUILDINFO but not set yet by repro.
• Packages where not reproducible (108 due to makepkg recording false sizes in .PKGINFO).
• PKGBUILD fetching logic failed (21 packages).
• Failed to download source files due to DNS issues (popt, libpipeline, acl, mlocate).
• Packages did not build due to OOM and other issues (lib32-gcc-libs, gcc-obj, gcc-libs, gcc-go, gcc-fortran, gcc, fakeroot).
• asp failed to get package due unknown reasons (libusb).
• Packages not reproducible (s-nail, amd-ucode, syslinux, texinfo, tzdata, patch, .. and more).
• libpcap GPG verification failed.
• Builds with different packages installed leading to a different BUILDINFO due to an issue in repro (unknown).

Logs of the process can be found here.

This shows that still a lot has still to be done for reproducible Arch Linux, in the next pacman release the size issue should be resolved. Which will lead to at least some reproducible packages! Repro has to be improved and non reproducible packages sorted out. In a few months I intend to retry reproducing [core] packages and have at least > 0% reproducibility!