Skip to content

Experimental Ubuntu 26.04 (resolute) build support#26

Merged
passcod merged 26 commits into
mainfrom
experimental-26.04
May 8, 2026
Merged

Experimental Ubuntu 26.04 (resolute) build support#26
passcod merged 26 commits into
mainfrom
experimental-26.04

Conversation

@passcod
Copy link
Copy Markdown
Member

@passcod passcod commented Apr 25, 2026

Summary

Adds the plumbing to build images against arbitrary Ubuntu suites and validates it against Ubuntu 26.04 (resolute), now that 26.04 LTS has shipped.

just arch=amd64 variant=cloud ubuntu_suite=resolute build

What changed

  • Suite parameterisation. UBUNTU_SUITE plumbed end-to-end through the justfile, image/build.sh, image/configure.sh, and the chroot scripts. New TAILSCALE_SUITE env/justfile var lets the Tailscale apt repo be set independently from the Ubuntu suite (defaults to ubuntu_suite).
  • Dracut hostonly: per-suite portability mechanism. Dracut's default is hostonly=yes (per the dracut.conf(5) resolute manpage), so without an override the shipped image's initramfs would be bound to the build host. On noble, hostonly=no is broken, so we keep the existing hostonly=yes + sloppy + force-include workaround. On 26.04+, we install /etc/dracut.conf.d/01-portable-image.conf setting hostonly="no", which auto-includes all kernel modules. Verified: resolute initramfs lands at ~82M (vs. typical hostonly=yes ~30-50M).
  • Installer specialises to the target. The installer now strips the image's 01-portable-image.conf before its post-install dracut --force, so dracut's default (hostonly=yes) takes effect and the rebuilt initramfs is specialised to the target machine. Existing fstab/UUID/LUKS workarounds in the installer remain valid.
  • Renamed 01-fix-hostonly-noble.conf01-fix-hostonly.conf (filename is no longer suite-specific even though it's still gated to noble).
  • Caddy not shipped on 26.04+. The Ubuntu archive's caddy is too old, and we no longer have a strong reason to ship our own in the base image. Spec, install list, and test all gated.
  • Podman on 26.04+ comes from the Ubuntu archive. Resolute ships podman 5.7.0, well past our 5.0 floor. The bes-tools repo's podman is unsatisfiable on resolute (depends on noble's t64-transition libs that don't exist there). New per-suite apt pin: wildcard 999 on noble (existing); bestool 999 + wildcard 100 on 26.04+, so apt prefers Ubuntu's archive for everything except bestool.
  • cloud-init verification updated. 26.04's cloud-init dropped the unified cloud-init.service wrapper; cloud-init.target is wired in dynamically by cloud-init-generator at boot. Test now checks for the generator binary on non-noble suites.
  • Metal IMAGE_SIZE bumped to 8G. Resolute's package set plus linux-firmware doesn't fit in 5G — metal builds hit btrfs ENOSPC mid-install. Cloud (which doesn't ship linux-firmware) stays at 5G.

Status

End-to-end verified locally on every arch/variant combination (with the latest dracut hostonly fix):

Suite Arch Variant Structure-test result
noble amd64 cloud 188 / 0 (regression check — no change)
resolute amd64 cloud 165 / 0 (with portable hostonly=no config)
resolute amd64 metal 170 / 0
resolute arm64 cloud 163 / 0 (cross-built; from before the dracut fix — re-verify after merge)
resolute arm64 metal 170 / 0 (cross-built; from before the dracut fix — re-verify after merge)

Filed as a draft because:

  • Not yet boot-tested in QEMU (host is missing OVMF firmware; CI will catch this).
  • arm64 builds were verified pre-dracut-fix; should be re-run after the new commits land but they exercise the same code path so I'm not expecting issues.
  • CI promotion, AMI naming, and docs/spec/ci-cd.md are intentionally untouched.

Test plan

  • amd64 cloud build + structure test on resolute (with portable hostonly=no).
  • amd64 metal build + structure test on resolute (LUKS + dracut path validates).
  • arm64 cloud + metal cross-builds + structure tests (pre-dracut-fix; re-verify needed).
  • noble amd64 cloud regression — 188/0, unchanged.
  • Confirmed 26.04 initramfs is non-hostonly (~82M includes full kernel module set).
  • Boot the 26.04 cloud image; confirm cloud-init runs at first boot, ssh comes up, and chrony is synced.
  • Boot the 26.04 metal image; confirm cloud-init runs at first boot, ssh comes up, chrony is synced, and LUKS rotates its key.
  • Boot test the installer end-to-end on 26.04 to verify the hostonly-strip step actually produces a specialised target initramfs.

@passcod passcod force-pushed the experimental-26.04 branch from 7dc26ab to befe594 Compare April 25, 2026 06:32
@passcod passcod changed the base branch from main to chrony-time-sync April 25, 2026 06:32
@passcod passcod force-pushed the experimental-26.04 branch 3 times, most recently from c0e9db1 to cfeb277 Compare April 25, 2026 09:39
@passcod passcod force-pushed the experimental-26.04 branch from cfeb277 to 32c26bf Compare May 6, 2026 07:36
@passcod passcod force-pushed the chrony-time-sync branch from 1c7d261 to 29629b0 Compare May 6, 2026 07:40
@passcod passcod force-pushed the experimental-26.04 branch from 32c26bf to 24666f9 Compare May 6, 2026 07:40
@passcod passcod force-pushed the chrony-time-sync branch from 29629b0 to ab76a8f Compare May 6, 2026 07:48
@passcod passcod force-pushed the experimental-26.04 branch from 24666f9 to 8624c22 Compare May 6, 2026 07:48
@passcod passcod force-pushed the chrony-time-sync branch from ab76a8f to fe92e76 Compare May 6, 2026 08:36
@passcod passcod force-pushed the experimental-26.04 branch 3 times, most recently from 4f39b12 to 92226b9 Compare May 6, 2026 09:22
@passcod passcod force-pushed the chrony-time-sync branch from fe92e76 to e7df674 Compare May 6, 2026 09:22
@passcod passcod force-pushed the experimental-26.04 branch from 92226b9 to 7562e1e Compare May 6, 2026 09:22
@passcod passcod marked this pull request as ready for review May 6, 2026 09:23
@passcod passcod changed the base branch from chrony-time-sync to fix-arm64-partition-race May 6, 2026 12:42
@passcod passcod force-pushed the experimental-26.04 branch from 7562e1e to 282b2ab Compare May 6, 2026 12:42
@passcod passcod force-pushed the fix-arm64-partition-race branch from 4405c04 to c36b485 Compare May 6, 2026 12:47
@passcod passcod force-pushed the experimental-26.04 branch from 282b2ab to 0842b6a Compare May 6, 2026 12:47
Base automatically changed from fix-arm64-partition-race to main May 6, 2026 23:47
@passcod passcod force-pushed the experimental-26.04 branch from 8280226 to 2eeb9b7 Compare May 7, 2026 00:53
passcod added 4 commits May 7, 2026 12:54
The dracut hostonly settings are not suite-specific.
Let 'just ubuntu_suite=resolute ubuntu_version=26.04 tailscale_suite=noble
variant=cloud build' produce an experimental 26.04-based image without
touching the 24.04 CI path.

The Tailscale apt repo is keyed by Ubuntu codename and typically lags
new Ubuntu releases during the RC window; TAILSCALE_SUITE defaults to
UBUNTU_SUITE and can be overridden independently. The Tailscale
packages are compatible across recent Ubuntu versions.
The hostonly=yes / sloppy config is a workaround for a dracut bug in
noble's release of dracut where non-hostonly mode produces a
non-functional initramfs. Newer Ubuntu suites run dracut in its default
mode, which already includes the required hardware/cloud modules. The
force-include directives in 03-hardware-drivers.conf and
04-cloud-drivers.conf only apply under the hostonly workaround.
Dracut's default is hostonly=yes per the resolute manpage, contrary to
my earlier assumption. With no override on 26.04 the shipped initramfs
ends up bound to the build environment, not the target hardware.

Fix: on 26.04+ the image must explicitly set hostonly="no" to produce
a portable initramfs. The installer is then responsible for stripping
that override before its own dracut --force, so the target machine gets
a hostonly=yes (default) initramfs specialised to its hardware.

Updates r[image.boot.dracut], r[image.boot.hardware-drivers],
r[image.boot.cloud-drivers], and adds a new install-time step to
r[installer.boot.regen].
@passcod passcod force-pushed the experimental-26.04 branch 10 times, most recently from 08d48cb to 4c1c5e3 Compare May 7, 2026 09:53
passcod added 14 commits May 8, 2026 00:06
Adds /etc/dracut.conf.d/01-portable-image.conf with hostonly="no"
on non-noble suites, so the shipped image's initramfs includes
modules for all kernel-supported hardware rather than just the
build host's. Noble still uses the hostonly=yes + force-include
workaround.
The 26.04+ images carry /etc/dracut.conf.d/01-portable-image.conf
(hostonly="no") so the shipped initramfs is portable across
hardware. For the target machine, we want dracut's default
(hostonly=yes) so the rebuilt initramfs is specialised. Remove
the override before running dracut in chroot.
Mirrors the impl change: assert /etc/dracut.conf.d/01-portable-image.conf
exists and contains hostonly="no" on non-noble suites.
Adds a (arch × suite) matrix to images-cloud, images-metal, iso,
container-test, and register-amis. Resolute (26.04) now runs the full
pipeline: build → ISO → container-test → AMI registration on tagged
releases.
upload-artifact v7 added 'archive: false' to skip the zip wrapper. For
multi-GB uploads the local zip overhead is real, even with
compression-level: 0 (it still streams every byte through the zip
encoder). Switch every upload step to archive: false.
archive: false requires single-file uploads. The original 'Upload
converted formats' step matched both *.vmdk and *.qcow2, which trips
'Found 2 files to upload'. Split into separate per-format uploads so
each artifact is one file.
archive: false ignores the upload step's name: parameter — the artifact
is named after the file's basename. Updates downstream jobs:

- iso, register-ami: download cloud raw via pattern (the basename has a
  build-date suffix that varies between runs); merge-multiple flattens
  it into the expected directory structure.
- container-test: ISO basenames are deterministic (no date), so an
  exact name lookup works.
- release: switch from per-artifact-dir loops to pattern: '*' +
  merge-multiple, then glob by extension.
A long build crossing midnight UTC produces inconsistent filenames
between parallel matrix entries, which breaks the pattern-matched
downloads in iso/register-ami. Have a single top-level prep job emit
the build date once; every matrix leg picks it up via $BUILD_DATE so
all uploads share one stamp.
archive: false ignores the upload step's name: parameter — those
fields are silently dead. The .raw.size sidecar globs in the raw
uploads also match nothing (no producer in tree). Cleaning up both.
tracey's parser ignores the first of two consecutive '# r[...]'
comment lines (no other location in the codebase uses that pattern,
which is the clue). Joining them onto a single comment line makes
both ci.output-arch and ci.output-suite annotations visible.
GitHub keeps at most ONE pending run per concurrency group, so a group
keyed only on the ref evicts (cancels) older queued runs when new
events fire — particularly noisy with stacked PRs, where pushing to
one branch cascades synchronize events into the dependent PRs.
Including the SHA gives each unique commit its own queue: same SHA
retriggered still dedups, different SHAs run independently. Same
treatment applied to both build.yml and checks.yml.
With the per-commit concurrency group, the only collision left is the
same SHA being retriggered — and there 'cancel the older run, take the
latest' is what we want. (checks.yml already had it true.)
.img is more widely recognised by image-flashing tools (rpi-imager,
balena-etcher, gnome-disks, etc.). Rename the disk image file extension
across the full pipeline: image build, iso source-image lookup, AMI
registration, release artifacts, manual-testing docs.
@passcod passcod force-pushed the experimental-26.04 branch from 4c1c5e3 to 9d1062b Compare May 7, 2026 13:10
The 26.04 cryptsetup package's systemd-cryptsetup binary moved to a
separate package (systemd-cryptsetup), and is only a Recommends of
cryptsetup — so --no-install-recommends drops it. Without that
binary, dracut's 71systemd-cryptsetup module check() returns 1 and is
dropped from the initramfs. The 70crypt module then installs only its
crypt-generator.sh (which delegates to systemd-cryptsetup when systemd
is present in the initramfs) and skips the cryptsetup binary. Result:
nothing actually unlocks the LUKS root at boot.

Pulling systemd-cryptsetup in explicitly fixes it.
@passcod passcod force-pushed the experimental-26.04 branch from 9d1062b to 4e0ee5f Compare May 7, 2026 13:35
@passcod passcod merged commit fb68e25 into main May 8, 2026
27 checks passed
@passcod passcod deleted the experimental-26.04 branch May 8, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant