Skip to content

Commit c18ec25

Browse files
committed
docker: centralize digest handling and helpers; improve Nix, X11, USB, and KVM handling; add pin/pin-and-run helpers and docs
- Add print_digest_info() and consolidate digest output/formatting - Add docker/get_digest.sh and docker/pin-and-run.sh helpers - Resolve digest env/file precedence and normalize digest formats - Add Nix preflight checks, auto-install/enable flakes options, and rebuild logic - Harden Xauthority creation and safer USB cleanup - Rename run_pinned.sh → pin-and-run.sh and update README and qemu docs Signed-off-by: Thierry Laurion <insurgo@riseup.net>
1 parent 81dbf23 commit c18ec25

9 files changed

Lines changed: 1179 additions & 232 deletions

File tree

README.md

Lines changed: 150 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,60 @@ Building heads with prebuilt and versioned docker images
3636
Heads now builds with Nix built docker images since https://github.com/linuxboot/heads/pull/1661.
3737

3838
The short path to build Heads is to do what CircleCI would do (./docker_repro.sh under heads git cloned directory):
39-
- Install _docker-ce_ for your OS of choice (refer to their documentation)
39+
- Install Docker (docker-ce) for your OS by following Docker's official installation instructions: https://docs.docker.com/engine/install/
4040
- run `./docker_repro.sh make BOARD=XYZ`
4141

42+
Note: `./docker_repro.sh` is the canonical, reproducible way to build and test Heads. The `docker_local_dev.sh` helper is intended for developers who need to modify the local image built from `flake.nix`/`flake.lock` and is not recommended for general testing.
43+
44+
Important: the supported and tested workflow uses the provided Docker
45+
wrappers (`./docker_repro.sh`, `./docker_local_dev.sh`, or
46+
`./docker_latest.sh`). Host-side installation of QEMU, `swtpm`, or other
47+
QEMU-related tooling is unnecessary for the standard workflow and is not
48+
part of the tested configuration. Only advanced or edge-case workflows
49+
may require installing those tools on the host (see `targets/qemu.md`
50+
for guidance).
51+
52+
The Docker images produced by our Nix build include QEMU
53+
(`qemu-system-x86_64`), `swtpm` / `libtpms`, `canokey-qemu` (a virtual
54+
OpenPGP smartcard), and other userspace tooling required to build and
55+
test QEMU boards. If you use `./docker_repro.sh` you only need Docker on
56+
the host (for example, `docker-ce`). For KVM acceleration the host
57+
must expose `/dev/kvm` (load `kvm_intel` / `kvm_amd` as appropriate);
58+
our wrapper scripts mount `/dev/kvm` automatically when it exists.
59+
60+
If you plan to manage disk images or use `qemu-img` snapshots on the
61+
host (outside containers), install the `qemu-utils` package locally
62+
(which provides `qemu-img`).
63+
64+
If you do not specify `USB_TOKEN` when running QEMU targets, the container will use the included `canokey-qemu` virtual token by default; set `USB_TOKEN` (or use `hostbus`/`hostport`/`vendorid,productid`) to forward a hardware token instead.
65+
66+
Wrapper options & environment variables
67+
---
68+
69+
`./docker_repro.sh` `./docker_latest.sh` `./docker_local_dev.sh`:
70+
- `HEADS_DISABLE_USB=1` — disable automatic USB passthrough and the
71+
automatic USB cleanup (default: `0`).
72+
- `HEADS_X11_XAUTH=1` — force mounting your `${HOME}/.Xauthority` into the container for X11 authentication. When set the helper will bypass programmatic Xauthority generation and mount your `${HOME}/.Xauthority` (if present); if the file is missing the helper will warn and will not attempt automatic cookie creation (GUI may fail).
73+
74+
`./docker_local_dev.sh`:
75+
- `HEADS_SKIP_DOCKER_REBUILD=1` — skip automatically rebuilding the local image when `flake.nix`/`flake.lock` are dirty
76+
- `HEADS_NIX_EXTRA_FLAGS` — extra flags to append to Nix commands during rebuild (for example: `--extra-experimental-features 'nix-command flakes'`)
77+
- `HEADS_NIX_VERBOSE=1` — stream Nix output live during rebuilds (default: on for dev scripts)
78+
- `HEADS_AUTO_INSTALL_NIX=1` — automatically attempt to install Nix (single-user) when missing (interactive prompt suppressed)
79+
- `HEADS_AUTO_ENABLE_FLAKES=1` — automatically enable flakes by writing `experimental-features = nix-command flakes` to `$HOME/.config/nix/nix.conf` (interactive prompt suppressed)
80+
- `HEADS_MIN_DISK_GB` — minimum free disk space in GB required on `/nix` (or `/` if `/nix` missing) for building (default: `50`)
81+
- `HEADS_SKIP_DISK_CHECK=1` — skip the preflight disk-space check
82+
- `HEADS_STRICT_REBUILD=1` — when set, treat rebuild failures (including `No 'fromImage' provided`) as fatal
83+
- `HEADS_ALLOW_UNPINNED_LATEST=1` — when set, bypass the interactive warning that using `:latest` in `./docker_latest.sh` is a supply-chain risk (otherwise `:latest` requires confirmation or set `DOCKER_LATEST_DIGEST`)
84+
- `DOCKER_REPRO_DIGEST` — pin the image used by `./docker_repro.sh` to an immutable digest: `tlaurion/heads-dev-env@<digest>` (recommended for reproducible and secure builds)
85+
86+
For details about selecting or forwarding a physical USB token to QEMU
87+
(handled by the `USB_TOKEN` make variable), see `targets/qemu.md`.
88+
89+
Note: when USB passthrough is active the wrappers will detect processes that may be holding a USB token (for example `scdaemon` or `pcscd`). The wrapper will warn and, on interactive shells, give a 3s abort window before attempting to kill those processes to free the token. Set `HEADS_DISABLE_USB=1` to opt out of this automatic cleanup.
90+
91+
Example: `HEADS_DISABLE_USB=1 ./docker_repro.sh make BOARD=qemu-coreboot-fbwhiptail-tpm2 run`
92+
4293
Using Nix local dev environement / building docker images with Nix
4394
==
4495

@@ -61,13 +112,30 @@ Build docker from nix develop layer locally
61112
* `mkdir -p ~/.config/nix`
62113
* `echo 'experimental-features = nix-command flakes' >>~/.config/nix/nix.conf`
63114

115+
Notes on automation and requirements:
116+
117+
- The `./docker_local_dev.sh` helper will attempt to ensure Nix and flakes are available when you run it interactively. If Nix is missing it can optionally install it for you and prompt to enable flakes; set `HEADS_AUTO_INSTALL_NIX=1` / `HEADS_AUTO_ENABLE_FLAKES=1` to suppress prompts.
118+
- Building the Docker image and populating `/nix` can require significant disk space — we recommend at least **50 GB** free on `/nix` (or `/` if `/nix` is not present). Adjust via `HEADS_MIN_DISK_GB` or skip the check with `HEADS_SKIP_DISK_CHECK=1`.
119+
- The Nix installer requires a downloader; either `curl` or `wget` must be available on the host. The helper will guide you to install one if neither is present.
120+
- For reproducible builds prefer `./docker_repro.sh`; `./docker_local_dev.sh` is intended for development and will rebuild the local image when `flake.nix`/`flake.lock` are dirty (unless `HEADS_SKIP_DOCKER_REBUILD=1`).
64121

65122
#### Build image
66123

67124
* Have docker and Nix installed
68125

69126
* Build nix developer local environment with flakes locked to specified versions
70-
* `./docker_local_dev.sh`
127+
* Manual: `nix --print-build-logs --verbose build .#dockerImage && docker load < result`
128+
* Helper: `./docker_local_dev.sh` will perform a conditional rebuild when `flake.nix`/`flake.lock` are dirty (unless `HEADS_SKIP_DOCKER_REBUILD=1`).
129+
130+
Using `./docker_local_dev.sh`
131+
132+
* `./docker_local_dev.sh` is a developer helper that ensures a local Nix-based Docker image (`linuxboot/heads:dev-env`) is available for interactive development. It performs a few preflight checks and interactive prompts to make the process easier:
133+
- Ensures `nix` is installed and **flakes** are enabled; if missing it will prompt to install Nix and enable flakes. Set `HEADS_AUTO_INSTALL_NIX=1` and/or `HEADS_AUTO_ENABLE_FLAKES=1` to suppress prompts and proceed automatically.
134+
- Requires either `curl` or `wget` to fetch the Nix installer; if neither is present the script will print how to install one and abort.
135+
- Checks disk space on `/nix` (or `/` if `/nix` is absent); default minimum is **50 GB** (`HEADS_MIN_DISK_GB=50`) — override or skip the check with `HEADS_SKIP_DISK_CHECK=1`.
136+
- If `flake.nix` or `flake.lock` are dirty (uncommitted changes), the helper will rebuild the local Docker image. Skip automatic rebuilds with `HEADS_SKIP_DOCKER_REBUILD=1`.
137+
- Nix output is streamed live by default (`HEADS_NIX_VERBOSE=1`). You can pass additional Nix flags via `HEADS_NIX_EXTRA_FLAGS`.
138+
- If the Nix build reports `No 'fromImage' provided` (expected when no base image is used), the helper continues by default; set `HEADS_STRICT_REBUILD=1` to make such errors fatal.
71139

72140
On some hardened OSes, you may encounter problems with ptrace.
73141
```
@@ -88,16 +156,17 @@ Your local docker image "linuxboot/heads:dev-env" is ready to use, reproducible
88156

89157
Jump into nix develop created docker image for interactive workflow
90158
====
91-
There is 3 helpers:
92-
- `./docker_local_dev.sh`: for developers wanting to customize docker image built from flake.nix(nix devenv creation) and flake.lock (pinned versions used by flake.nix)
93-
- `./docker_latest.sh`: for Heads developers, wanting to use latest published docker images to develop Heads
94-
- `./docker_repro.sh`: versioned docker image used under CircleCI to produce reproducivle builds, both locally and under CircleCI. **Use this one if in doubt**
159+
There are three helpers:
160+
- `./docker_local_dev.sh`: developer-only — customize the local image built from `flake.nix`/`flake.lock` (not recommended for general testing)
161+
- `./docker_latest.sh`: convenience — use the latest published Docker image for development
162+
- `./docker_repro.sh`: canonical, reproducible builds that match CircleCI; **this is the recommended way to build and test Heads**
95163

96164
ie: `./docker_repro.sh` will jump into CircleCI used versioned docker image for that Heads commit id to build images reproducibly if git repo is clean (not dirty).
97165

98166
From there you can use the docker image interactively.
99167

100-
`make BOARD=board_name` where board_name is the name of the board directory under `./boards` directory.
168+
Use `./docker_repro.sh make BOARD=board_name` to run builds and tests (this runs `make` inside the canonical Docker image).
169+
If you are already inside the container interactively, run `make BOARD=board_name` as usual.
101170

102171

103172
One such useful example is to build and test qemu board roms and test them through qemu/kvm/swtpm provided in the docker image.
@@ -153,6 +222,80 @@ docker push "$docker_hub_repo:latest"
153222

154223
This can be put in reproducible oneliners to ease maintainership.
155224

225+
Maintenance tip: to make pinned, reproducible images easy to manage inside the repository, maintainers should pin the canonical reproducible image using the repository file `docker/DOCKER_REPRO_DIGEST`, which is read by `./docker_repro.sh`.
226+
227+
Acceptable formats include `sha256:<64-hex>`, `sha256-<64-hex>` (normalized to `sha256:<hex>`), or just `<64-hex>` (normalized to `sha256:<hex>`). The helper will normalize these formats and produce an image reference like `tlaurion/heads-dev-env@sha256:<hex>`.
228+
229+
If you need to pin the convenience `./docker_latest.sh` wrapper, set the `DOCKER_LATEST_DIGEST` environment variable locally; we do not maintain a `docker/DOCKER_LATEST_DIGEST` file in the repository because 'latest' is a user-level convenience and should be explicitly chosen. Without a digest, `./docker_latest.sh` will prompt before using an unpinned `:latest` unless `HEADS_ALLOW_UNPINNED_LATEST=1` is set in the environment.
230+
231+
Example: obtain the immutable digest for a published image and use it to force `docker_latest.sh` to use an immutable image:
232+
233+
```bash
234+
# 1) Obtain the digest for a published image (exact repo:name:tag form is required)
235+
#
236+
# Tip: inspect tags on Docker Hub: https://hub.docker.com/layers/tlaurion/heads-dev-env/
237+
# Click a tag to see details (Content type, Digest (sha256:...), Size, Last updated).
238+
# Use the shown tag name with docker pull, e.g.:
239+
# docker pull tlaurion/heads-dev-env:v0.2.6
240+
#
241+
# Example: pull the image and then obtain its digest locally
242+
# docker pull tlaurion/heads-dev-env:v0.2.6
243+
# ./docker/get_digest.sh tlaurion/heads-dev-env:v0.2.6
244+
#
245+
# Or: query the registry for the digest and optionally pull it when prompted
246+
# ./docker/get_digest.sh tlaurion/heads-dev-env:v0.2.6
247+
# (the script will show the remote digest and ask if you want to pull the image to create a local repo@digest)
248+
#
249+
# Use -y to auto-pull and return the digest in one go:
250+
# ./docker/get_digest.sh -y tlaurion/heads-dev-env:v0.2.6
251+
252+
./docker/get_digest.sh tlaurion/heads-dev-env:v0.2.6
253+
# Output (example): tlaurion/heads-dev-env@sha256:50a9110c...\nsha256:50a9110c...
254+
255+
# 2) If the image is not present locally, the helper will offer to pull it so a local repo@digest is available.
256+
# Use '-y' / '--yes' to skip the interactive prompt and pull automatically.
257+
./docker/get_digest.sh -y tlaurion/heads-dev-env:latest
258+
259+
# 3) Export the raw digest into the env var expected by the wrapper
260+
export DOCKER_LATEST_DIGEST=$(./docker/get_digest.sh tlaurion/heads-dev-env:latest | tail -n1)
261+
262+
# 4) Run the convenience wrapper using the pinned digest
263+
DOCKER_LATEST_DIGEST=$DOCKER_LATEST_DIGEST ./docker_latest.sh make BOARD=qemu-coreboot-fbwhiptail-tpm2
264+
265+
Note: when a digest is discovered, helpers print a concise summary to help auditing, for example:
266+
267+
Image: tlaurion/heads-dev-env@sha256:50a9...
268+
Digest: sha256:50a9...
269+
Resolved from: local|registry API|env|file
270+
Tip: export DOCKER_LATEST_DIGEST=sha256:50a9...
271+
272+
This makes it easy to copy/pin digests or verify provenance.
273+
# Convenience: helper to obtain a digest and run a wrapper pinned to it
274+
# Example: obtains digest and runs the 'latest' wrapper pinned to that digest
275+
./docker/pin-and-run.sh tlaurion/heads-dev-env:v0.2.6 -- make BOARD=qemu-coreboot-fbwhiptail-tpm2
276+
# Auto-pull and run
277+
./docker/pin-and-run.sh -y tlaurion/heads-dev-env:v0.2.6 -- make BOARD=qemu-coreboot-fbwhiptail-tpm2
278+
279+
# To use a different wrapper (e.g. repro):
280+
./docker/pin-and-run.sh tlaurion/heads-dev-env:v0.2.6 -- ./docker_repro.sh make BOARD=qemu-coreboot-fbwhiptail-tpm2
281+
282+
283+
```
284+
285+
Alternative (manual) commands without the helper script:
286+
287+
```bash
288+
docker pull tlaurion/heads-dev-env:latest
289+
# prints full repo@digest (if available)
290+
docker inspect --format='{{index .RepoDigests 0}}' tlaurion/heads-dev-env:latest
291+
# to get only the digest portion:
292+
docker inspect --format='{{index .RepoDigests 0}}' tlaurion/heads-dev-env:latest | cut -d'@' -f2
293+
```
294+
295+
Notes: some registries or Docker versions may require `docker manifest inspect` or `skopeo inspect` to obtain an authoritative digest; the helper script tries `docker inspect` first, then `docker manifest inspect` when available.
296+
297+
Update the appropriate file after publishing a new image to keep the repo in sync.
298+
156299
Test image in dirty mode:
157300
```
158301
docker_version="vx.y.z" && docker_hub_repo="tlaurion/heads-dev-env" && sed "s@\(image: \)\(.*\):\(v[0-9]*\.[0-9]*\.[0-9]*\)@\1\2:$docker_version@" -i .circleci/config.yml && nix --print-build-logs --verbose develop --ignore-environment --command true && nix --print-build-logs --verbose build .#dockerImage && docker load < result && docker tag linuxboot/heads:dev-env "$docker_hub_repo:$docker_version" && docker push "$docker_hub_repo:$docker_version"

docker/DOCKER_REPRO_DIGEST

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Optional: pin the Docker image used by ./docker_repro.sh to an immutable digest
2+
# This file is read by docker_repro.sh if DOCKER_REPRO_DIGEST is not set in the
3+
# environment. The first non-empty, non-comment line is used as the digest value.
4+
# Acceptable formats are:
5+
# - sha256:<64-hex>
6+
# - sha256-<64-hex> (will be normalized to sha256:<hex>)
7+
# - <64-hex> (will be normalized to sha256:<hex>)
8+
# Example:
9+
# sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
10+
11+
# Place the digest on the first non-comment line below (remove the leading '#')
12+
sha256-50a9110cdfc6a74a383169d7c624139c3b3e05567b87203498118a8a33dd79f1

0 commit comments

Comments
 (0)