Skip to content

Commit f4c3a85

Browse files
committed
docker: centralize X11/KVM/USB handling
- Add `docker/common.sh` with helpers: `build_docker_opts`, `run_docker`, `usage`, and `kill_usb_processes`. - Programmatic Xauthority (`/tmp/.docker.xauth-<uid>`) with fallback to `~/.Xauthority` and `HEADS_X11_XAUTH` override. - Detect and mount `/dev/kvm` when available; handle USB passthrough and token cleanup. Add `HEADS_DISABLE_USB` and avoid prompting for sudo in non-interactive shells (prompt only on TTY). - Refactor `docker_*` wrappers to source shared helpers and remove duplicated device/X11/KVM/USB logic. Docs: - `README.md` and `targets/qemu.md`: document `canokey-qemu` inclusion and default token behavior; make Docker wrappers the canonical workflow and mark host-side QEMU/swtpm installs as unnecessary. Tests: - `bash -n` and `shellcheck` passed; runtime `build_docker_opts` validation created Xauthority and showed expected flags. Signed-off-by: Thierry Laurion <insurgo@riseup.net>
1 parent 81dbf23 commit f4c3a85

6 files changed

Lines changed: 340 additions & 213 deletions

File tree

README.md

Lines changed: 46 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,48 @@ Building heads with prebuilt and versioned docker images
3636
Heads now builds with Nix built docker images since https://github.com/linuxboot/heads/pull/1661.
3737

3838
The short path to build Heads is to do what CircleCI would do (./docker_repro.sh under heads git cloned directory):
39-
- Install _docker-ce_ for your OS of choice (refer to their documentation)
39+
- Install Docker (docker-ce) for your OS by following Docker's official installation instructions: https://docs.docker.com/engine/install/
4040
- run `./docker_repro.sh make BOARD=XYZ`
4141

42+
Note: `./docker_repro.sh` is the canonical, reproducible way to build and test Heads. The `docker_local_dev.sh` helper is intended for developers who need to modify the local image built from `flake.nix`/`flake.lock` and is not recommended for general testing.
43+
44+
Important: the supported and tested workflow uses the provided Docker
45+
wrappers (`./docker_repro.sh`, `./docker_local_dev.sh`, or
46+
`./docker_latest.sh`). Host-side installation of QEMU, `swtpm`, or other
47+
QEMU-related tooling is unnecessary for the standard workflow and is not
48+
part of the tested configuration. Only advanced or edge-case workflows
49+
may require installing those tools on the host (see `targets/qemu.md`
50+
for guidance).
51+
52+
The Docker images produced by our Nix build include QEMU
53+
(`qemu-system-x86_64`), `swtpm` / `libtpms`, `canokey-qemu` (a virtual
54+
OpenPGP smartcard), and other userspace tooling required to build and
55+
test QEMU boards. If you use `./docker_repro.sh` you only need Docker on
56+
the host (for example, `docker-ce`). For KVM acceleration the host
57+
must expose `/dev/kvm` (load `kvm_intel` / `kvm_amd` as appropriate);
58+
our wrapper scripts mount `/dev/kvm` automatically when it exists.
59+
60+
If you plan to manage disk images or use `qemu-img` snapshots on the
61+
host (outside containers), install the `qemu-utils` package locally
62+
(which provides `qemu-img`).
63+
64+
If you do not specify `USB_TOKEN` when running QEMU targets, the container will use the included `canokey-qemu` virtual token by default; set `USB_TOKEN` (or use `hostbus`/`hostport`/`vendorid,productid`) to forward a hardware token instead.
65+
66+
Wrapper options & environment variables
67+
---
68+
69+
- `HEADS_DISABLE_USB=1` — disable automatic USB passthrough and the
70+
automatic USB cleanup (default: `0`).
71+
- `HEADS_X11_XAUTH=1` — force mounting your `${HOME}/.Xauthority` into
72+
the container for X11 authentication.
73+
74+
For details about selecting or forwarding a physical USB token to QEMU
75+
(handled by the `USB_TOKEN` make variable), see `targets/qemu.md`.
76+
77+
Note: when USB passthrough is active the wrappers will detect processes that may be holding a USB token (for example `scdaemon` or `pcscd`). The wrapper will warn and, on interactive shells, give a 3s abort window before attempting to kill those processes to free the token. Set `HEADS_DISABLE_USB=1` to opt out of this automatic cleanup.
78+
79+
Example: `HEADS_DISABLE_USB=1 ./docker_repro.sh make BOARD=qemu-coreboot-fbwhiptail-tpm2 run`
80+
4281
Using Nix local dev environement / building docker images with Nix
4382
==
4483

@@ -88,16 +127,17 @@ Your local docker image "linuxboot/heads:dev-env" is ready to use, reproducible
88127

89128
Jump into nix develop created docker image for interactive workflow
90129
====
91-
There is 3 helpers:
92-
- `./docker_local_dev.sh`: for developers wanting to customize docker image built from flake.nix(nix devenv creation) and flake.lock (pinned versions used by flake.nix)
93-
- `./docker_latest.sh`: for Heads developers, wanting to use latest published docker images to develop Heads
94-
- `./docker_repro.sh`: versioned docker image used under CircleCI to produce reproducivle builds, both locally and under CircleCI. **Use this one if in doubt**
130+
There are three helpers:
131+
- `./docker_local_dev.sh`: developer-only — customize the local image built from `flake.nix`/`flake.lock` (not recommended for general testing)
132+
- `./docker_latest.sh`: convenience — use the latest published Docker image for development
133+
- `./docker_repro.sh`: canonical, reproducible builds that match CircleCI; **this is the recommended way to build and test Heads**
95134

96135
ie: `./docker_repro.sh` will jump into CircleCI used versioned docker image for that Heads commit id to build images reproducibly if git repo is clean (not dirty).
97136

98137
From there you can use the docker image interactively.
99138

100-
`make BOARD=board_name` where board_name is the name of the board directory under `./boards` directory.
139+
Use `./docker_repro.sh make BOARD=board_name` to run builds and tests (this runs `make` inside the canonical Docker image).
140+
If you are already inside the container interactively, run `make BOARD=board_name` as usual.
101141

102142

103143
One such useful example is to build and test qemu board roms and test them through qemu/kvm/swtpm provided in the docker image.

docker/common.sh

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
#!/bin/bash
2+
3+
# Shared common Docker helpers for Heads dev scripts
4+
# Meant to be sourced from docker_latest.sh / docker_local_dev.sh / docker_repro.sh
5+
6+
set -euo pipefail
7+
8+
usage() {
9+
cat <<'USAGE'
10+
Usage: $0 [OPTIONS] -- [COMMAND]
11+
Options:
12+
Environment variables (opt-ins / opt-outs):
13+
HEADS_DISABLE_USB=1 Disable automatic USB passthrough (default: enabled when /dev/bus/usb exists)
14+
HEADS_X11_XAUTH=1 Explicitly mount $HOME/.Xauthority into the container for X11 auth
15+
Command:
16+
The command to run inside the Docker container, e.g., make BOARD=BOARD_NAME
17+
USAGE
18+
}
19+
20+
# Track whether we supply Xauthority into the container
21+
DOCKER_XAUTH_USED=0
22+
23+
# Kill scdaemon/pcscd when USB passthrough is present (minimal, automatic)
24+
kill_usb_processes() {
25+
[ -d /dev/bus/usb ] || return 0
26+
[ "${HEADS_DISABLE_USB:-0}" = "1" ] && { echo "HEADS_DISABLE_USB=1: skipping USB cleanup" >&2; return 0; }
27+
28+
# Collect scdaemon and pcscd PIDs (simple, per-origin/master behavior)
29+
mapfile -t pids_array < <(pgrep -x scdaemon || true; pgrep -x pcscd || true)
30+
[ ${#pids_array[@]} -eq 0 ] && { [ "${HEADS_USB_VERBOSE:-0}" = "1" ] && echo "No scdaemon/pcscd running." >&2; return 0; }
31+
32+
echo "Detected scdaemon/pcscd processes: ${pids_array[*]}" >&2
33+
34+
# Warn before attempting kills. Non-interactive shells continue; interactive shells get a short abort window.
35+
echo "WARNING: About to kill the above processes to free USB devices for passthrough. To skip this automatic action set HEADS_DISABLE_USB=1 in your environment." >&2
36+
if [ -t 1 ]; then
37+
echo "Press Ctrl-C to abort within 3 seconds if you do NOT want these processes killed." >&2
38+
sleep 3
39+
fi
40+
41+
# Try to kill: prefer running as root, else try sudo without prompting in non-interactive shells
42+
if [ "$(id -u)" = "0" ]; then
43+
kill -9 "${pids_array[@]}" && echo "Killed PIDs: ${pids_array[*]}" >&2 || echo "Failed to kill some PIDs: ${pids_array[*]}" >&2
44+
elif sudo -n true 2>/dev/null; then
45+
sudo kill -9 "${pids_array[@]}" && echo "Killed PIDs: ${pids_array[*]}" >&2 || echo "Failed to kill some PIDs: ${pids_array[*]}" >&2
46+
elif [ -t 1 ]; then
47+
sudo kill -9 "${pids_array[@]}" && echo "Killed PIDs: ${pids_array[*]}" >&2 || echo "Failed to kill some PIDs: ${pids_array[*]}" >&2
48+
else
49+
echo "Non-interactive: sudo would prompt but cannot proceed; please run: sudo kill -9 ${pids_array[*]}" >&2
50+
fi
51+
}
52+
53+
# Build docker options (returns single string on stdout)
54+
build_docker_opts() {
55+
local opts=( -e "DISPLAY=${DISPLAY:-}" --network host --rm -ti )
56+
57+
# USB passthrough
58+
if [ -d "/dev/bus/usb" ] && [ "${HEADS_DISABLE_USB:-0}" != "1" ]; then
59+
opts+=( --device=/dev/bus/usb:/dev/bus/usb )
60+
echo "--->USB passthrough enabled; to disable set HEADS_DISABLE_USB=1" >&2
61+
elif [ -d "/dev/bus/usb" ]; then
62+
echo "--->Host USB present; USB passthrough disabled by HEADS_DISABLE_USB=1" >&2
63+
fi
64+
65+
# KVM passthrough
66+
if [ -e /dev/kvm ]; then
67+
opts+=( --device=/dev/kvm:/dev/kvm )
68+
echo "--->Host KVM device found; enabling /dev/kvm passthrough" >&2
69+
elif [ -e /proc/kvm ]; then
70+
echo "--->Host reports KVM available but /dev/kvm is missing; load kvm module" >&2
71+
fi
72+
73+
# X11 forwarding: mount socket and try programmatic Xauthority when possible
74+
if [ -d "/tmp/.X11-unix" ]; then
75+
opts+=( -v /tmp/.X11-unix:/tmp/.X11-unix )
76+
if command -v xauth >/dev/null 2>&1; then
77+
local XAUTH_HOST
78+
XAUTH_HOST="/tmp/.docker.xauth-$(id -u)"
79+
: >"$XAUTH_HOST" 2>/dev/null || true
80+
xauth nlist "${DISPLAY}" 2>/dev/null | sed -e 's/^..../ffff/' | xauth -f "$XAUTH_HOST" nmerge - 2>/dev/null || true
81+
if [ -s "$XAUTH_HOST" ]; then
82+
DOCKER_XAUTH_USED=1
83+
opts+=( -v "$XAUTH_HOST:$XAUTH_HOST:ro" -e "XAUTHORITY=$XAUTH_HOST" )
84+
echo "--->Using programmatic Xauthority $XAUTH_HOST for X11 auth" >&2
85+
elif [ -f "${HOME}/.Xauthority" ]; then
86+
DOCKER_XAUTH_USED=1
87+
opts+=( -v "${HOME}/.Xauthority:/root/.Xauthority:ro" -e "XAUTHORITY=/root/.Xauthority" )
88+
echo "--->Falling back to mounting ${HOME}/.Xauthority into container" >&2
89+
else
90+
echo "--->X11 socket present but no Xauthority found; GUI may fail" >&2
91+
fi
92+
else
93+
if [ -f "${HOME}/.Xauthority" ]; then
94+
opts+=( -v "${HOME}/.Xauthority:/root/.Xauthority:ro" -e "XAUTHORITY=/root/.Xauthority" )
95+
echo "--->Mounting ${HOME}/.Xauthority into container for X11 auth (xauth missing)" >&2
96+
fi
97+
fi
98+
elif [ "${HEADS_X11_XAUTH:-0}" != "0" ] && [ -f "${HOME}/.Xauthority" ]; then
99+
opts+=( -v "${HOME}/.Xauthority:/root/.Xauthority:ro" -e "XAUTHORITY=/root/.Xauthority" )
100+
echo "--->HEADS_X11_XAUTH=1: mounting ${HOME}/.Xauthority into container" >&2
101+
fi
102+
103+
# If host xhost does not list LOCAL, warn the user about enabling access only when
104+
# we did NOT supply an Xauthority cookie. We do NOT modify xhost automatically (security).
105+
if [ "${DOCKER_XAUTH_USED:-0}" = "0" ] && command -v xhost >/dev/null 2>&1 && ! xhost | grep -q "LOCAL:"; then
106+
echo "--->X11 auth may be strict; no automatic 'xhost' changes are performed. Provide Xauthority (install xauth) or run 'xhost +SI:localuser:root' manually if you accept the security risk." >&2
107+
fi
108+
109+
local joined
110+
printf -v joined "%s " "${opts[@]}"
111+
echo "${joined}"
112+
}
113+
114+
# Common run helper
115+
run_docker() {
116+
local image="$1"; shift
117+
local opts host_workdir container_workdir DOCKER_OPTS_ARRAY
118+
opts=$(build_docker_opts)
119+
# Convert the single-line opts string into an array for safe expansion
120+
read -r -a DOCKER_OPTS_ARRAY <<< "$opts"
121+
host_workdir="$(pwd)"
122+
container_workdir="${host_workdir}"
123+
124+
parts=()
125+
case "${opts}" in *"/dev/kvm"*) parts+=(KVM=on) ;; *) parts+=(KVM=off) ;; esac
126+
case "${opts}" in *"/dev/bus/usb"*) parts+=(USB=on) ;; *) parts+=(USB=off) ;; esac
127+
case "${opts}" in *"/tmp/.X11-unix"*) parts+=(X11=on) ;; *) parts+=(X11=off) ;; esac
128+
129+
echo "---> Running container with: ${parts[*]} ; mount ${host_workdir} -> ${container_workdir}" >&2
130+
echo "---> Full docker command: docker run ${DOCKER_OPTS_ARRAY[*]} -v ${host_workdir}:${container_workdir} -w ${container_workdir} ${image} -- $*" >&2
131+
132+
exec docker run "${DOCKER_OPTS_ARRAY[@]}" -v "${host_workdir}:${container_workdir}" -w "${container_workdir}" "${image}" -- "$@"
133+
}
134+
135+
trap "echo 'Script interrupted. Exiting...'; exit 1" SIGINT
136+
for arg in "$@"; do
137+
case "$arg" in -h|--help) usage; exit 0 ;; esac
138+
done
139+
140+
# Run the USB cleanup common action
141+
kill_usb_processes
142+
143+
# Informational reminder printed by each docker wrapper
144+
echo "----"
145+
echo "Usage reminder: The minimal command is 'make BOARD=XYZ', where additional options, including 'V=1' or 'CPUS=N' are optional."
146+
echo "For more advanced QEMU testing options, refer to targets/qemu.md and boards/qemu-*/*.config."
147+
echo
148+
echo "Type exit within docker image to get back to host if launched interactively!"
149+
echo "----"
150+
echo

docker_latest.sh

Lines changed: 9 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -3,56 +3,16 @@
33
# Inform the user that the latest published Docker image is being used
44
echo "Using the latest Docker image: tlaurion/heads-dev-env:latest"
55
DOCKER_IMAGE="tlaurion/heads-dev-env:latest"
6+
# Check if Docker is installed
7+
if ! command -v docker >/dev/null 2>&1; then
8+
echo "Error: Docker is not installed or not in PATH. Install Docker to use this script." >&2
9+
exit 1
10+
fi
611

7-
# Function to display usage information
8-
usage() {
9-
echo "Usage: $0 [OPTIONS] -- [COMMAND]"
10-
echo "Options:"
11-
echo " CPUS=N Set the number of CPUs"
12-
echo " V=1 Enable verbose mode"
13-
echo "Command:"
14-
echo " The command to run inside the Docker container, e.g., make BOARD=BOARD_NAME"
15-
}
16-
17-
# Function to kill GPG toolstack related processes using USB devices
18-
kill_usb_processes() {
19-
# check if scdaemon or pcscd processes are using USB devices
20-
if [ -d /dev/bus/usb ]; then
21-
if sudo lsof /dev/bus/usb/00*/0* 2>/dev/null | awk 'NR>1 {print $2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' >/dev/null; then
22-
echo "Killing GPG toolstack related processes using USB devices..."
23-
sudo lsof /dev/bus/usb/00*/0* 2>/dev/null | awk 'NR>1 {print $2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print $1}' | xargs -r sudo kill -9
24-
fi
25-
fi
26-
}
27-
28-
# Handle Ctrl-C (SIGINT) to exit gracefully
29-
trap "echo 'Script interrupted. Exiting...'; exit 1" SIGINT
30-
31-
# Check if --help or -h is provided
32-
for arg in "$@"; do
33-
if [[ "$arg" == "--help" || "$arg" == "-h" ]]; then
34-
usage
35-
exit 0
36-
fi
37-
done
38-
39-
# Kill processes using USB devices
40-
kill_usb_processes
12+
# Source shared docker helper functions
13+
source "$(dirname "$0")/docker/common.sh"
4114

42-
# Inform the user about entering the Docker container
43-
echo "----"
44-
echo "Usage reminder: The minimal command is 'make BOARD=XYZ', where additional options, including 'V=1' or 'CPUS=N' are optional."
45-
echo "For more advanced QEMU testing options, refer to targets/qemu.md and boards/qemu-*/*.config."
46-
echo
47-
echo "Type exit within docker image to get back to host if launched interactively!"
48-
echo "----"
49-
echo
5015

5116
# Execute the docker run command with the provided parameters
52-
if [ -d "/dev/bus/usb" ]; then
53-
echo "--->Launching container with access to host's USB buses (some USB devices were connected to host)..."
54-
docker run --device=/dev/bus/usb:/dev/bus/usb -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) $DOCKER_IMAGE -- "$@"
55-
else
56-
echo "--->Launching container without access to host's USB buses (no USB devices was connected to host)..."
57-
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) $DOCKER_IMAGE -- "$@"
58-
fi
17+
# Delegate to shared run_docker so all docker_* scripts share identical device/X11/KVM handling
18+
run_docker "$DOCKER_IMAGE" "$@"

docker_local_dev.sh

Lines changed: 4 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -27,67 +27,9 @@ echo "For using the latest published Docker image, refer to ./docker_latest.sh."
2727
echo "For producing reproducible builds as CircleCI, refer to ./docker_repro.sh."
2828
echo ""
2929

30-
# Function to display usage information
31-
usage() {
32-
echo "Usage: $0 [OPTIONS] -- [COMMAND]"
33-
echo "Options:"
34-
echo " CPUS=N Set the number of CPUs"
35-
echo " V=1 Enable verbose mode"
36-
echo "Command:"
37-
echo " The command to run inside the Docker container, e.g., make BOARD=BOARD_NAME"
38-
}
39-
40-
# Function to kill GPG toolstack related processes using USB devices
41-
kill_usb_processes() {
42-
# check if scdaemon or pcscd processes are using USB devices
43-
if [ -d /dev/bus/usb ]; then
44-
if sudo lsof /dev/bus/usb/00*/0* 2>/dev/null | awk 'NR>1 {print $2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' >/dev/null; then
45-
echo "Killing GPG toolstack related processes using USB devices..."
46-
sudo lsof /dev/bus/usb/00*/0* 2>/dev/null | awk 'NR>1 {print $2}' | xargs -r ps -p | grep -E 'scdaemon|pcscd' | awk '{print $1}' | xargs -r sudo kill -9
47-
fi
48-
fi
49-
}
50-
51-
# Handle Ctrl-C (SIGINT) to exit gracefully
52-
trap "echo 'Script interrupted. Exiting...'; exit 1" SIGINT
53-
54-
# Check if --help or -h is provided
55-
for arg in "$@"; do
56-
if [[ "$arg" == "--help" || "$arg" == "-h" ]]; then
57-
usage
58-
exit 0
59-
fi
60-
done
61-
62-
# Check if the git repository is dirty and if flake.nix or flake.lock are part of the uncommitted changes
63-
if [ -n "$(git status --porcelain | grep -E 'flake\.nix|flake\.lock')" ]; then
64-
echo "**Warning: Uncommitted changes detected in flake.nix or flake.lock. The Docker image will be rebuilt!**"
65-
echo "If this was not intended, please CTRL-C now, commit your changes and rerun the script."
66-
echo "Building the Docker image from flake.nix..."
67-
nix --print-build-logs --verbose develop --ignore-environment --command true
68-
nix --print-build-logs --verbose build .#dockerImage && docker load <result
69-
else
70-
echo "Git repository is clean. Using the previously built Docker image when repository was unclean and flake.nix/flake.lock changes were uncommited."
71-
sleep 1
72-
fi
73-
74-
# Kill processes using USB devices
75-
kill_usb_processes
76-
77-
# Inform the user about entering the Docker container
78-
echo "----"
79-
echo "Usage reminder: The minimal command is 'make BOARD=XYZ', where additional options, including 'V=1' or 'CPUS=N' are optional."
80-
echo "For more advanced QEMU testing options, refer to targets/qemu.md and boards/qemu-*/*.config."
81-
echo
82-
echo "Type exit within docker image to get back to host if launched interactively!"
83-
echo "----"
84-
echo
30+
# Source shared docker helper functions
31+
source "$(dirname "$0")/docker/common.sh"
8532

8633
# Execute the docker run command with the provided parameters
87-
if [ -d "/dev/bus/usb" ]; then
88-
echo "--->Launching container with access to host's USB buses (some USB devices were connected to host)..."
89-
docker run --device=/dev/bus/usb:/dev/bus/usb -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) $DOCKER_IMAGE -- "$@"
90-
else
91-
echo "--->Launching container without access to host's USB buses (no USB devices was connected to host)..."
92-
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) $DOCKER_IMAGE -- "$@"
93-
fi
34+
# Delegate to shared run_docker so all docker_* scripts share identical device/X11/KVM handling
35+
run_docker "$DOCKER_IMAGE" "$@"

0 commit comments

Comments
 (0)