Skip to content

[feat] add mirror dryrun#322

Open
ldmonster wants to merge 6 commits intomainfrom
feat/add-dryrun
Open

[feat] add mirror dryrun#322
ldmonster wants to merge 6 commits intomainfrom
feat/add-dryrun

Conversation

@ldmonster
Copy link
Copy Markdown
Collaborator

@ldmonster ldmonster commented Apr 7, 2026

Summary

Adds a --dry-run flag to d8 mirror pull that resolves versions, extracts image digests, and prints the complete list of images that would be downloaded — without actually pulling any blobs or writing bundle files. This enables fast validation of CLI flags, filters, and registry connectivity in seconds instead of the 30–60+ minutes a real pull takes.

Motivation

  • E2E tests are too slow. A full d8 mirror pull downloads tens of GBs of images. There was no way to quickly verify that flag combinations, module filters, and version resolution work correctly.
  • Operator validation. Users can verify their --source, --license, --deckhouse-tag, --modules, --no-modules, --no-security-db flags produce the expected image list before committing to a long download.
  • CI/CD pipelines. Dry-run enables cheap smoke tests that confirm registry access and version resolution without transferring data.

Usage

d8 mirror pull --dry-run /tmp/bundle \
  --source registry.deckhouse.io/deckhouse/fe \
  --license <token> \
  --deckhouse-tag v1.69.0 \
  --no-modules \
  --no-security-db

Exit code 0 on success, non-zero on errors (invalid flags, registry unreachable, etc.). Bundle directory remains empty.

Changes

Flag registration & threading

File Change
internal/mirror/cmd/pull/flags/flags.go New DryRun bool variable + --dry-run flag registered in AddFlags()
internal/mirror/cmd/pull/pull.go Wire DryRun: pullflags.DryRun into PullServiceOptions; skip computeGOSTDigests and finalCleanup when dry-run
internal/mirror/pull.go Add DryRun bool to PullServiceOptions; propagate to all 4 sub-service Options structs in NewPullService()

Service-level guards

Each service adds DryRun bool to its Options struct and inserts a guard after version/image resolution but before blob download and bundle packing:

Service File Behavior in dry-run
Platform internal/mirror/platform/platform.go Pulls release channels + installer images to tmpDir (needed to extract images_digests.json). Skips pullDeckhouseImages (the large blob download), skips generateDeckhouseReleaseManifests, gracefully handles missing images_digests.json (warn instead of error). Prints all resolved image refs (Deckhouse, ReleaseChannel, Install, InstallStandalone) then returns.
Modules internal/mirror/modules/modules.go Runs module discovery and version resolution. Skips release channel blob pull (PullImages). Prints per-module image list. Skips OCI layout post-processing (sort, aliases, packing).
Security internal/mirror/security/security.go Calls FillSecurityImages() to resolve image refs. Prints them and returns before PullImages.
Installer internal/mirror/installer/installer.go Calls FillInstallerImages() to resolve image refs. Prints them and returns before pullInstaller.

Key design decisions

  1. Platform installer IS pulled to tmpDir even in dry-run. This is intentional — images_digests.json is embedded inside the installer image and must be extracted to produce the complete image list. Only the main Deckhouse release blobs (~GBs) are skipped.

  2. No bundle output. The bundle directory is never written to in dry-run mode. GOST digest computation and final cleanup are skipped at the command level.

  3. Graceful degradation. If images_digests.json extraction fails in dry-run (e.g., stub registry), the service warns and continues rather than returning an error. The user still sees all version/channel refs that were resolved.

Test execution

# Unit tests (all use stub registry, run in ~1s)
go test -v -run "DryRun" ./internal/mirror/...

# Integration test with real registry (~80s)
D8_TEST_REGISTRY=registry.deckhouse.io/deckhouse/fe \
D8_TEST_LICENSE_TOKEN=<token> \
go test -v -run TestDryRunRealRegistry ./internal/mirror/cmd/pull/

Documentation

docs/mirror-pull-dry-run.md — CLI usage, exit codes, unit & integration test instructions, internal call tree, temp file reference.

Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
@ldmonster ldmonster self-assigned this Apr 7, 2026
- Stream images_digests.json directly from remote registry using ExtractFileFromImage (layer-by-layer), avoiding full installer image download to disk
- Skip OCI layout creation, release channel pulls, and standalone installer pulls in dry-run mode - none are needed for image list resolution
- Separate dry-run logic into dedicated path (pullDeckhousePlatformDryRun) keeping the normal pull path free of conditional checks
- Remove implicit VEX scanning from dry-run - (~319 unnecessary network calls)

Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
- Dry-run methods (pullDeckhousePlatformDryRun, extractImageDigestsFromRemote) live in platform_dryrun.go for clear separation from the normal pull path
- Grouped dry-run output with per-category headers and sorted image refs

Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Glitchy-Sheep
Glitchy-Sheep previously approved these changes Apr 8, 2026
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants