[draft] ci: add pinned Linux CI image recipe and offline validation

[pingzh]

Which issue does this PR close?

Part 1 of #5490. This PR does not close the issue; image publication and CI adoption will follow separately.

Rationale for this change

Linux Spark CI jobs repeatedly install toolchains and download the same dependencies. Before switching those jobs to a prebuilt image, we need a pinned toolchain recipe and a check that its dependency caches are sufficient without carrying forward Comet binaries or previous build outputs.

What changes are included in this PR?

• Add a Linux/amd64 pilot for Spark 4.1.3 and Zulu JDK 17, with pinned base-image digests, tool versions, Spark sources, and SBT launcher checksums.
• Build Comet and Spark in an intermediate stage, then export third-party Cargo, Maven, Coursier, SBT, and Ivy caches under /opt/comet-ci. The final image starts from the toolchain stage and excludes Comet artifacts, SNAPSHOTs, local Maven installations, credentials, and workspace build outputs.
• Add a fresh-source validator that disables networking, mounts an empty runtime home and workspace, checks dependency fingerprints, rebuilds native and JVM Comet, compiles Catalyst/SQL/Hive tests, and runs two Spark smoke suites.
• Document the build and cache contract, and add dependency-free regression tests. Existing CI workflows remain unchanged.

Publishing, scheduled refreshes, digest-based adoption, rollback, and performance measurements are outside this PR.

How are these changes tested?

• python3 dev/ci/images/test_image.py: 21 tests passed.
• Complete image build passed, including native compilation, the full Maven reactor (-DskipTests, with test classes compiled), Catalyst/SQL/Hive test compilation, and 104 Spark smoke tests: 42 in LiteralExpressionSuite and 62 in MathFunctionsSuite.
• COMET_CI_BUILD_JOBS=16 bash dev/ci/images/validate.sh: passed the same builds and all 104 smoke tests in a fresh container with --network=none, an empty runtime home, and no host dependency-cache mounts.
• Toolchain checks passed with networking disabled. ShellCheck, Ruff, Prettier, and Maven RAT checks passed.