feat!: carve the wasm pipeline into @datadog/libdatadog-wasm-pipeline

[bengl]

What

Carves the pipeline wasm crate out into a new package and leaves everything else where it is.

package	source	contents
@datadog/libdatadog	repo root	unchanged in shape: native Node-API prebuilds plus the library_config and datadog-js-zstd wasm modules
@datadog/libdatadog-wasm-pipeline	packages/wasm-pipeline/	only prebuilds/pipeline/

Versioned in lockstep, neither depends on the other, both expose the same loader API, so migrating a consumer is a package-name swap in the require.

Only the one crate we actually need separately moves. load.js at the root is untouched — it still resolves native binaries and still falls back to the wasm modules that stay in that package.

Packed contents, verified locally with npm pack --dry-run:

@datadog/libdatadog                    @datadog/libdatadog-wasm-pipeline
  index.js load.js package.json          index.js load.js package.json
  README.md LICENSE                      README.md LICENSE
  prebuilds/<platform>-<arch>/*.node     prebuilds/pipeline/pipeline_bg.wasm
  prebuilds/<platform>-<arch>/…-receiver prebuilds/pipeline/pipeline.js
  prebuilds/library_config/…             prebuilds/pipeline/pipeline.d.ts
  prebuilds/datadog-js-zstd/…            prebuilds/pipeline/snippets/…/http_transport.js

How the split is enforced

action-prebuildify's final job merges every artifact matching prebuilds-* into the single prebuilds artifact that @datadog/libdatadog publishes. That merge is load-bearing for the crates that stay — it is how library_config and datadog-js-zstd reach npm today — and fatal for a carved-out one.

So scripts/wasm-crates.js owns the crate → package mapping and derives everything from it: the wasm-pack output directory, the CI artifact name (prebuilds-wasm-<crate> for crates that stay, wasm-<crate> for carved-out ones, deliberately outside the merge pattern), and the assertions in check-packages.js. The composite action queries it rather than repeating any of it.

scripts/check-packages.js then runs npm pack --dry-run over both packages and asserts every crate is published by exactly the package that owns it, that only @datadog/libdatadog ships .node binaries, and that nothing leaks either way. It gates every PR (package-contents) and both publishes, because each failure mode here is silent.

Other changes

• scripts/build-wasm.js routes output through the mapping and deletes the .gitignore that wasm-pack writes into its output directory. npm honours nested ignore files, so that file silently drops the whole module from a locally packed tarball — this is a pre-existing bug for library_config/datadog-js-zstd too, which CI only escapes because upload-artifact excludes hidden files by default. It also now builds every crate when given no argument, so the crate list lives in one place instead of being spelled out in package.json.
• Tests split structurally: test/wasm/<crate>/ is the wasm suite, everything else is native. test/pipeline.js → test/wasm/pipeline/index.js and test/http_transport.js → test/wasm/pipeline/http_transport.js (pure renames). scripts/test.sh <native|wasm> [crate] replaces test-wasm.js; yarn test is native-only, which is what the prebuildify matrix runs. The library_config and datadog-js-zstd suites are untouched and still load through the root loader.
• The maybeLoad/skip guard at the top of the pipeline suite is gone — it only existed because the pipeline wasm was absent from the native matrix.
• scripts/check-versions.js enforces the lockstep version, in the lint job and as a release gate.
• publish-wasm-pipeline runs after publish, so a failed native build can never leave half of a lockstep pair on npm.

Verification

Everything below was run locally against real wasm-pack output for all three crates.

check	result
node scripts/build-wasm.js (no args)	prebuilds/{library_config,datadog-js-zstd} + packages/wasm-pipeline/prebuilds/pipeline, no .gitignore in any of them
bash scripts/test.sh wasm	zstd ✔, library_config ✔, http_transport.js 18/18, pipeline/index.js 51/51
check-packages.js, expected layout	pass: native = 21 files / 4 .node / [library_config, datadog-js-zstd]; pipeline = 11 files / 0 native / [pipeline]
pipeline leaked into prebuilds/	exit 1, lists all six offending paths
a .node placed in the pipeline package	exit 1
library_config dropped from the native package	exit 1, "missing the library_config wasm module"
test.sh native selection	test/process-discovery.js + test/crashtracker/index.js only
test.sh wasm [crate] / bad crate / bad mode	correct subset / exit 1 / exit 1
wasm-crates.js mapping	pipeline → packages/wasm-pipeline/prebuilds/pipeline + artifact wasm-pipeline; others → prebuilds/<crate> + prebuilds-wasm-<crate>
eslint ., check-versions.js, workflow YAML	clean

Not run locally: the native Rust build and suite. This checkout's prebuilt .node files are rejected by macOS code-signing policy, and nothing here touches Rust, Cargo.toml or copy-artifacts.js; native runner selection was verified with a stubbed node.

Before merging

• @datadog/libdatadog-wasm-pipeline must exist on npm before trusted publishing can be configured for it — npm requires the package to exist before OIDC can be enabled (npm/cli#8544). First publish needs a granular token; after that publish-wasm-pipeline works unattended.
• dd-trace-js: add @datadog/libdatadog-wasm-pipeline and repoint only the pipeline require. library_config and datadog-js-zstd keep working unchanged. Both packages are exact-versioned deps, so a single PR is an atomic cutover.