Skip to content

Add configuration-driven OpenRTB auction providers - #1016

Open
ChristianPavilonis wants to merge 7 commits into
mainfrom
docs/config-first-auction-provider-spec
Open

Add configuration-driven OpenRTB auction providers#1016
ChristianPavilonis wants to merge 7 commits into
mainfrom
docs/config-first-auction-provider-spec

Conversation

@ChristianPavilonis

@ChristianPavilonis ChristianPavilonis commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Replaces the hard-coded Prebid Server and Amazon Publisher Services auction wiring with map-shaped provider and bidder configuration compiled into one immutable auction plan.
  • Adds a shared Open Real-Time Bidding (OpenRTB) 2.6 execution path with typed Standard, Prebid Server, and Amazon Publisher Services profiles, including bidder routing, request signing, notification policy, timeout handling, and response identity tracking.
  • Uses the same validated plan across adapters, runtime routing, browser demand generation, examples, and operator documentation so these paths cannot disagree about enabled providers or bidders.

The old design tied provider identity, routing, transport, and response handling to singleton implementations. Adding another standards-compliant endpoint required more provider-specific code, and each adapter could derive backend behavior independently. This implementation moves those decisions into validated configuration while preserving existing Prebid Server and Amazon Publisher Services behavior.

Changes

Changed files

Root configuration and guidance

File Change
Cargo.lock Refreshes the lockfile after removing an obsolete adapter dependency.
README.md Points operators to the new map-shaped auction provider configuration.
TESTING.md Updates auction test setup and verification guidance for the config-first implementation.
trusted-server.example.toml Replaces the legacy provider list with provider and bidder maps, typed profiles, routes, and notification settings.

Adapters and CLI

File Change
crates/trusted-server-adapter-axum/src/app.rs Compiles and reuses the validated auction plan in Axum request handling.
crates/trusted-server-adapter-axum/src/platform.rs Adds Axum target capability and backend-name validation for configured providers.
crates/trusted-server-adapter-axum/tests/routes.rs Covers startup and route behavior with config-defined auction providers.
crates/trusted-server-adapter-cloudflare/src/app.rs Builds the auction runtime from the validated plan for Cloudflare Workers.
crates/trusted-server-adapter-cloudflare/src/platform.rs Implements deterministic provider backend naming and target validation for Cloudflare Workers.
crates/trusted-server-adapter-fastly/Cargo.toml Removes the Fastly-only dependency no longer needed by backend construction.
crates/trusted-server-adapter-fastly/src/app.rs Compiles the auction plan at startup and passes it into Fastly request handling.
crates/trusted-server-adapter-fastly/src/backend.rs Reworks dynamic backend construction around canonical provider backend specifications.
crates/trusted-server-adapter-fastly/src/platform.rs Shares Fastly backend naming, timeout canonicalization, and target capability validation with the plan compiler.
crates/trusted-server-adapter-fastly/src/tinybird.rs Reports the separated provider, returned seat, and delivery bidder identities in auction telemetry.
crates/trusted-server-adapter-spin/src/app.rs Builds the auction runtime from the validated plan for Spin.
crates/trusted-server-adapter-spin/src/platform.rs Implements deterministic provider backend naming and target validation for Spin.
crates/trusted-server-cli/src/prebid_bundle.rs Removes the legacy provider-list conversion from generated Prebid configuration.
crates/trusted-server-cli/tests/config_env_overlay.rs Covers environment overlays against the new provider and bidder maps.

Auction core

File Change
crates/trusted-server-core/src/auction/README.md Documents the plan-driven auction architecture and request flow.
crates/trusted-server-core/src/auction/endpoints.rs Adds canonical endpoint validation used by provider-plan compilation.
crates/trusted-server-core/src/auction/formats.rs Carries returned upstream seat identity separately from the browser delivery bidder.
crates/trusted-server-core/src/auction/mod.rs Admits requests into the compiled-plan flow and handles disabled or empty auctions without provider I/O.
crates/trusted-server-core/src/auction/openrtb.rs Implements the shared OpenRTB 2.6 request, signing, transport, response, and notification driver.
crates/trusted-server-core/src/auction/openrtb/test_executor.rs Provides deterministic transport support for OpenRTB driver tests.
crates/trusted-server-core/src/auction/openrtb/tests.rs Adds golden and error-path coverage for Standard, Prebid Server, and Amazon Publisher Services profiles.
crates/trusted-server-core/src/auction/orchestrator.rs Replaces singleton dispatch with plan-driven fanout, routing, logical budgets, bid collection, and mediation.
crates/trusted-server-core/src/auction/plan.rs Compiles raw configuration into a sorted, immutable plan and validates target capabilities and collisions.
crates/trusted-server-core/src/auction/profile.rs Defines the typed profile registry and compiled Standard, Prebid Server, and Amazon Publisher Services profile variants.
crates/trusted-server-core/src/auction/provider.rs Adds the generic planned provider that dispatches through the shared OpenRTB driver.
crates/trusted-server-core/src/auction/routing.rs Validates bidder ownership and builds explicit and all-eligible provider routes.
crates/trusted-server-core/src/auction/telemetry.rs Keeps provider ID, returned seat, and delivery bidder distinct in telemetry.
crates/trusted-server-core/src/auction/test_support.rs Adds reusable plan, provider, and backend fixtures for auction tests.
crates/trusted-server-core/src/auction/types.rs Adds plan-driven auction input types and the returned-seat field.

Configuration, platform, and request handling

File Change
crates/trusted-server-core/src/auction_config_types.rs Defines map-shaped provider and bidder configuration with typed profile, routing, and notification options.
crates/trusted-server-core/src/config.rs Validates the new auction schema and rejects the retired list-shaped provider configuration.
crates/trusted-server-core/src/config_payload.rs Updates configuration payload parsing and validation for provider and bidder maps.
crates/trusted-server-core/src/creative_opportunities.rs Derives browser auction opportunities from authoritative configured bidder routes.
crates/trusted-server-core/src/html_processor.rs Passes plan-derived bidder data into injected browser configuration.
crates/trusted-server-core/src/platform/backend_naming.rs Adds the shared pure backend-name codec used by startup validation and adapters.
crates/trusted-server-core/src/platform/http.rs Adds HTTP fields needed by canonical provider transport.
crates/trusted-server-core/src/platform/mod.rs Exports the shared backend naming and target validation support.
crates/trusted-server-core/src/platform/test_support.rs Extends platform fixtures for provider backend prediction and transport tests.
crates/trusted-server-core/src/platform/traits.rs Updates platform contracts for canonical backend specifications and timeout handling.
crates/trusted-server-core/src/publisher.rs Generates publisher JavaScript and auction responses from the compiled provider plan.
crates/trusted-server-core/src/settings.rs Aligns runtime settings with compiled auction configuration ownership.
crates/trusted-server-core/src/test_support.rs Makes shared test settings construct the new auction configuration.

Integrations

File Change
crates/trusted-server-core/src/integrations/adserver_mock.rs Keeps mock mediation separate from bidder-provider plans while preserving mediation behavior.
crates/trusted-server-core/src/integrations/aps.rs Converts Amazon Publisher Services into a typed OpenRTB profile while preserving request, response, renderer, and diagnostics behavior.
crates/trusted-server-core/src/integrations/didomi.rs Reads auction state through the plan-backed settings API.
crates/trusted-server-core/src/integrations/google_tag_manager.rs Reads configured browser bidders from the compiled plan.
crates/trusted-server-core/src/integrations/gpt_diagnostics.rs Updates diagnostics to use plan-backed auction state.
crates/trusted-server-core/src/integrations/mod.rs Removes the former static Prebid Server and Amazon Publisher Services provider registration path.
crates/trusted-server-core/src/integrations/nextjs/mod.rs Uses plan-derived bidder configuration when generating Next.js integration data.
crates/trusted-server-core/src/integrations/prebid.rs Converts Prebid Server into a typed OpenRTB profile and derives browser demand from validated bidder routes.
crates/trusted-server-core/src/integrations/registry.rs Stops treating auction providers as singleton integration registrations.
crates/trusted-server-core/src/integrations/sourcepoint.rs Reads auction state through the plan-backed settings API.

Browser and integration tests

File Change
crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml Migrates the integration fixture to provider and bidder maps.
crates/trusted-server-js/lib/src/integrations/prebid/index.ts Builds browser demand only for bidders authorized by the server plan.
crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts Covers authoritative bidder filtering and mixed browser/server demand.
crates/trusted-server-js/lib/test/prebid-artifact-integration.test.mjs Updates built-artifact expectations for plan-derived Prebid configuration.

Operator and architecture documentation

File Change
docs/guide/api-reference.md Documents the updated auction request and response behavior.
docs/guide/architecture.md Describes the compiled auction plan in the system architecture.
docs/guide/auction-orchestration.md Rewrites auction orchestration guidance around providers, profiles, bidder routes, and mediation.
docs/guide/configuration.md Documents the map-shaped auction schema and validation rules.
docs/guide/ec-setup-guide.md Updates examples that reference auction configuration.
docs/guide/error-reference.md Adds startup and request errors for invalid provider plans and routes.
docs/guide/fastly.md Explains Fastly backend handling for configured providers.
docs/guide/first-party-proxy.md Updates configuration references to the new schema.
docs/guide/getting-started.md Uses provider and bidder maps in setup instructions.
docs/guide/integration-guide.md Explains how integrations consume plan-backed auction state.
docs/guide/integrations-overview.md Updates the relationship between integrations and auction profiles.
docs/guide/integrations/aps.md Documents the Amazon Publisher Services profile and bidder routing configuration.
docs/guide/integrations/prebid.md Documents the Prebid Server profile and authoritative browser bidder behavior.
docs/guide/proxy-signing.md Updates signing references for the shared OpenRTB driver.
docs/superpowers/plans/2026-08-11-config-first-auction-provider-architecture-implementation-plan.md Records the implementation sequence, parity requirements, and verification gates.
docs/superpowers/specs/2026-08-10-config-first-auction-provider-architecture-design.md Records the configuration-first provider architecture and its runtime contracts.

Scope

This is a large change because provider configuration is now the single source of truth for startup validation, adapter backend registration, runtime dispatch, browser bidder exposure, telemetry, and operator documentation. Shipping only part of that path would leave the old and new models active at the same time and allow them to disagree. The pull request keeps the work focused on auction-provider configuration and execution; it does not replace the existing static mock mediator or add runtime-loadable provider plugins.

Target-aware validation before ts config push remote I/O remains blocked on publishing and pinning the required EdgeZero callback dependency. The shared target-independent compiler and adapter startup validation are included here.

Closes

Closes #1026

Test plan

Full verification still needs to run on the rebased implementation head. The current remote head has completed only the JavaScript and TypeScript CodeQL check.

  • cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin
  • ./scripts/test-cli.sh
  • cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity
  • cargo fmt --all -- --check
  • cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-cloudflare-wasm && cargo clippy-spin-native && cargo clippy-spin-wasm
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM builds for Fastly, Cloudflare, and Spin
  • CodeQL: JavaScript and TypeScript analysis

Checklist

  • Changes follow CLAUDE.md conventions
  • No unwrap() in production code; use expect("should ...")
  • Uses log macros, not println!
  • New Rust and JavaScript behavior has test coverage in this pull request
  • No secrets or credentials are intentionally included

@ChristianPavilonis ChristianPavilonis changed the title Define how auction providers are configured and routed Spec: auction provider refactor Aug 11, 2026
@ChristianPavilonis
ChristianPavilonis force-pushed the docs/config-first-auction-provider-spec branch from 8de9eab to 0c98095 Compare August 11, 2026 23:42
@ChristianPavilonis
ChristianPavilonis marked this pull request as draft August 13, 2026 15:53
@aram356 aram356 added this to the 202608 milestone Aug 17, 2026
@aram356 aram356 assigned aram356 and ChristianPavilonis and unassigned aram356 Aug 18, 2026
@aram356

aram356 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

@ChristianPavilonis to test in staging

@ChristianPavilonis
ChristianPavilonis marked this pull request as ready for review August 25, 2026 18:12
@ChristianPavilonis ChristianPavilonis changed the title Spec: auction provider refactor Add configuration-driven OpenRTB auction providers Aug 25, 2026
Make auction behavior derive from one validated provider plan so startup, runtime routing, browser demand, and platform backend handling cannot drift across adapters.\n\nPreserve existing Prebid and APS behavior while allowing multiple typed OpenRTB providers and rejecting the retired list-shaped configuration.

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Large, coherent rework: provider identity, routing, transport, and response handling all move out of singleton integrations into one immutable, validated AuctionPlan shared by every adapter. The plan compiler, profile registry, and pure backend-naming policy are well factored, and the new validation (endpoint canonicalization, backend-name collision prediction, notification seat limits, static-extension bounds) is thorough.

Two blocking issues: the required cargo test check is failing because of this PR, and the config schema break ships with no operator-facing migration note.

1 of the inline comments below carries a one-click GitHub suggestion — use Commit suggestion to apply it as a commit on the PR branch. The remaining comments describe the fix in prose because the change spans multiple files, touches lines outside the diff, or is a design question rather than a patch.

Blocking

wrench

  • Required cargo test job fails: template-cache harness configures no auction provider — see Cross-cutting below
  • Breaking config cutover with no CHANGELOG entry and no safe deploy ordering — see Cross-cutting below
  • Dead tautological assertion in the moved registration loop — see inline at crates/trusted-server-core/src/integrations/registry.rs:827

Non-blocking

thinking / refactor

  • One malformed envelope entry silently zeroes a slot's server-side demand — see inline at crates/trusted-server-core/src/auction/routing.rs:434
  • HTTPS-only endpoint canonicalization blocks loopback stub endpoints — see inline at crates/trusted-server-core/src/auction/plan.rs:570
  • #[cfg(test)] orchestrator harness re-implements the production dispatch path — see inline at crates/trusted-server-core/src/auction/orchestrator.rs:318
  • run_auction carries the same body twice under opposite cfg gates — see inline at crates/trusted-server-core/src/auction/orchestrator.rs:849
  • apply_prebid pairs imps to slots positionally — see inline at crates/trusted-server-core/src/auction/openrtb.rs:280
  • Browser shim ownership inverted: unowned bidders now fail open to client-side — see Cross-cutting below

praise

  • Response currency is finally checked — see inline at crates/trusted-server-core/src/auction/openrtb.rs:552

Cross-cutting / body-level findings

  • wrench — Required cargo test job fails: the template-cache harness no longer configures any auction provider. scripts/template-cache-local-test.sh is not touched by this PR, but the cargo test job runs it, and it patches the example config by literal string replacement:

    s = s.replace('[integrations.prebid]\nenabled = false\nserver_url = "https://prebid.example.com/openrtb2/auction"', ...)
    s = s.replace('providers = []', 'providers = ["prebid"]', 1)

    Both target strings were deleted from trusted-server.example.toml by this PR (grep -c 'providers = \[\]' and grep -c server_url both return 0), so both replacements are silent no-ops. The stub then runs with [auction] enabled = true and zero providers, so no bids are produced and 6 assertions fail in the cargo test job:

    FAIL a bids script is present — got '0', want '1'
    FAIL the seam carries slot definitions, not just bids — got '0', want '1'
    FAIL the slot definitions reach the guarded scheduler — got '0', want '1'
    FAIL the winning bid's bucketed price reaches the reader — got '0', want '1'
    FAIL the served seam failed the real GPT module contract: Error: served document has no executable seam payload
    FAIL cache hit streams: the article is delivered before the auction resolves — got 'no', want 'yes'
      15 passed, 6 failed
    

    Rewriting the replacements to the new map shape is not sufficient on its own: canonicalize_endpoint requires scheme == "https" (crates/trusted-server-core/src/auction/plan.rs:570) and the harness stub endpoint is http://127.0.0.1:{port}/bid. The previous path accepted any scheme, since server_url carried only #[validate(url)]. The harness needs either an HTTPS stub backend or an explicit loopback exemption in endpoint validation.

  • wrench — Breaking config cutover ships with no CHANGELOG entry and no safe deploy ordering. Two schema breaks land together: [auction].providers changes from a list to a map (the list shape is explicitly rejected), and PrebidIntegrationConfig is rebuilt as a browser-only config with #[serde(deny_unknown_fields)], dropping server_url, bidders, and the server-side override fields.

    IntegrationRegistry::with_plan calls prebid::register_for_plan, which does settings.integration_config::<PrebidIntegrationConfig>(PREBID_INTEGRATION_ID)?. A live config blob still carrying server_url therefore fails to parse, the error propagates out of build_state_from_settings, and the adapter comes up on the startup-error router. The reverse ordering fails too: a map-shaped blob does not parse on a binary that predates this PR. There is no deploy ordering that avoids an outage window — binary and config have to cut over together.

    CHANGELOG.md is untouched by this PR. The repo documents exactly this class of change under [Unreleased] / Changed with a Breaking marker and explicit upgrade/rollback ordering (see the sanitize_creatives and APS OpenRTB entries, both of which spell out "upgrade the binary first, then push the config" and the rollback constraint). This change needs the same treatment, including the fact that ts config push now rejects the old shape and that the previous log-and-strip tolerance for unknown bidders / client_side_bidders entries is now a hard startup error.

  • thinking — Browser shim ownership inverted: unowned bidders now fail open to client-side. In crates/trusted-server-js/lib/src/integrations/prebid/index.ts, installPrebidNpm previously folded every bidder not listed in clientSideBidders into the trustedServer envelope and stripped it from unit.bids. It now folds only codes present in serverSideBidders (that is, [auction.bidders]) and leaves everything else in browser demand:

    unit.bids = unit.bids.filter(
      (bid) => bid?.bidder === ADAPTER_CODE || !serverSideBidders.has(bid?.bidder ?? '')
    );

    An operator who upgrades the binary and pushes a config without populating [auction.bidders] silently moves all demand from the server-side auction to direct browser SSP calls. There is no error and no warning; validate_browser_bidder_ownership only rejects codes claimed by both sides at once. Worth calling out explicitly in the migration note above, since it is a first-party-proxying regression that will not show up as a failure anywhere.

CI Status

  • cargo test: FAIL (required)
  • cargo fmt: PASS (required)
  • format-typescript: PASS (required)
  • format-docs: PASS (required)
  • cargo test (axum native): PASS
  • cargo test (cloudflare native): not reported separately; cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • cargo test (cross-adapter parity): PASS
  • cargo test (ts CLI, native): PASS
  • vitest: PASS
  • integration tests: PASS
  • integration tests (Fastly EC lifecycle): PASS
  • browser integration tests: PASS
  • prepare integration artifacts: PASS
  • CodeQL: PASS
  • Analyze (rust): PASS
  • Analyze (javascript-typescript): PASS (reported twice, from two workflow runs)
  • Analyze (actions): PASS

The suggestion in this review was applied in an isolated worktree at ba2eea6 and verified against the full gate: cargo fmt --all -- --check, all six clippy aliases, cargo test-fastly / test-axum / test-cloudflare / test-spin, and the cross-adapter parity suite — all pass, with no drift between the approved bytes and the post-verification tree.

Comment thread crates/trusted-server-core/src/integrations/registry.rs Outdated
Comment thread crates/trusted-server-core/src/auction/routing.rs
Comment thread crates/trusted-server-core/src/auction/plan.rs
Comment thread crates/trusted-server-core/src/auction/orchestrator.rs
Comment thread crates/trusted-server-core/src/auction/orchestrator.rs
Comment thread crates/trusted-server-core/src/auction/openrtb.rs
Comment thread crates/trusted-server-core/src/auction/openrtb.rs
@ChristianPavilonis

Copy link
Copy Markdown
Collaborator Author

Addressed the requested changes in 35e1897:

  • migrated the template-cache harness to the provider-map schema and a trusted local HTTPS bidder;
  • made harness config rewrites strict, bounded network waits, and added a CI job timeout;
  • documented the breaking, atomic binary/config migration and rollback requirements;
  • removed the tautological registry assertion;
  • deduplicated plan-backed run_auction handling;
  • asserted and tested Prebid impression-to-slot parameter pairing.

I deliberately retained fail-closed malformed-envelope handling, HTTPS-only provider endpoints, and configured browser/server bidder ownership. The broader AuctionOrchestratorHarness migration remains follow-up work.

Validation passed across Fastly, Axum, Cloudflare, Spin, CLI, parity, clippy, JS, formatting, and both template-cache harness modes. All inline threads have replies and are resolved.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Large, disciplined rework: provider identity, routing, transport, and response handling move out of singleton integrations into one immutable, validated AuctionPlan shared by every adapter, with a common OpenRTB 2.6 driver and typed Standard / Prebid Server / APS profiles. The plan compiler and backend-naming consolidation are strong (deterministic ordering, strict validation that runs even when the auction is disabled, byte-stable Fastly backend names, unusually thorough tests). The blocking items are concentrated in three places: the all_eligible routing mode is only half-integrated for the prebid-server profile, the planned parsers add silent bid-admission rules under a parity claim, and the rewritten environment-override documentation teaches mechanisms the pinned EdgeZero overlay rejects.

16 of the inline comments below carry a one-click GitHub suggestion; use Commit suggestion (or Add suggestion to batch) to apply them. Every suggestion was verified in a scratch worktree: applied in isolation (rustfmt, target-matched clippy, adapter checks, prettier) and then all together against the full CI gate (all six clippy aliases, all four adapter test suites, the cross-adapter parity suite, vitest, and the docs/JS format checks). The remaining comments describe fixes in prose because they span multiple files or lines outside the diff.

Blocking

wrench

  • all_eligible is half-integrated for the prebid-server profile (cross-cutting, details below)
  • Array/table env-override docs do not work under the EdgeZero overlay - see inline at docs/guide/configuration.md:174-191, docs/guide/configuration.md:1282, docs/guide/integrations/prebid.md:313, docs/guide/integrations-overview.md:346, docs/guide/error-reference.md:129-130
  • AuctionProvider trait snippet documents a trait that does not exist - see inline at docs/guide/auction-orchestration.md:330
  • Failed CI check: CodeQL (cross-cutting, details below)

question

  • Planned PBS parser silently drops bids the legacy path delivered - see inline at crates/trusted-server-core/src/integrations/prebid.rs:2119
  • Unowned page bidders vanish from refresh auctions - see inline at crates/trusted-server-js/lib/src/integrations/prebid/index.ts:1238
  • APS enable/disable semantics inverted - see inline at crates/trusted-server-core/src/integrations/aps.rs:1878
  • All-providers-failed auctions now return HTTP success - see inline at crates/trusted-server-core/src/auction/orchestrator.rs:1773

Non-blocking

thinking / refactor / nitpick / note / seedling

  • Dead PII-retaining header snapshot in planned dispatch - see inline at crates/trusted-server-core/src/auction/orchestrator.rs:1788 (suggestion)
  • Harness mislabels select-failure drains as timeouts - see inline at crates/trusted-server-core/src/auction/orchestrator.rs:529
  • Standard-profile extraction requires optional w/h - see inline at crates/trusted-server-core/src/auction/openrtb.rs:619
  • Disabled-auction path emits no telemetry - see inline at crates/trusted-server-core/src/auction/endpoints.rs:182
  • dnt: Option<bool> can never be Some(false) - see inline at crates/trusted-server-core/src/auction/routing.rs:296
  • Direct slot demand bypasses MAX_BIDDER_ENTRIES - see inline at crates/trusted-server-core/src/auction/routing.rs:438
  • Injected-config escaping weaker than the bids-script escaper - see inline at crates/trusted-server-core/src/integrations/prebid.rs:1340
  • Commented-out tests dropped live coverage - see inline at crates/trusted-server-core/src/integrations/prebid.rs:4223
  • Explicit null renderer carrier poisons APS bids - see inline at crates/trusted-server-js/lib/src/integrations/prebid/index.ts:1022 (suggestion)
  • Suggestions for smaller items inline at: plan.rs:101, plan.rs:593, profile.rs:211, openrtb.rs:35, openrtb.rs:469, openrtb/test_executor.rs:44, openrtb/tests.rs:261, backend.rs:209 (fastly), registry.rs:822, settings.rs:181, docs/guide/api-reference.md:88

Cross-cutting / body-level findings

  • 🔧 all_eligible is half-integrated for the prebid-server profile. Three symptoms, one root cause. (1) A PBS provider with routing = "all_eligible" and browser bidderParams for a bidder with no [auction.bidders] route emits imp.ext = {"prebid":{}} (openrtb.rs:299-308; the golden at openrtb/tests.rs:740 pins exactly this shape). Prebid Server rejects any imp lacking a bidder or stored request, so one such imp 400s the entire multi-slot request and slot-level demand elsewhere in the auction is lost, with only an internal unroutable-bidder counter as a trace. (2) plan.browser_bidder_codes() returns only explicit route keys (plan.rs:495-497), so an all-eligible PBS provider injects "serverSideBidders":[] and the browser cannot suppress its own client-side requests for those bidders. (3) validate_browser_bidder_ownership draws from the same source and is blind to double-ownership for such providers. Recommended fix: reject routing = "all_eligible" for the prebid-server profile at plan compile time unless every browser bidder is mapped, or wire all three surfaces to include all-eligible providers. Changing the imp-construction fallback alone would require a deliberate golden change.
  • 🔧 Failed CI check: CodeQL, "6 new alerts including 6 high severity" (not in the required set). All six are rust/cleartext-logging in orchestrator.rs (lines 752, 1209, 1897, 1970, 2021, 2344) and all six are taint-analysis false positives: CodeQL taints the entire Settings/RuntimeServices object graph because construction touches validate_tinybird_secret / validate_admin_handler_passwords / secret-store readers, but the flagged statements log only provider IDs, backend names, and u32 timeout/count values; five of the six are in #[cfg(test)]-only or production-unreachable code, and the sixth logs a mediator ID plus four budgets. No statement in the changed files logs secret material. The file already carries lgtm[rust/cleartext-logging] suppressions inherited from main (new lines 1391, 1872) with the same reasoning; adding the identical two-line suppression at the six flagged statements keeps the check green without behavior change.
  • ♻️ Un-migrated runtime blobs fail startup with an unactionable error. Every pre-PR blob serializes auction.providers as an array, so during the unavoidable binary/blob mismatch window of a rolling migration every service prints Failed to deserialize JSON configuration: invalid type: sequence, expected a map with no field path and no pointer to the migration (settings.rs:2688-2695; reproduced empirically). Wrap the deserialization in serde_path_to_error (yields auction.providers: ...) or pre-check for the old array shape and return a targeted "config uses the removed provider-list schema; re-push after migrating" error. The TOML path already produces precise errors.
  • 🤔 has_enforceable_total_request_deadline lives in two unlinked places: the target descriptor (backend_naming.rs:227,242-244, consumed only by tests) and the PlatformHttpClient trait default (http.rs:299, the runtime consumer at orchestrator.rs:250). Fastly and Axum have lockstep tests; Cloudflare and Spin assert only the descriptor half because their clients are cfg-gated out of native tests. This is precisely the plan-vs-runtime drift shape the PR exists to close for backend names. Derive one side from the other, or at minimum comment the descriptor field naming the trait method that must stay in lockstep.
  • 🤔 GPC is detected but never transmitted. has_data includes consent.gpc (openrtb.rs:418-422), but RegsExt has no gpc field, so a gpc-only context emits regs: {"gdpr":0} and the signal is dropped; downstream bidders cannot honor an opt-out they never receive. If this is deliberate PBS-parity, document it at the parity comment; otherwise add gpc to regs.ext (IAB convention) in a follow-up.
  • 🤔 Literal seat "unknown" diverges across paths. The planned PBS parser keeps returned_seat = Some("unknown") for a literal seat (prebid.rs:2059-2064, test-pinned), while the mock-mediator restore treats "unknown" as absent (adserver_mock.rs:315-318). NotificationConfig.suppress_seats matches exact returned seats, so the same upstream seat suppresses notifications on the direct path but not after mediation. Align the mock restore with the direct-path semantics.
  • 🤔 TESTING.md debugging excerpts reference log lines that do not exist: "Using auction orchestrator" and "Requesting bids from: prebid" match nothing in the tree, and "Registering auction provider: prebid" now emits configured provider IDs (pbs-main), not profile names (TESTING.md:48, 133-150). Operators grepping for these strings will conclude the orchestrator is not running.
  • 📝 crates/trusted-server-core/src/auction/README.md top half is stale: the request-flow boxes cite mod.rs:149/mod.rs:274-322 (mod.rs is 233 lines and contains only plan compilation; parsing lives in endpoints.rs), and the route table says routes are defined in the Fastly main.rs with line numbers (the dispatch lives in app.rs). The rewritten bottom half is accurate; the top half should reference symbols, not dead line anchors.
  • 📝 docs/guide/auction-orchestration.md:913 still claims raw PBS debug.httpcalls requires [integrations.prebid].debug; after this PR that is the provider's profile_config.debug, and [integrations.prebid].debug is browser-only (error-reference.md:294 states this correctly). Direct contradiction of the PR's ownership split in the section operators will follow.
  • 📝 docs/guide/integrations/prebid.md:339-351 zone-override walkthrough still illustrates with kargo / client_side_abc / _s2sHeaderPlacement, values that appear nowhere in the rewritten example above it (example-server / example-header-placement).
  • 📝 Stale rollback-compatibility rationale: auction_config_types.rs:37-40 and trusted-server.example.toml:154 still say omitting rewrite_creatives keeps blobs rollback-safe, but the new always-serialized bidders field makes every re-pushed blob incompatible with the old binary regardless (the PR deleted default_auction_payload_is_accepted_by_legacy_schema and documents wholesale rollback). Rewrite both comments to the post-migration truth. Similarly, auction_config_types.rs:1 still explains the file split with a build.rs path-inclusion rationale that no longer holds (no build.rs references the file, and the new pub use would not compile under path inclusion).
  • ♻️ Test boilerplate: roughly 40 sites across publisher.rs, html_processor.rs, and the integration test modules hand-roll the identical 8-line IntegrationRegistry::with_plan(..., Arc::new(compile_auction_plan(...))) block, even though cfg(test) IntegrationRegistry::new exists for exactly this. A shared helper collapses the next plan-signature change from a 40-site edit to one.
  • 🏕 Legacy cfg(test) parity copies should be scheduled for deletion: the planned and legacy implementations of the APS debug headers, renderer, and both response parsers are hand-maintained near-duplicates (~1500 test-only lines across aps.rs and prebid.rs), and nothing forces the frozen legacy copy to track future planned-path fixes, so parity tests can keep passing while asserting stale behavior. Fine as a transition; please leave a tracking issue.
  • 📝 head_inserts_for_plan takes &self plus a browser_config parameter that is always self.config (prebid.rs:850-861, body never reads self); one of the two should go. The cfg-gated fallback body of head_inserts (lines 1361-1404) is unreachable in production and is a readability trap.
  • Hygiene sweep (no inline comments to keep the set focused): missing "should ..." assertion messages in new tests (auction/mod.rs:142-143,165, axum tests/routes.rs:100, fastly platform.rs:893-902, fastly backend.rs:864-865, creative_opportunities.rs:2000-2002, publisher.rs:13000, prebid.rs:4830-4838); expect() messages with mid-sentence "should" (config.rs:453,461, config_payload.rs:207); test imports stranded below test fns (adapter-cloudflare/src/platform.rs:775-776, adapter-spin/src/platform.rs:800-802); std::iter::IntoIterator::into_iter([...]) instead of [...].into_iter() (adapter-cloudflare/src/app.rs:701,745, adapter-spin/src/app.rs:877); APS planned parser uses the literal "http_status" where PBS uses ERROR_TYPE_HTTP_STATUS (aps.rs:921-923); doc comments bisected by #[cfg(test)] attributes (orchestrator.rs:926-932, 1096-1100); scripts/template-cache-local-test.sh builds relative to the caller's cwd instead of $REPO_ROOT and its preflight omits python3/lsof/curl; raw byte-string JSON bodies where json! is the convention (prebid.rs:8414,8460); two path spellings for the same re-exported type in publisher tests (publisher.rs:8795 vs 19555); backend-naming lost the length-arithmetic and ASCII-truncation comments the adapter versions carried (backend_naming.rs:15-17, 407-418), and predict_no_registration's injectivity silently depends on the ProviderId charset, worth a comment.
  • 🌱 Coverage pins worth adding: a single-provider plan accepted by validate_for_target on Cloudflare/Spin (the >1 boundary is tested only from above); invalid provider config still failing compile when auction.enabled = false (holds today only by code ordering in mod.rs); the starts_with("missing field ") tolerance at settings.rs:191 exercised with an integration that has genuinely required fields (testlight); both existing tests use all-default structs and never reach the branch, and the serde error-string coupling is otherwise unpinned. Also: AuctionTargetId::from_adapter_id has no production callers (module doc promises CLI validation that does not reference it), and provider/bidder maps have no sanity cap outside the Fastly backend budget (a fat-fingered 10,000-provider config compiles on Axum and clones the common request per provider at request time).
  • 📌 Downstream note: AuctionResponse.provider now carries operator-chosen provider IDs (pbs-main) instead of the literals prebid/aps on the planned path. Bid.bidder semantics are preserved (GAM hb_bidder targeting and renderer selection are safe), but anything matching response.provider == "prebid" must be re-verified.

CI Status

  • browser integration tests: PASS
  • integration tests (Fastly EC lifecycle): PASS
  • integration tests: PASS
  • CodeQL: FAIL (6 high rust/cleartext-logging alerts; all six verified as taint-analysis false positives, see the cross-cutting finding above)
  • Analyze (rust): PASS
  • format-typescript: PASS (required)
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo test: PASS (required)
  • Analyze (actions): PASS
  • cargo test (axum native): PASS
  • cargo fmt: PASS (required)
  • format-docs: PASS (required)
  • Analyze (javascript-typescript): PASS
  • prepare integration artifacts: PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • cargo test (cross-adapter parity): PASS
  • cargo test (ts CLI, native): PASS
  • vitest: PASS

Comment on lines +2119 to +2128
fn planned_prebid_bid_is_allowed(bid: &AuctionBid, input: &ProviderAuctionInput) -> bool {
input.slots().iter().any(|slot| {
slot.slot().id == bid.slot_id
&& slot
.slot()
.formats
.iter()
.any(|format| (format.width, format.height) == (bid.width, bid.height))
})
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: The planned PBS parser silently drops bids the legacy path delivered. planned_prebid_bid_is_allowed requires an exact (w, h) match against a requested format, but bid.w/bid.h are optional in OpenRTB (adapters commonly omit them when the creative matches the requested size); such bids default to 0x0 in parse_planned_prebid_bid and are discarded with no log line and no metadata (the Ok(_) => {} arm records nothing). The negative-price filter and the response-level currency gate at lines 2046-2057 are also new relative to legacy parse_bid, which accepted every parsed bid. That is direct revenue loss on responses that previously rendered, and it contradicts the PR's parity claim.

Please confirm this hardening is intended, and either way: (a) when the routed slot has exactly one requested format, default missing w/h to it instead of 0x0; (b) count dropped bids in response metadata the way the planned APS parser counts drop_reasons, so operators can see rejection volume. Apply manually; the fix spans the parser and the gate and cannot be expressed as a single-range suggestion.

Comment on lines 1238 to 1241
unit.bids = unit.bids.filter(
(bid) => bid?.bidder === ADAPTER_CODE || clientSideBidders.has(bid?.bidder ?? '')
(bid) => bid?.bidder === ADAPTER_CODE || !serverSideBidders.has(bid?.bidder ?? '')
);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: A page bidder owned by neither serverSideBidders nor clientSideBidders participates in the initial auction (this filter deliberately keeps it) but silently vanishes from every refresh auction: capturePublisherAdUnitSnapshot (line 690) snapshots an unowned bid only when it is in clientSideBidders, and the synthetic refresh unit (lines 1472-1475) re-attaches only the TS adapter plus clientSideBidsForRefresh, which again filters to configured client-side bidders. On main the bidder was server-folded, so its params survived refresh.

This is the exact migration mistake an operator makes when moving to [auction.bidders] and forgetting a route, and with an empty plan it applies to every publisher bidder. Which semantics are intended? At minimum, treat unowned bidders as client-side for snapshot/refresh purposes, or emit a one-time log.warn naming page bidders owned by neither list. Apply manually; the fix touches the snapshot capture and the refresh construction.

Comment on lines +1878 to +1881
let rendering_mode = settings
.integration_config::<ApsConfig>(APS_INTEGRATION_ID)?
.map(|config| config.rendering_mode)
.unwrap_or_default();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: APS enable/disable semantics changed here. Registration now gates only on plan.has_profile("aps"); integration_config returns None both when the [integrations.aps] block is absent and when it carries enabled = false, and unwrap_or_default() yields ApsRenderingMode::TrustedServer, which also registers the renderer proxy route. Two consequences: (a) with an APS provider in the plan, enabled = false no longer suppresses the head injector or the /integrations/aps/renderer route as it did on main; (b) a disabled block's explicit rendering_mode = "publisher_native" is silently ignored and the mode reverts to trusted_server, changing which route and script attributes ship. Both are silent and untested, and the doc comment above ("when APS is enabled") no longer matches the behavior.

Please either require an enabled (or at least present) browser config in addition to the plan profile, or error when a disabled block carries a non-default rendering_mode; and update the doc comment.

Comment on lines 174 to 191
**Array Field (JSON)**:

```bash
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS='["kargo","rubicon"]'
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS='["example-browser-a","example-browser-b"]'
```

**Array Field (Indexed)**:

```bash
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS__0=kargo
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS__1=rubicon
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS__0=example-browser-a
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS__1=example-browser-b
```

**Array Field (Comma-Separated)**:

```bash
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS=kargo,rubicon,appnexus
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS=example-browser-a,example-browser-b
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench: All three array override forms documented here do not work under the pinned EdgeZero v0.0.4 overlay. coerce_env_value hard-errors on array leaves ("env overlay supports scalar leaves only"), and walk_and_overlay only descends tables, so the __0/__1 indexed segments never match anything. An operator following these examples gets a hard ts config validate failure (JSON and comma forms) or a silent no-op (indexed form). The runtime's legacy config-crate env source that did support these shapes is #[cfg(test)]-only.

Suggested change
**Array Field (JSON)**:
```bash
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS='["kargo","rubicon"]'
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS='["example-browser-a","example-browser-b"]'
```
**Array Field (Indexed)**:
```bash
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS__0=kargo
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS__1=rubicon
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS__0=example-browser-a
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS__1=example-browser-b
```
**Array Field (Comma-Separated)**:
```bash
TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS=kargo,rubicon,appnexus
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS=example-browser-a,example-browser-b
```
**Arrays and tables**: the environment overlay supports scalar leaves only.
Array values such as `client_side_bidders` and table values such as
`profile_config` cannot be overridden through environment variables; edit them
in `trusted-server.toml` and re-push the configuration. Provider IDs containing
hyphens (for example `PBS-MAIN`) are not valid POSIX shell identifiers; set
them via `env 'NAME=value' ts config validate` or your deploy environment.

TRUSTED_SERVER__INTEGRATIONS__PREBID__TIMEOUT_MS=1000
TRUSTED_SERVER__INTEGRATIONS__PREBID__CLIENT_SIDE_BIDDERS=example-browser
TRUSTED_SERVER__AUCTION__PROVIDERS__PBS-MAIN__ENDPOINT=https://prebid.example.com/openrtb2/auction
TRUSTED_SERVER__AUCTION__PROVIDERS__PBS-MAIN__PROFILE_CONFIG='{"debug":false,"test_mode":false,"consent_forwarding":"both"}'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench: This PROFILE_CONFIG='{...}' example targets a TOML table. The EdgeZero overlay's walk_and_overlay recurses into tables and never looks up an env var for the table itself, so this override is silently ignored: the worst failure mode for an operator. Replace it with a working scalar-leaf example, for instance TRUSTED_SERVER__AUCTION__PROVIDERS__PBS-MAIN__PROFILE_CONFIG__DEBUG=true (works because debug is a boolean scalar under the profile_config table). Apply manually; the surrounding block also needs its prose adjusted.

Comment on lines +182 to +193
if !orchestrator.is_enabled() {
log::info!("/auction: auction is disabled; returning no-bid response");
let auction_request = convert_tsjs_to_auction_request(
&body,
settings,
services,
&http_req,
consent_context,
ec_id,
None,
)?;
let empty_result = OrchestrationResult {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking: This disabled-auction path emits no telemetry, while the consent gate directly below emits AuctionTerminalOutcome::Skipped { reason: "consent_denied" }. An operator who flips auction.enabled = false (or misdeploys config) sees /auction traffic vanish from auction telemetry entirely, exactly when the event stream should explain the no-bids. Mirror the consent path with a Skipped { reason: "auction_disabled" } event. The two duplicated empty-OrchestrationResult blocks here and at lines 246-258 could share a small helper at the same time.

Comment on lines +296 to +301
let dnt = inbound_request
.headers()
.get("dnt")
.and_then(|value| value.to_str().ok())
.is_some_and(|value| value.trim() == "1")
.then_some(true);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking: dnt is typed Option<bool> but .is_some_and(|value| value.trim() == "1").then_some(true) makes Some(false) unreachable: an explicit DNT: 0 and a missing header both produce None. OpenRTB distinguishes 0 (tracking allowed, explicitly) from absent (unknown). Either make the type honest with .map(|value| value.trim() == "1") (yielding Some(false) for explicit non-1 values) or change the field to bool and document the collapse.

Comment on lines +438 to +454
let mut direct = bidders
.iter()
.filter(|(key, _)| key.as_str() != TRUSTED_SERVER_ENVELOPE)
.collect::<Vec<_>>();
direct.sort_by_key(|(left, _)| *left);
for (raw_bidder, params) in direct {
let Ok(bidder) = raw_bidder.parse::<BidderId>() else {
diagnostics.record_malformed_direct_demand();
continue;
};
if bidder.as_str() == TRUSTED_SERVER_ENVELOPE || !is_usable_params(params) {
diagnostics.record_malformed_direct_demand();
continue;
}
demand.bidder_params.insert(bidder, params.clone());
}
demand

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking: normalize_envelope enforces MAX_BIDDER_ENTRIES (128) on the envelope path, but this direct loop iterates every non-envelope key of the browser-supplied map with no count limit; each valid-format entry is cloned into demand.bidder_params before the routability check drops unknowns. The 256 KiB body cap bounds the damage, so this is transient allocation rather than a real DoS, but the asymmetry with the envelope's documented admission bound is unexplained. Apply the same cap here, or add a comment stating the direct path is intentionally bounded only by the body cap.

Comment on lines +1340 to +1348
fn serialize_injected_prebid_config(payload: &impl Serialize) -> String {
// Escape `</` to prevent breaking out of the script tag.
serde_json::to_string(payload)
.unwrap_or_else(|error| {
log::warn!("Prebid: failed to serialize client config: {error}");
"{}".to_string()
})
.replace("</", "<\\/")
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking: Hardening: BidderId admits almost any non-control string (for example x<!--<script), and this serializer only escapes </. Quotes are JSON-escaped and </script> is neutralized, so there is no direct breakout, but an id containing <!-- followed by <script puts the HTML parser into the script-data-double-escaped state, and the genuine </script> no longer closes the element. Operator config is trusted, so this is defense-in-depth, but the PR's own creative_opportunities test asserts this boundary is strict while the id charset and escaping are not. Either tighten BidderId to a Prebid-code-safe charset (compare ProviderId's ^[a-z][a-z0-9-]{0,62}$) or escape like html_escape_for_script in publisher.rs (a global < to \u003c replacement is safe in JSON string output).

);
}

/* Browser-only Prebid permits no managed external bundle URL.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactor: This commented-out block dropped live regression coverage: the assertion it carried (missing external_bundle_url on an enabled prebid config is rejected) is still true of the new path (validate_external_bundle_url_allowed, lines 743-752, errors "external_bundle_url is required when prebid is enabled"), so the test could have been ported to with_plan instead of disabled, and the header claiming the requirement no longer exists contradicts the code. Same pattern at lines 3785 and 4584: please port or delete with rationale rather than leaving /* ... */ blocks.

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Follow-up pass on 1e737a8b, scoped to what the existing reviews on this head do not already cover: the new shared backend-naming codec, cross-adapter naming parity, the CI-executed shell harness, and the browser-exposed surface. The plan/OpenRTB/orchestrator/integration/doc findings from the standing CHANGES_REQUESTED review are not repeated here.

No inline comment below carries a one-click suggestion block; each proposed change is given as a plain fenced block to apply manually.

Coverage note, stated plainly: this pass verified platform/backend_naming.rs, the four adapter naming paths, scripts/template-cache-local-test.sh, .github/workflows/test.yml, CHANGELOG.md, and the plan-to-browser data path (html_processor.rs, creative_opportunities.rs, AuctionPlan::browser_bidder_codes). It did not independently re-review orchestrator.rs, openrtb.rs, integrations/prebid.rs, integrations/aps.rs, or the operator documentation.

Non-blocking

thinking

  • Axum backend naming collapses -, ., and space, so provider sets valid on every other target fail startup on Axum — see inline at crates/trusted-server-core/src/platform/backend_naming.rs:424

refactor

  • The CI-gating template-cache harness hand-reimplements the Fastly backend-name codec in Python — see inline at scripts/template-cache-local-test.sh:337

note

  • The production dispatch loop lost the lgtm[rust/cleartext-logging] suppression its cfg(test) twin kept — see inline at crates/trusted-server-core/src/auction/orchestrator.rs:1154

praise

  • The breaking-change entry is unusually complete — see inline at CHANGELOG.md:12

Cross-cutting / body-level findings

  • 📝 The six CodeQL rust/cleartext-logging alerts are false positives, independently confirmed. Reading all six flagged statements at orchestrator.rs:752, 1209, 1897, 1970, 2021, and 2344: they log provider IDs, backend names, and u32 timeout and count values only. provider_name() returns the operator-chosen provider ID from the compiled plan, and no interpolated argument carries secret material. This matches the conclusion already recorded on this PR; noting it here only as a second, independent verification so the failing check can be dispositioned rather than re-litigated.
  • 👍 The plan-to-browser data path is clean. AuctionPlan::browser_bidder_codes (plan.rs:495-497) yields bidder route keys and nothing else, so no provider endpoint, profile_config value, credential, or predicted backend name reaches injected browser configuration through it. The html_processor.rs changes in this PR are confined to #[cfg(test)] registry construction, and creative_opportunities.rs feeds AdSlot values into the server-side request rather than the page. Worth stating explicitly given that the PR moves provider identity into configuration that several browser-facing surfaces now read from.

CI Status

  • browser integration tests: PASS
  • integration tests (Fastly EC lifecycle): PASS
  • integration tests: PASS
  • CodeQL: FAIL — 6 high-severity rust/cleartext-logging alerts; independently verified as false positives (see above). Not in the branch-protection required set.
  • Analyze (rust): PASS
  • Analyze (actions): PASS
  • Analyze (javascript-typescript): PASS
  • format-typescript: PASS (required)
  • format-docs: PASS (required)
  • cargo fmt: PASS (required)
  • cargo test: PASS (required)
  • cargo test (axum native): PASS
  • cargo test (cross-adapter parity): PASS
  • cargo test (ts CLI, native): PASS
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • prepare integration artifacts: PASS
  • vitest: PASS

}

fn normalize_axum_segment(value: &str) -> String {
value.to_uppercase().replace(['-', '.', ' '], "_")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinkingnormalize_axum_segment uppercases and maps -, ., and space to _, and the Axum name is the only one of the four that has neither a disambiguating digest nor a raw discriminator. predict_fastly appends a SHA-256 digest over a length-prefixed canonical spec, and predict_no_registration (Cloudflare and Spin) interpolates the discriminator verbatim as _p_{value}. Only Axum normalizes the discriminator lossily.

ProviderId accepts any non-control UTF-8 string up to MAX_ID_BYTES with no surrounding whitespace, so -, ., space, and mixed case are all legal in a provider ID. Two providers named pbs-main and pbs.main, pointing at the same host and port, therefore both predict HTTPS_<HOST>_443_p_PBS_MAIN on Axum. AuctionPlan::validate_for_target inserts predictions into a BTreeMap and errors on the second insert (plan.rs:456-462), so that configuration fails startup on Axum while Fastly accepts it (distinct digests) and Cloudflare and Spin accept it (distinct raw discriminators). pbs-main and PBS-MAIN collide the same way.

The failure is closed and the error message is clear, so this is not a correctness or security problem. Flagging it because it is precisely the class of cross-adapter disagreement the plan compiler exists to eliminate: an operator whose production Fastly config compiles cannot boot the Axum dev server against the same blob, and the error names the Axum-normalized string rather than the provider IDs that actually differ.

Two ways to close it, both fine:

// Option A — give the Axum name the same disambiguation the other targets have,
// so no two distinct specs can normalize onto one name.
fn normalize_axum_segment(value: &str) -> String {
    value.to_uppercase().replace(['-', '.', ' '], "_")
}
// ... and in predict_axum, append a short digest of the canonical spec, mirroring
// predict_fastly's `fastly_spec_digest(&fastly_canonical_spec(spec, port))`.

Option B, if the lossy Axum segment is deliberate because the name has to stay usable as an environment-variable segment: reject the collision uniformly at compile time rather than per target, so a provider pair that cannot be represented on every supported target is refused everywhere instead of only where it happens to collide.

Either way, a regression test pinning that pbs-main and pbs.main are accepted or rejected consistently across all four targets would keep the two halves from drifting again.


# The application registers provider backends dynamically. Pre-register the exact
# deterministic name so Viceroy reuses a local backend that trusts the temporary CA.
python3 - "$WORK/fastly.toml" "$WORK/ca-cert.pem" "$BID_PORT" <<'PYEOF'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactor — This block is a fourth hand-maintained implementation of the Fastly backend-name codec, in a different language from the other three. It replicates, by hand:

  • push_field's length-prefix framing, as f"{len(value)}:{value}"
  • fastly_canonical_spec's exact field order and its n/s presence markers for the Host override and discriminator
  • fastly_spec_digest's 32-hex-character truncation of the SHA-256 digest
  • the readable prefix format from predict_fastly

The PR's central argument is that backend naming had to become one shared pure codec so startup validation and every adapter agree. This script reintroduces the drift on the test side, and it is not optional: .github/workflows/test.yml:63 and :66 run it in both esi and inline modes as part of the cargo test job, which is a required check.

It also matches only by a coincidence that is not stated anywhere. The name embeds fb5000_bb5000, but the runtime name is built from canonicalize_transport_timeout_ms(remaining_ms, provider.timeout_ms), not from the configured value. It resolves to exactly 5000 here only because the harness sets the provider timeout_ms = 5000 under [auction] timeout_ms = 10000, so remaining_ms >= configured_ms returns the configured value unchanged. Lower the auction timeout below 5000 and the runtime name quantizes to a different bucket.

When either the codec or those two timeouts change, the pre-registered Viceroy backend stops matching, the application falls back to a dynamic backend that does not trust the temporary CA, the TLS handshake to the stub bidder fails, and the run reports zero bids. The bid assertions at lines 499, 511, 527, and 529 do catch that, so it fails loudly rather than passing vacuously — but the symptom is "no bids" in a 45-minute Viceroy job, which points nowhere near a naming-codec change.

Cheapest fix that keeps the failure fast and self-describing: pin the exact string in a Rust test next to the codec, so a codec change fails in seconds with a message naming this script.

// In crates/trusted-server-core/src/platform/backend_naming.rs, tests module.
// scripts/template-cache-local-test.sh recomputes this name in Python to
// pre-register the Viceroy backend that trusts its temporary CA. If this
// assertion changes, update that script's PYEOF block in the same commit.
#[test]
fn fastly_prediction_matches_template_cache_harness_backend() {
    let spec = PlatformBackendSpec {
        scheme: "https".to_string(),
        host: "localhost".to_string(),
        port: Some(9100),
        certificate_check: true,
        host_header_override: None,
        discriminator: Some("pbs-main".to_string()),
        first_byte_timeout: core::time::Duration::from_millis(5000),
        between_bytes_timeout: core::time::Duration::from_millis(5000),
    };
    let predicted = BackendNamingPolicy::Fastly
        .predict(&spec)
        .expect("should predict harness backend name");
    assert_eq!(
        predicted.name,
        "backend_https_localhost_9100_p_pbs-main_fb5000_bb5000_<digest>",
        "template-cache harness pre-registers this exact backend name"
    );
}

Fill <digest> from the current codec output, and adjust the PlatformBackendSpec field names if they differ from the above. Apply manually — this adds a test outside the changed range in this file.

for provider_name in provider_names {
let provider = match self.providers.get(provider_name) {
for provider_name in &provider_names {
let provider = match self.providers.get(*provider_name) {
Some(p) => p,
None => {
log::warn!("Provider '{}' not registered, skipping", provider_name);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 note — This log::warn! is missing the lgtm[rust/cleartext-logging] suppression that the byte-identical statement in the #[cfg(test)] copy of this loop carries at line 1874:

                None => {
                    // lgtm[rust/cleartext-logging]
                    // The provider name is a static config identifier (e.g. "prebid"), not a secret.
                    log::warn!("Provider '{}' not registered, skipping", provider_name);
                    continue;
                }

main carries three such suppressions in this file (at its lines 748, 988, and 1446); this head carries two (1391 and 1872). The one that went missing is the production half of this duplicated loop — run_providers_parallel is not cfg-gated, while its twin at 1874 is #[cfg(test)], so the annotation survived on the copy that CodeQL is least interested in and was dropped from the one that ships.

This is not itself among the six alerts CodeQL currently reports, and the log statement is genuinely safe — provider_name is a plan-derived provider ID. Worth restoring anyway, both because the reasoning comment is the only thing recording why it is safe, and because the asymmetry between the two copies is a good early signal that the production and cfg(test) dispatch loops have diverged.

Apply manually — the fix is the two comment lines above, inserted before the log::warn!.

Comment thread CHANGELOG.md

### Changed

- **Breaking:** Auction providers and bidder routes now use the configuration-first `[auction.providers.<id>]` and `[auction.bidders.<id>]` maps. The removed `[auction].providers = [...]` list and removed server fields under `[integrations.prebid]` and `[integrations.aps]` are rejected even when those integrations are disabled, and `ts config push` rejects the old shape before publication. Move PBS `server_url` to provider `endpoint`, server timeout to provider `timeout_ms`, request controls and bidder-parameter overrides to the `prebid-server` `profile_config`, notification suppression to `notifications`, and each former server bidder to an `[auction.bidders.<id>]` route. Move APS endpoint, timeout, account, inventory, debug, and creative controls to an `aps` provider and its `profile_config`. Browser Prebid settings remain under `[integrations.prebid]`; values such as timeout and debug that previously affected both browser and server behavior must now be configured for each owner. Provider endpoints must be absolute HTTPS URLs. Only bidder codes present in `[auction.bidders]` are folded into Trusted Server requests; unlisted publisher bids remain native browser demand. This schema has no mixed-version-safe deployment order: old binaries reject the maps and new binaries reject the retired fields, so activate the new binary and config blob together. Rollbacks must restore an old-schema blob together with the old binary.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 praise — Worth calling out, because the diff stat (+2 lines on this file) badly undersells it. This single entry does what breaking-change notes usually omit: it names the retired shape and the replacement shape, walks each moved field to its new owner (server_url to endpoint, server timeout to timeout_ms, request controls and bidder-parameter overrides to profile_config, notification suppression to notifications, each former server bidder to an [auction.bidders.<id>] route), states that removed fields are rejected even when the integration is disabled, and draws the browser-versus-server ownership line for values like timeout and debug that previously did double duty.

Most valuable part is the deployment paragraph: stating outright that there is no mixed-version-safe ordering, that old binaries reject the maps while new binaries reject the retired fields, that the binary and config blob must be activated together, and that a rollback needs an old-schema blob restored alongside the old binary. That is the detail an operator actually needs on the day of the migration and the one most often discovered the hard way.

ChristianPavilonis added a commit that referenced this pull request Aug 28, 2026
# Conflicts:
#	crates/trusted-server-adapter-spin/src/platform.rs
#	crates/trusted-server-core/src/auction/endpoints.rs
#	crates/trusted-server-core/src/auction_config_types.rs
#	crates/trusted-server-core/src/config.rs
#	crates/trusted-server-core/src/config_payload.rs
#	crates/trusted-server-core/src/creative_opportunities.rs
#	crates/trusted-server-core/src/integrations/prebid.rs
#	crates/trusted-server-core/src/integrations/registry.rs
#	crates/trusted-server-core/src/publisher.rs
#	crates/trusted-server-core/src/settings.rs
#	docs/guide/api-reference.md
#	docs/guide/auction-orchestration.md
#	docs/guide/configuration.md
#	docs/guide/ec-setup-guide.md
#	docs/guide/error-reference.md
#	docs/guide/fastly.md
#	docs/guide/first-party-proxy.md
#	docs/guide/getting-started.md
#	docs/guide/integrations/aps.md
#	docs/guide/proxy-signing.md
#	scripts/template-cache-local-test.sh
#	trusted-server.example.toml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement config-first generic OpenRTB auction providers

3 participants