feat(api-proxy): use runtime model metadata and pricing overlays by lpcox · Pull Request #6695 · github/gh-aw-firewall

lpcox · 2026-07-28T17:18:16Z

Summary

preserve normalized provider model metadata and use versioned Copilot runtime pricing ahead of bundled catalogs
support apiProxy.providers operator pricing overlays end-to-end, including the previously dropped default-pricing assembly path
add long-context pricing and provenance to AI-credit accounting, logs, /reflect, and models.json
fail closed for unverifiable Copilot auto requests when model policy, multiplier caps, or AI-credit pricing cannot be proven
add claude-opus-5 and the gpt-5.6-{sol,terra,luna} family to static model/pricing catalogs
extend catalog consistency tests to catch future drift from docs/model-api-mapping.json

Validation

root Jest suite: 271 suites, 4,543 tests
API proxy Jest suite: 212 suites, 2,659 tests
TypeScript type-check and build
ESLint and changed-document Markdown lint

Add provider pricing overlays and Copilot runtime pricing tiers. Fail closed for unverifiable auto routing and check catalog consistency. Closes #6682 Closes #6674 Refs #6688 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a9cf3033-ffae-4b83-b174-b8b2099c5416

github-actions · 2026-07-28T17:18:47Z

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 869e267

github-actions · 2026-07-28T17:21:03Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	99.17%	99.20%	📈 +0.03%
Statements	99.07%	99.10%	📈 +0.03%
Functions	99.45%	99.45%	➡️ +0.00%
Branches	95.75%	95.79%	📈 +0.04%

📁 Per-file Coverage Changes (2 files)

File	Lines (Before → After)	Statements (Before → After)
`src/commands/validators/api-proxy-validator.ts`	96.7% → 97.0% (+0.30%)	97.0% → 97.3% (+0.25%)
`src/log-directory-setup.ts`	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Adds runtime/provider pricing metadata to API-proxy accounting and expands Copilot model support.

Changes:

Wires operator pricing overlays through configuration.
Adds runtime Copilot pricing, long-context tiers, and provenance.
Expands model catalogs and fail-closed auto handling.

Show a summary per file

File	Description
`src/types/rate-limit-options.ts`	Types provider overlays.
`src/services/api-proxy-env-config.ts`	Exports overlay environment data.
`src/services/api-proxy-env-config.test.ts`	Tests overlay export.
`src/copilot-model.ts`	Adds supported models.
`src/copilot-model-catalog-sync.test.ts`	Expands catalog consistency checks.
`src/config-mapper.ts`	Maps provider configuration.
`src/config-file.ts`	Types provider configuration.
`src/config-file-mapping.test.ts`	Tests provider mapping.
`src/commands/validators/config-assembly-model-detection.test.ts`	Tests runtime validation deferral.
`src/commands/validators/api-proxy-validator.ts`	Defers unknown direct models.
`src/commands/build-config.ts`	Preserves pricing configuration.
`src/commands/build-config.test.ts`	Tests end-to-end configuration.
`src/awf-config-schema.json`	Adds providers schema.
`docs/awf-config.schema.json`	Publishes providers schema.
`docs/awf-config-spec.md`	Documents pricing precedence.
`docs/api-proxy-sidecar.md`	Documents runtime metadata.
`containers/api-proxy/token-tracker-shared.js`	Persists pricing provenance.
`containers/api-proxy/token-budget-log.test.js`	Tests provenance fields.
`containers/api-proxy/token-budget-log.js`	Logs pricing provenance.
`containers/api-proxy/server.token-guards.test.js`	Tests fail-closed `auto`.
`containers/api-proxy/server.network.test.js`	Tests metadata discovery.
`containers/api-proxy/server.js`	Exposes runtime metadata.
`containers/api-proxy/runtime-model-catalog.test.js`	Tests runtime normalization.
`containers/api-proxy/runtime-model-catalog.js`	Implements runtime catalog.
`containers/api-proxy/providers/index.js`	Documents adapter metadata fields.
`containers/api-proxy/providers/copilot.js`	Versions Copilot discovery.
`containers/api-proxy/provider-pricing-overlays.js`	Resolves operator overlays.
`containers/api-proxy/model-discovery.js`	Preserves discovered metadata.
`containers/api-proxy/management.js`	Adds metadata to reflection.
`containers/api-proxy/key-validation.js`	Maintains runtime snapshots.
`containers/api-proxy/guards/model-policy-guard.test.js`	Tests `auto` policy behavior.
`containers/api-proxy/guards/model-policy-guard.js`	Fails closed for dynamic policy.
`containers/api-proxy/guards/max-model-multiplier-guard.test.js`	Tests dynamic multiplier handling.
`containers/api-proxy/guards/max-model-multiplier-guard.js`	Guards unverifiable multipliers.
`containers/api-proxy/guards/ai-credits-guard.test.js`	Tests pricing precedence.
`containers/api-proxy/guards/ai-credits-guard.js`	Applies runtime tiered pricing.
`containers/api-proxy/Dockerfile`	Packages new runtime modules.
`containers/api-proxy/copilot-byok.test.js`	Tests versioned discovery headers.
`containers/api-proxy/ai-credits-pricing.js`	Adds static model prices.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 39/39 changed files
Comments generated: 5
Review effort level: Medium

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a9cf3033-ffae-4b83-b174-b8b2099c5416

github-actions · 2026-07-28T18:22:55Z

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

github-actions · 2026-07-28T18:29:48Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-07-28T18:29:49Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-07-28T18:29:51Z

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

github-actions · 2026-07-28T18:29:51Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-07-28T18:29:51Z

❌ Contribution Check failed. Please review the logs for details.

github-actions · 2026-07-28T18:29:51Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

github-actions · 2026-07-28T18:29:54Z

✅ Build Test Suite completed successfully!

github-actions · 2026-07-28T18:29:54Z

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

github-actions · 2026-07-28T18:29:55Z

✅ Smoke Claude passed

github-actions · 2026-07-28T18:29:55Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-07-28T18:30:00Z

✅ Smoke Gemini completed. All facets verified. 💎

github-actions · 2026-07-28T18:30:02Z

❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...

github-actions · 2026-07-28T18:30:05Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-07-28T18:30:06Z

❌ Security Guard failed. Please review the logs for details.

github-actions · 2026-07-28T18:30:28Z

📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-07-28T18:31:53Z

Smoke Test: Claude Engine Validation

Check	Result
API status	✅ PASS
gh check	✅ PASS
File status	✅ PASS

Overall result: PASS ✅

Generated by Smoke Claude for #6695 · haiku45 · 55.1 AIC · ⊞ 3.3K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:31:54Z

Smoke Test: Copilot BYOK (Direct) — PASS

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

Test	Result
GitHub MCP connectivity	✅
GitHub.com connectivity (HTTP 200)	✅
File write/read	✅
BYOK inference (agent → api-proxy → api.githubcopilot.com)	✅

Overall: PASS

@lpcox

🔑 BYOK report filed by Smoke Copilot BYOK
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:32:10Z

@lpcox — smoke test results for this PR:

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com): HTTP 200
✅ Blocked domain (example.com): connection denied (403 from proxy)

Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:32:31Z

🔬 Smoke Test Results

Test	Status
GitHub MCP connectivity	⚠️ PR data filtered by secrecy policy
GitHub.com HTTP	✅ 200 OK
File write/read	❌ File not found (pre-step skipped/template not resolved)

Overall: PARTIAL — template variables were not resolved (${{ steps.smoke-data.outputs.* }}), indicating the workflow ran without the pre-compute step populating outputs. MCP read filtered by secrecy policy.

PR author: @lpcox

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:32:46Z

🔬 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Notes
1. Module Loading	✅ Pass	`otel.js` loads successfully; exports: `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled`
2. Test Suite	✅ Pass	59 tests across 2 suites (`otel.test.js`, `otel-fanout.test.js`) — all passed
3. Env Var Forwarding	✅ Pass	`src/services/api-proxy-env-config.test.ts` confirms forwarding of `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`, `OTEL_SERVICE_NAME`
4. Token Tracker Integration	✅ Pass	`onUsage` callback present in `token-tracker-http.js` (line 406) as the OTEL hook point
5. OTEL Diagnostics	✅ Pass	Module initializes; falls back to `/var/log/api-proxy/otel.jsonl` when no OTLP endpoint configured

All 5 scenarios pass. OTEL tracing integration is functional.

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:33:04Z

Smoke Test Results — Services Connectivity

Check	Result
Redis PING	❌ `host.docker.internal` not resolvable
PostgreSQL pg_isready	❌ no response
PostgreSQL SELECT 1	❌ host not resolvable

Overall: FAIL — host.docker.internal DNS resolution fails in this environment. Service containers are not reachable.

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:33:17Z

Smoke Test Results

feat: add sealed probes for private repositories #6673: feat: add sealed probes for private repositories
chore: upgrade gh-aw extension to v0.83.4 pre-release and recompile workflows #6681: [Filtered by secrecy policy]
GitHub MCP: ❌
Connectivity: ❌
File Writing: ✅
Bash Tool: ✅

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:34:23Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for #6695 · sonnet46 · 43.3 AIC · ⊞ 8.4K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:34:38Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	`Python 3.12.13`	`Python 3.12.13`	✅ YES
Node.js	`v24.18.0`	`v22.23.1`	❌ NO
Go	`go1.22.12`	`go1.22.12`	✅ YES

Result: ❌ FAILED — Node.js version mismatch (host: v24.18.0, chroot: v22.23.1).

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:34:58Z

Smoke test results for feat(api-proxy): use runtime model metadata and pricing overlays

merged PRs: chore: upgrade gh-aw extension to v0.83.4 pre-release and recompile workflows; feat: add sealed probes for private repositories
GitHub query: ✅
Playwright title check: ✅
Temp file write/read: ✅
npm ci && npm run build: ❌ (403 fetching yocto-queue)
Overall: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T18:35:46Z

🔥 Smoke Test: Docker Sbx — PASS

Test	Result
GitHub MCP connectivity	✅ Connected
GitHub.com HTTP	✅ 200 OK
File write/read	✅ Verified (`smoke-test-docker-sbx-30387655442.txt`)

Overall: ✅ PASS

📰 BREAKING: Report filed by Smoke Docker Sbx
_{Add label ready-for-aw to run again}

Copilot AI review requested due to automatic review settings July 28, 2026 17:18

Copilot started reviewing on behalf of lpcox July 28, 2026 17:18 View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

lpcox changed the title ~~Use runtime model metadata and provider pricing overlays~~ feat(api-proxy): use runtime model metadata and pricing overlays Jul 28, 2026

fix(api-proxy): harden runtime pricing accounting

55def6d

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a9cf3033-ffae-4b83-b174-b8b2099c5416

lpcox added the ready-for-aw label Jul 28, 2026

lpcox had a problem deploying to aoai-model July 28, 2026 18:29 — with GitHub Actions Failure

github-actions Bot added smoke-claude smoke-copilot-byok labels Jul 28, 2026

lpcox temporarily deployed to aoai-model July 28, 2026 18:32 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Jul 28, 2026

lpcox temporarily deployed to aoai-model July 28, 2026 18:32 — with GitHub Actions Inactive

github-actions Bot mentioned this pull request Jul 28, 2026

[aw] Contribution Check failed #6696

Closed

github-actions Bot added the build-test label Jul 28, 2026

github-actions Bot mentioned this pull request Jul 28, 2026

[aw] Security Guard failed #6697

Closed

github-actions Bot added the smoke-docker-sbx label Jul 28, 2026

lpcox merged commit 1714eb7 into main Jul 28, 2026
141 of 146 checks passed

lpcox deleted the runtime-model-metadata branch July 28, 2026 18:53

This was referenced Jul 28, 2026

[aw] No-Op Runs #6693

Closed

Expose Azure CLI state and ADO auth token inside AWF sandbox #6690

Merged

[aw] No-Op Runs #6709

Open

Uh oh!

Conversation

lpcox commented Jul 28, 2026

Summary

Validation

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Preview

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

Smoke Test: Claude Engine Validation

Uh oh!

github-actions Bot commented Jul 28, 2026

Smoke Test: Copilot BYOK (Direct) — PASS

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading