Remove model experiments from AI gateway

[chrarnoldus]

Summary

• remove model-experiment membership, bucketing, direct routing, prompt capture, and usage attribution from the AI Gateway
• stop publishing experiment models and remove their automatic-routing special cases
• retain historical admin/data access while relocating shared direct-provider and metadata helpers out of the experiment namespace

Validation

• pnpm format:changed
• git diff --check
• local tests intentionally skipped; CI will validate the change

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the model-experiment removal across 50 files; deletions are internally consistent, no stale imports/type references remain, and no high-confidence defects were found in the changed lines.

Files Reviewed (50 files)

• apps/web/src/app/admin/api/auto-routing/benchmark-config/route.test.ts
• apps/web/src/app/admin/api/auto-routing/benchmark-config/route.ts
• apps/web/src/app/admin/model-experiments/ModelExperimentsContent.tsx
• apps/web/src/app/admin/model-experiments/metadata-json.ts
• apps/web/src/app/api/auto-routing/settings/route.test.ts
• apps/web/src/app/api/gateway/audience-auth.integration.test.ts
• apps/web/src/app/api/gateway/model-validation.integration.test.ts
• apps/web/src/app/api/openrouter/[...path]/route.test.ts
• apps/web/src/app/api/openrouter/[...path]/route.ts
• apps/web/src/app/api/openrouter/models/route.test.ts
• apps/web/src/app/api/openrouter/models/route.ts
• apps/web/src/app/api/openrouter/models/validate/route.test.ts
• apps/web/src/app/api/openrouter/models/validate/route.ts
• apps/web/src/components/auto-routing/AutoRoutingModeCard.test.ts
• apps/web/src/components/auto-routing/AutoRoutingModeCard.tsx
• apps/web/src/lib/ai-gateway/auto-routing-pool-validation.test.ts
• apps/web/src/lib/ai-gateway/auto-routing-pool-validation.ts
• apps/web/src/lib/ai-gateway/experiments/list-available-experiment-models.test.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/list-available-experiment-models.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/membership.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/persist.test.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/persist.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/pick-variant.test.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/pick-variant.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/pick-variant.types.ts (deleted)
• apps/web/src/lib/ai-gateway/experiments/reserved-ids.ts (deleted)
• apps/web/src/lib/ai-gateway/is-free-model.ts
• apps/web/src/lib/ai-gateway/llm-proxy-helpers.test.ts
• apps/web/src/lib/ai-gateway/llm-proxy-helpers.ts
• apps/web/src/lib/ai-gateway/local-fake-llm.ts
• apps/web/src/lib/ai-gateway/model-utils.ts
• apps/web/src/lib/ai-gateway/models.test.ts
• apps/web/src/lib/ai-gateway/models.ts
• apps/web/src/lib/ai-gateway/processUsage.types.ts
• apps/web/src/lib/ai-gateway/providers/build-direct-provider.test.ts
• apps/web/src/lib/ai-gateway/providers/build-direct-provider.ts
• apps/web/src/lib/ai-gateway/providers/get-provider.local-fake.test.ts
• apps/web/src/lib/ai-gateway/providers/get-provider.ts
• apps/web/src/lib/ai-gateway/providers/types.ts
• apps/web/src/lib/ai-gateway/providers/upstream-attempt.ts
• apps/web/src/lib/organizations/organization-auto-model.ts
• apps/web/src/lib/organizations/organization-models.test.ts
• apps/web/src/lib/organizations/organization-models.ts
• apps/web/src/lib/r2/client.ts
• apps/web/src/lib/r2/experiment-prompts.ts
• apps/web/src/lib/redis-keys.ts
• apps/web/src/routers/admin/model-experiments-router.ts
• apps/web/src/routers/organizations/organization-modes-router.test.ts
• apps/web/src/routers/organizations/organization-settings-router.test.ts
• apps/web/src/tests/openrouter-models.test.ts

Verification performed: repository-wide scan for imports of the deleted experiments/* modules and removed symbols (isPublicIdExperimented, pickModelExperimentVariant, listAvailableExperimentModels, findExperimentReservedModelIds, persistExperimentAttribution, buildExperimentPromptCapture, ResolvedExperimentUpstream, EXPERIMENTED_PUBLIC_IDS_REDIS_KEY, putPromptOrNull), and inspection of current get-provider.ts, the openrouter route, the admin router, and r2/experiment-prompts.ts for dangling references and unused symbols. No stale references or type/lint breaks were identified.

---

_{Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0}

_{Review guidance: REVIEW.md from base branch main}