Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
a3df586
fix(workflow): target tmux session unambiguously in dispatch-role.sh
iscekic Jul 28, 2026
837a92e
feat(auto-routing-contracts): shared Efficient model pool and profile…
iscekic Jul 28, 2026
e6e6d14
feat(auto-routing-benchmark): exact Pool-entry identity for runs and …
iscekic Jul 28, 2026
0bcf9cb
Merge remote-tracking branch 'origin/main' into efficient-custom-pool…
iscekic Jul 28, 2026
8ad331c
Merge remote-tracking branch 'origin/main' into efficient-custom-pool…
iscekic Jul 28, 2026
89988df
feat(web): serve exact efficient catalog variants at gateway resolution
iscekic Jul 28, 2026
e74331c
feat(auto-routing): owner pools and custom routing on decide
iscekic Jul 28, 2026
e2d3516
feat(auto-routing-benchmark): global benchmark profile registry with …
iscekic Jul 28, 2026
0bfe68c
test(auto-routing-benchmark): execute admission SQL on real SQLite
iscekic Jul 28, 2026
2ccb56d
feat(web): add auto-routing settings API with catalog pool validation
iscekic Jul 28, 2026
6df44f5
feat(auto-routing-benchmark): profile purpose runs, single-slot drain…
iscekic Jul 28, 2026
460fcdf
feat(web): Efficient custom pool editor on Auto routing card
iscekic Jul 28, 2026
a5b072c
Merge remote-tracking branch 'origin/main' into efficient-custom-pool…
iscekic Jul 28, 2026
3314c77
fix(auto-routing): commit mode and pool atomically in setSettings
iscekic Jul 28, 2026
4d91c2c
fix(web): keep Auto routing settings safe against pre-deploy workers
iscekic Jul 28, 2026
2f51284
style(auto-routing): apply oxfmt to setSettings transaction files
iscekic Jul 28, 2026
d72b294
Merge remote-tracking branch 'origin/main' into efficient-custom-pool…
iscekic Jul 28, 2026
608e3d5
docs(auto-routing): record final pool atomicity, deploy-order, and mi…
iscekic Jul 28, 2026
22e222e
docs(workflow): record E2E fixture and gateway-auth learnings
iscekic Jul 28, 2026
3239b9d
fix(web): degrade malformed worker settings responses to a graceful 502
iscekic Jul 28, 2026
9a4a886
Merge remote-tracking branch 'origin/main' into efficient-custom-pool…
iscekic Jul 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .kilo_workflow/learnings/auto-routing-benchmark-d1-fixtures.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Benchmark profile D1 fixtures for owner-pool E2E (auto-routing)

Deterministic fixtures for `services/auto-routing-benchmark` local D1 (verified 2026-07-28):

- Engine identity is deterministic: `computeEngineIdentity('decider')` → `v1:8f2eee90`
(run `pnpm exec tsx -e "import {computeEngineIdentity} from './src/run.ts'; console.log(computeEngineIdentity('decider'))"`
from the service dir). Seed `benchmark_profiles.engine_identity` with exactly this value and
`repetitions` matching `benchmark_config.decider_repetitions`, or rows are stale and auto re-admit.
- `benchmark_config.switch_cost_factor` must be >= 1 (`BenchmarkConfigSchema` zod min 1). A seeded 0.1
makes `/admin/custom-routing-table` 500 (ZodError at buildCustomRoutingTable), which
`loadCustomRoutingTable` converts to a SILENT null table → decide returns null → gateway serves
the balanced fallback (qwen/qwen3.7-plus) with no product error. Symptom looks like "pool ignored".
- `benchmark_config` alone is not enough: `getBenchmarkConfig` returns null (→ 400 on all profile
endpoints) unless `config_classifier_models` and `config_decider_models` each have >= 1 row.
- Custom table assembly needs BOTH ready `benchmark_profiles` rows (with `run_id`) AND
`model_summaries` rows for that exact run_id + exact (model, variant) pair (provenance binding).
- Profile status lookup auto re-admits stale rows as pending WITHOUT charging
`profile_request_events` — verify ledger count stays flat across engine-drift.
- Generate multi-statement SQL with node, not a shell for-loop: zsh does NOT word-split an unquoted
`$VAR` (`for R in $ROUTES` iterates once with the whole string), producing one garbage row key.
22 changes: 22 additions & 0 deletions .kilo_workflow/learnings/web-gateway-org-context-and-fake-login.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Gateway/auth facts for local efficient-routing E2E (apps/web)

Verified 2026-07-28 on the efficient-custom-pool-e6e3 worktree:

- Session-cookie requests NEVER thread organizationId through `getUserFromAuth` (server.ts: org
header is read only in the `Authorization:` bearer path, "only used for extension-originated
requests"). Org-context routing assertions must use the user's own API JWT (rendered in the
profile page API-key textbox) as `Authorization: Bearer` plus `x-kilocode-organizationid`.
Never print the token; extract it in-page via `page.evaluate` and use it there.
- Org-context billing charges the ORGANIZATION balance; personal-context charges the user. Top up
the right entity before real calls.
- Balance top-up needs the denormalized counter: `UPDATE kilocode_users SET
total_microdollars_acquired=...` (same for `organizations`). Inserting only a
`credit_transactions` row does NOT change the balance (402 persists).
- Fake login in a browser: `/users/sign_in?fakeUser=...` 307-redirects to NEXTAUTH_URL, which in an
offset worktree points at a dead port (e.g. 192.168.1.10:3000). The session cookie is set by the
307 response before the failed redirect — navigate to the target page directly afterwards.
Robust alternative: `GET /api/auth/csrf` then POST `/api/auth/callback/fake-login`
(form-encoded csrfToken + email + json:true) via page.evaluate from any loaded page.
- Org model deny list for fixtures: `organizations.settings.model_deny_list` (jsonb array of model
ids) — immediately makes saved pool entries render `Unavailable` through the web
annotateConfiguredPool path (enterprise orgs).
12 changes: 11 additions & 1 deletion CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Scope

Kilo Code Cloud hosts Kilo Code agents, integrations, and automation. This contract defines Code Reviewer, Security Agent, and Cost Insights language plus ownership boundaries used across review execution, analytics, sync, web, email, remediation, billing alerts, tests, and product documentation.
Kilo Code Cloud hosts Kilo Code agents, integrations, and automation. This contract defines Code Reviewer, Security Agent, Cost Insights, and Auto Routing language plus ownership boundaries used across review execution, analytics, sync, web, email, remediation, billing alerts, model selection, tests, and product documentation.

Covered domains must use this contract's canonical terms in code, docs, task descriptions, tests, and agent outputs. Do not introduce a synonym for a governed concept without updating this contract. Scoped `AGENTS.md` files link to this contract instead of copying it.

Expand All @@ -16,6 +16,7 @@ Covered domains must use this contract's canonical terms in code, docs, task des
| **Security Agent Email Delivery** | Dispatch-time revalidation, email rendering, owner-aware links, and Mailgun delivery | `apps/web/src/app/api/internal/security-agent/`, `apps/web/src/lib/email.ts`, `apps/web/src/emails/` | Accepts notification identity only and loads current data before sending |
| **Shared Security Notification Policy** | Canonical config parsing, defaults, severity thresholds, and pure event eligibility rules | `packages/worker-utils/src/security-notification-policy.ts` | Web and Worker must use same policy contract |
| **Cost Insights** | Spend evidence dashboard, Spend Alerts policy, alert history, and owner-scoped spend alerting | Billing, usage ingestion, usage analytics, and subscription-management surfaces | Applies to both personal users and organizations |
| **Auto Routing** | Efficient model-pool settings, shared benchmark profiles, and per-request model selection | `packages/auto-routing-contracts/`, `services/auto-routing/`, `services/auto-routing-benchmark/`, `apps/web/src/components/auto-routing/` | Personal and organization settings constrain the existing `kilo-auto/efficient` model |

## Canonical Terms

Expand Down Expand Up @@ -44,6 +45,9 @@ Covered domains must use this contract's canonical terms in code, docs, task des
| **Cost Suggestion** | Optional owner-scoped recommendation based on observed Credit spend that may improve cost efficiency through an eligible Coding Plan or Kilo Pass | Referring to recommendation evaluation, dashboard cards, emails, CTA destinations, dismissal, or settings | Alert, warning, guaranteed savings, automatic optimization |
| **Suggestion dismissal** | Authorized owner action that hides one Cost Suggestion without changing billing or future suggestion eligibility | Referring to dismissing a recommendation | Alert acknowledgment, unsubscribe, disable suggestions |
| **Spend owner** | Personal user or organization whose credit balance is charged for Credit spend | Referring to the Spend Alerts policy and evaluation boundary | Account when personal/org ambiguity matters |
| **Efficient model pool** | The exact model and thinking-variant pairs that `kilo-auto/efficient` may consider for an owner | Referring to either the platform default or an owner-configured pool | Custom mode, custom efficient model |
| **Pool entry** | One concrete model paired with its canonical thinking-variant key, or the model's default only when it exposes no variants | Referring to benchmark and routing identity inside an Efficient model pool | Model alone, normalized effort |
| **Benchmark profile** | Globally shared, current benchmark results for one Pool entry | Referring to readiness and measured routing evidence reused across owners | User benchmark, pool benchmark |
| **Spend Anomaly Alert** | Spend Alert triggered when short-window owner Credit spend exceeds that owner's normal usage pattern | Referring to hourly burst-detection Spend Alerts | Low-balance alert, threshold alert |
| **Variable Credit spend** | Credit spend created by request-metered product usage such as token usage or metered tool/API usage | Referring to spend that can burst unexpectedly during active usage | Scheduled Credit spend |
| **Scheduled Credit spend** | Predictable Credit spend created by subscription-like purchases, renewals, or hosting deductions | Referring to expected recurring or explicitly purchased credit deductions | Variable Credit spend |
Expand All @@ -67,6 +71,9 @@ Covered domains must use this contract's canonical terms in code, docs, task des
- A **Security Remediation** belongs to one **Security Finding** and can have one or more **Security Remediation Attempts**.
- A **Security Finding Activity Event** belongs to one Security Agent owner and one Security Finding, including after that finding is deleted.
- **Spend Alerts** and **Cost Suggestions** belong to exactly one **Spend owner**: one personal user or one organization.
- An **Efficient model pool** belongs to one personal user or organization; an organization pool overrides members' personal pools, while no configured pool inherits the next available setting and then the platform default.
- A **Pool entry** has at most one current **Benchmark profile** for a benchmark-engine version, shared by every owner that selects that entry.
- `kilo-auto/efficient` considers only ready, request-compatible Pool entries from the effective Efficient model pool; its balanced failure fallback remains outside pool membership.
- All Credit spend charged to a **Spend owner** counts toward that owner's Spend Alerts and Cost Suggestion evaluation.
- **Cost Suggestions** are enabled by default, independent from Spend Alerts, and recommend an eligible Coding Plan or Kilo Pass when observed Credit spend indicates potential cost-efficiency benefit.
- Cost Suggestions are advisory. They do not guarantee savings, automatically purchase or change subscriptions, or alter spend behavior.
Expand Down Expand Up @@ -189,6 +196,8 @@ Covered domains must use this contract's canonical terms in code, docs, task des
- Do not describe a Cost Suggestion as an alert, warning, guaranteed savings, automatic optimization, or required action.
- Use **Spend Threshold Alert** and **Spend threshold** for the independent rolling 24-hour, rolling 7-day, and rolling 30-day threshold windows. Do not introduce warning or critical threshold tiers.
- Keep Spend Alerts alert-only. Do not describe them as spend blocks, hard limits, pauses, throttles, or request admission controls.
- Use **Efficient model pool** for the candidate set and **Pool entry** for an exact model and canonical thinking variant. Do not introduce a second efficient mode or model ID.
- Treat **Benchmark profiles** as global evidence, never owner-specific benchmark results.

## Ambiguities

Expand All @@ -215,6 +224,7 @@ Covered domains must use this contract's canonical terms in code, docs, task des
- **Shared Security Notification Policy** defines common parsing and pure eligibility behavior; it does not perform persistence or recipient lookup.
- Cross-context dispatch sends only stable notification ID from **Security Sync** to authenticated **Security Agent Email Delivery** boundary.
- **Spend Alerts** evaluate personal and organization Credit spend at the owner boundary, not per product by default.
- **Auto Routing** owns effective pool resolution and per-request selection. The benchmark worker owns Benchmark profile measurement and publication; web owns catalog validation, permissions, and settings UI.
- Do not assume Spend Alerts apply to owners who have not opted in.
- Do not make Spend Alerts block, pause, throttle, suppress auto-top-up, reject paid requests, or emit Spend Alerts-specific HTTP 402 responses.
- Do not hide v1 Cost Insights behind a release-toggle gate unless a later product decision supersedes public opt-in.
Expand Down
Loading
Loading