feat(self-driving): prioritise codebase-detected tools in the connected-tools ask - #1022
Merged
Conversation
…ed-tools ask The self-driving connected-tools multi-select (STEP 5) grew from 5 to 35 sources and is heading toward the full ~500-source warehouse catalog, so it overflows the terminal. The fix agreed in the thread: surface the tools we can detect in the user's codebase first, then the SaaS basics, then an "others" bucket — rather than dumping the whole list. The wizard already has the detection for this: the same deterministic `detectWarehouseSources` scanner that powers the warehouse program already covers ~16 of the self-driving inbox tools (GitHub, GitLab, Gitea, Linear, Jira, Sentry, Rollbar, Bugsnag, Zendesk, Freshdesk, Front, Gorgias, Intercom, and more). This wires that detection into the self-driving flow and feeds it to the agent. - The self-driving detect step now runs the scan and stashes the result under the shared `DETECTED_WAREHOUSE_SOURCES_KEY` (best-effort; a scan failure or an empty result never blocks the flow). - `buildSelfDrivingPrompt` renders a "Tools detected in this codebase" block and points STEP 4/STEP 5 at it. Empty scan states so explicitly, so the agent falls back to the skill's default ordering instead of inventing its own scan. The wizard only provides the deterministic facts; which tools are inbox-relevant and how the ask is ordered stays in the context-mill skill (companion change), keeping product knowledge out of wizard code.
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
Member
Author
|
Companion context-mill PR (restructures the step-5 ask to consume the detected-tools block): PostHog/context-mill#291 |
gewenyu99
approved these changes
Jul 29, 2026
Collaborator
Screen.Recording.2026-07-29.at.6.58.59.PM.movMuch better, but I'll make a change to that button name in another PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The self-driving connected-tools multi-select (STEP 5) ballooned from 5 → 35 sources and is heading toward the full ~500-source warehouse catalog. It overflows the terminal — you can't see the whole list (thread).
The agreed fix: show the tools we can detect in the user's codebase first, then the SaaS basics, then an "others" bucket — instead of dumping the whole catalog.
Changes
The wizard already has the detection for this. The same deterministic
detectWarehouseSourcesscanner that powers the warehouse program already covers ~16 of the self-driving inbox tools (GitHub, GitLab, Gitea, Linear, Jira, Sentry, Rollbar, Bugsnag, Zendesk, Freshdesk, Front, Gorgias, Intercom, …). This wires it into the self-driving flow and feeds the result to the agent:DETECTED_WAREHOUSE_SOURCES_KEY. Best-effort — a scan failure or an empty result never blocks the flow (nodetectError, unlike the standalone warehouse program).buildSelfDrivingPromptrenders a "Tools detected in this codebase" block and points STEP 4 (enable matching sources) and STEP 5 (order the ask) at it. An empty scan says so explicitly, so the agent falls back to the skill's default ordering rather than inventing its own flaky scan.The boundary
The wizard only provides the deterministic facts (what's in the codebase). Which tools are inbox-relevant, and how the ask is ordered (detected → basics → others), stays in the context-mill
self-drivingskill — that's the companion PR (context-mill), which restructures5-connected-tools.mdstep 5 to consume this block. Keeps product knowledge out of wizard code, matching the repo's design discipline.This half is inert without the skill change: the prompt block is additive context, and the current skill will simply ignore it until step 5 is updated to read it. Safe to land in either order.
Test plan
pnpm build && pnpm test && pnpm fix— build passes, 1666 tests pass, 0 lint errors.New tests:
self-driving-detect.test.ts— a@sentry/nodedep gets stashed as a detectedSentrytool; a bare dir writes nothing (and still clears detection).self-driving-prompt.test.ts— the block lists each tool with itssource_typeand matched signal, sits before STEP 5, points STEP 5 at "detected → basics → others", and states "none found" on an empty scan (so no invented scan).Not yet run end-to-end against a real self-driving run — the prompt block is verified by construction, and it's inert until the skill consumes it.