feat(claude-agents)!: emit invoke_agent, chat and execute_tool spans - #31
feat(claude-agents)!: emit invoke_agent, chat and execute_tool spans#31apucacao wants to merge 5 commits into
Conversation
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
c73f2a4 to
41575b5
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 41575b5. Configure here.
41575b5 to
8c8604f
Compare
|
bugbot run |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 078c7a5. Configure here.
|
bugbot run |
078c7a5 to
8e1dd90
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
2 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 8e1dd90. Configure here.
8e1dd90 to
9c7d549
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
2 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 9c7d549. Configure here.
|
bugbot run |
9c7d549 to
e58ffc4
Compare
|
bugbot run |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
4 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 05567c2. Configure here.
05567c2 to
fa26c6f
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
4 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit fa26c6f. Configure here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
4 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit fa26c6f. Configure here.
fa26c6f to
b0c07c1
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
4 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b0c07c1. Configure here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
4 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b0c07c1. Configure here.
b0c07c1 to
686fbe4
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 686fbe4. Configure here.
686fbe4 to
f2d2cfe
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f2d2cfe. Configure here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f2d2cfe. Configure here.
One flat span named claude.query becomes the tree the TypeScript SDK emits: an
invoke_agent root, one `chat {model}` child per inference the Agent SDK
reports, one `execute_tool {name}` child per tool call.
BREAKING CHANGE: the span this handler emits is renamed from `claude.query` and
`claude.query.stream` to `invoke_agent`. Queries selecting on the old names
will not match. Prompt and completion content is no longer on spans unless the
caller passes capture_content=True.
Subagent identity was invisible. The Agent SDK reports a request id, a session
id and a subagent type for every inference, and none of it reached a span, so a
trace could not say which subagent ran or group turns by conversation. The chat
span now carries gen_ai.response.id and gen_ai.agent.name, and
gen_ai.conversation.id goes on all three span types: the root learns it from
the CLI's init message, the chat span from the inference, and the tool span
from the hook input, which is where this side sees it without waiting for a
message.
Cache tokens now reach the span, folded into the input total, because Anthropic
reports cache reads and writes beside the input count rather than inside it.
The finish reason is written when the SDK reports one, which in practice is
almost never: measured against Agent SDK 0.3.220, stop_reason is null on every
assistant message and only the run-level result carries one. Deriving a reason
from the presence of a tool-use block would put a value on the span the
provider never returned, so the write stays guarded and usually absent.
The streaming path needed more than a finally. The blocking path already held
the vendor's query generator in a variable and awaited aclose() on it, because
a bare return inside `async for` abandons it and asyncio's finalizer then
raises RuntimeError when it is suspended inside a real await in the SDK. The
streaming path iterated the generator inline with no held reference and had no
such cleanup, so it carried the same bug the blocking path was patched for.
Both now close the generator in the same finally that ends the spans, and the
streaming path gets the test the blocking path already had.
A run abandoned mid-stream now closes any tool span whose PostToolUse hook
never fired, which is otherwise the one span with no path to being ended.
Tests: 56 to 90. The telemetry tests are rewritten rather than extended,
because they pinned the old flat span, and every test that was not about
telemetry is preserved under its original name.
_build_query_options passed tools=[] whenever a config had no native tools. An explicit empty list is not the same as omitting the key: it tells the Agent SDK there are no tools, which switches off the Claude Code built-ins. A run with only MCP tools, or none at all, silently lost Read, Bash and the rest. main omitted the key in that case, leaving the SDK default. Restored, so the condition is back where it was and only a non-empty list is ever passed. Two tests, one per branch. The empty case fails when the regression is put back. Found by Bugbot on #31 at High severity. This is a behaviour regression the span port introduced, not a telemetry change.
…spans The streaming teardown reached close_open_spans, which records an exception and sets ERROR on every tool span still open. That is right for a failure and wrong for abandonment. A consumer stopping early is normal, and the root and chat spans on that same path are deliberately left UNSET with launchdarkly.stream.abandoned, so a tool span whose PostToolUse hook never fired reported an error nobody had. Adds abandon_open_spans beside close_open_spans and uses it on the abandonment path only. The openai-agents and langchain-agents handlers already drew this distinction, so this also settles a three-way disagreement about what one abandoned run looks like across the SDK. Two tests: abandonment leaves UNSET with the marker, and a genuine failure still sets ERROR.
Both paths wrote the all-zero per-response sum when the stream ended without a ResultMessage and without absorbing a single assistant turn. Zeros on the root say the run cost nothing, which is a different claim from not knowing what it cost, and a config-scoped cost query cannot tell the two apart once they are written. Absent usage means unknown, which is the honest answer here. The error and abandonment paths in this same file already guarded on reported, which is what makes the success paths' omission look accidental rather than considered. It was. Three tests: neither path writes usage when nothing reported, and a run that did report one turn still reports it. Found by Bugbot on #31.
The prompt write ran before the try that fails the root, so a raise while serialising it left the root open: never ended, never exported, so the run disappeared from AI Config Monitoring along with the feature_flag event it carries. Both paths had it. Two tests, one per path. Found by Bugbot on #34, which is this shape in langchain-messages.
f2d2cfe to
d3cf334
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit d3cf334. Configure here.
1 participant
Replaces one flat span per call with the tree the TypeScript SDK emits, for
claude-agents.Subagent identity was invisible
The Agent SDK reports a request id, a session id and a subagent type for every inference, and none of it reached a span. A trace could not say which subagent ran, or group turns by conversation.
gen_ai.response.idandgen_ai.agent.namenow go on the chat span.gen_ai.conversation.idgoes on all three span types: the root learns it from the CLI's init message, the chat span from the inference, and the tool span from the hook input, which is where this side sees it without waiting for a message.The streaming path had a real bug
The blocking path already held the vendor's query generator in a variable and awaited
aclose()on it, because a barereturninsideasync forabandons it and asyncio's finalizer then raisesRuntimeErrorwhen it is suspended inside a real await in the SDK. The streaming path iterated the generator inline with no held reference and no such cleanup, so it carried the same bug the blocking path was patched for. Both now close it in the samefinallythat ends the spans, and the streaming path gets the test the blocking path already had.A run abandoned mid-stream now also closes any tool span whose
PostToolUsehook never fired, which is otherwise the one span with no path to being ended.Other changes
stop_reasonis null on every assistant message and only the run-level result carries one. Deriving a reason from the presence of a tool-use block would put a value on the span the provider never returned, so the write stays guarded and usually absent.gen_ai.response.modelon the chat span is the model the turn actually used, not the requested name. This handler andopenai-messagesare the only two where those differ.Breaking change
The span is renamed from
claude.querytoinvoke_agent. Queries selecting on the old name will not match. Prompt and completion content is no longer on spans unless the caller passescapture_content=True.Where this sits
Needs the usage layer (#28) and the content layer (#29). Independent of the other five handler PRs; the stack orders them only because
gh stackis linear.Tests: 729 to 763. This is the largest of the six handlers.
Note
Overview
Replaces the single flat
claude.queryspan with the same three-level tree as the TypeScript SDK and other handlers:invoke_agent(LD identity and run-level usage),chat {model}per inference turn, andexecute_tool {name}siblings for tool calls. Span construction moves intospans.py;InferenceSpansgroups Agent SDK messages bymessage_idbecausequery()does not expose request boundaries.Tool telemetry uses
build_tool_hooksso PreToolUse/PostToolUse open and closeexecute_toolspans; streaming abandonment ends open tool spans as UNSET withlaunchdarkly.stream.abandoned, not ERROR.capture_content(default off) gates prompt/output/tool content on spans.Identity and usage:
gen_ai.conversation.id,gen_ai.response.id, andgen_ai.agent.nameon the appropriate spans; cache tokens folded into input totals; chatgen_ai.response.modelreflects the turn’s actual model. Non-successResultMessagesubtypes fail the run; usage is omitted when nothing was reported (no fake zero totals).Fixes: streaming path now
aclose()the vendorquerygenerator (parity with blocking); input serialization errors still end the root span;toolsis omitted when there are no native tools so Claude Code built-ins stay enabled.Breaking: span name
invoke_agent; content on spans only withcapture_content=True. Tests switch to a realInMemorySpanExporterand scripted SDK message streams.Reviewed by Cursor Bugbot for commit d3cf334. Bugbot is set up for automated code reviews on this repo. Configure here.