feat(openai-messages)!: emit invoke_agent, chat and execute_tool spans - #32
feat(openai-messages)!: emit invoke_agent, chat and execute_tool spans#32apucacao wants to merge 8 commits into
Conversation
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 5b44877. Configure here.
5b44877 to
b2f5fa1
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b2f5fa1. Configure here.
b2f5fa1 to
77d1074
Compare
|
bugbot run |
77d1074 to
910e8a7
Compare
|
bugbot run |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 910e8a7. Configure here.
910e8a7 to
43fc252
Compare
|
bugbot run |
43fc252 to
ba774df
Compare
|
bugbot run |
|
bugbot run |
ba774df to
2c0c94e
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2c0c94e. Configure here.
2c0c94e to
440799c
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 38b0cec. Configure here.
38b0cec to
538fc03
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 538fc03. Configure here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 538fc03. Configure here.
538fc03 to
a28267f
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit a28267f. Configure here.
a28267f to
c8b2cd2
Compare
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit c8b2cd2. Configure here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit c8b2cd2. Configure here.
One flat span named openai.response becomes the tree the TypeScript SDK emits:
an invoke_agent root, one `chat {model}` child per model turn, one
`execute_tool {name}` child per tool call.
BREAKING CHANGE: the span this handler emits is renamed from `openai.response`
and `openai.response.stream` to `invoke_agent`. Queries selecting on the old
names will not match. Prompt and completion content is no longer on spans
unless the caller passes capture_content=True.
Cached tokens were absent entirely. OpenAI reports them under
input_tokens_details.cached_tokens, which nothing here read, so every span
understated what a prompt-cached call actually reused. They are now reported in
gen_ai.usage.cache_read.input_tokens and, unlike Anthropic, not added on top of
the input figure: OpenAI already counts them inside it, and adding them would
double-count. Cache creation is always zero, because OpenAI has no such
concept.
This handler is the only one of the six that reports the model which actually
answered rather than the one requested, on both the root and the chat spans.
OpenAI resolves an alias like gpt-4o to a dated snapshot, and this handler has
the resolved value to hand.
Finish reasons are derived, not mapped. The Responses API has no finish_reason
field, so the shared mapping table does not apply and is deliberately not
imported. The value comes from a closed three-way check: a function call in the
output means tool_calls, an incomplete status means length or content_filter
depending on the reported cause, a completed status means stop, and anything
else writes no attribute at all. The function-call check comes first because
status alone reports completed for a turn that stopped to call a tool.
The streaming path gets a finally, so a consumer that breaks out of the
iteration no longer leaves the root span unended and unexported, taking the
whole run out of AI Config Monitoring along with the feature_flag event it
carries.
Tests: 62 to 80.
…apper The wrapper never passed capture_content to the factory, so it stayed in kwargs and reached config(), which takes no such argument. A caller asking for content on spans got a TypeError rather than content. Lifted out alongside variables, which was already handled the same way and for the same reason: one configures the handler, the other belongs to the invocation, and config() accepts neither. Two tests, one per branch, asserting the flag reaches the factory and does not reach config(). Found by Bugbot on #33 against openai-agents. Five of the six wrappers had it; each is fixed in its own layer.
… its span The success-side content write and the span finish sat outside the try, so a raise while recording the result skipped both the finish and the failure path. The tool span was never ended, so the exporter never saw it: the run showed a root marked ERROR and no sign the tool had been called. Reachable rather than theoretical. Serialising a tool result raises TypeError whenever capture_content is on and the result is not JSON-serialisable, which is any object a handler happens to return. Inherited from the claude-messages handler this one was modelled on, which had it in the wrong place. The TypeScript handlers have always done this inside the try. Found by Bugbot on #34.
…t ends it The content writes on both sides of the provider call sat outside the try that fails the chat span, so a raise while serialising conversation content failed only the root. The chat span was never ended and never exported: a run showed an errored root with no sign a model call had happened. Reachable through capture_content, where serialising any non-JSON-serialisable value raises TypeError. The tool path in this same file already kept its serialisation inside the guard, which is what makes the model path's omission look accidental rather than considered. It was. Found by Bugbot on #32.
… reads Span construction moved to spans.py, which holds the real _HAS_OTEL. The handler kept its own copy, plus the two imports it needed, alive only by a noqa. Nothing read any of it. That mattered because the tests patched the dead one. 7 tests set handler._HAS_OTEL to False and believed they were exercising the install without the otel extra; the flag was unread, so they exercised nothing and passed either way. They now patch spans._HAS_OTEL, which is the flag start_root_span actually consults: with it patched, span creation returns None, and with it set it does not. Found by Bugbot on #32. Five of the six handlers carried the dead gate, and four had tests aimed at it.
The streaming finally closed the model span and the root, but the in-flight execute_tool span was held only by a local. except Exception does not see a CancelledError or a GeneratorExit, so a tool cancelled mid-flight left its span open and unexported: the trace showed a closed parent above a child that never arrived, which reads as a tool that is still running long after the run ended. Tracked in open_tool_span and abandoned in the finally, the same way the model span already was. The tracker is cleared on the two paths that end the span and deliberately not in a finally, because a finally would also clear it for the BaseException case, which is the one case where the outer finally is the only thing left to close it. Found by Bugbot on #32. openai-agents already did this through its hook object; three other handlers share the gap and are fixed in their own layers.
…ntent fails Moving the content write inside the span guard left the accounting behind it, so a raise while serialising a response dropped that turn from the run total. The provider had already billed the call. Failing to serialise its content is our problem, and it is not a reason to report the run as having spent less than it did: the root is the only span a config-scoped cost query can read the total from. The usage is taken and accumulated straight after the provider returns, before anything that can raise. Found by Bugbot on #32, reviewing the span-leak fix that introduced it.
The content write and the span finish sat after the try that fails the chat span, and the usage was accumulated last of all. A raise while serialising the response left the span for the finally to end as abandoned, which reads as a consumer who walked away rather than as the failure it was, and dropped a turn the provider had already billed. The blocking path in this same file already did both correctly, which is what made the streaming path's ordering look accidental rather than considered. It was. Two tests: the span is failed rather than abandoned, and the tokens survive. Found by auditing every handler for the ordering Bugbot reported on #30 and #34. This path had the same defect and had not been reported.
c8b2cd2 to
654d77e
Compare
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 654d77e. Configure here.
| id=call_id if isinstance(call_id, str) else None, | ||
| name=str(_attr(item, "name") or ""), | ||
| arguments=_attr(item, "arguments"), | ||
| ) |
There was a problem hiding this comment.
Tool args left as JSON strings
Medium Severity
Responses API arguments arrive as a JSON string, and output_item_parts / split_input_messages store that string on tool_call parts unchanged. Other handlers put a parsed object there. SpanMessagePart.to_text then json.dumpss it again, so with capture_content=True OpenLLMetry completion text double-encodes tool arguments and diverges from Anthropic/LangChain spans.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 654d77e. Configure here.
1 participant
Replaces one flat span per call with the tree the TypeScript SDK emits, for
openai-messages.Cached tokens were absent entirely
OpenAI reports them under
input_tokens_details.cached_tokens, which nothing here read, so every span understated what a prompt-cached call actually reused.They now appear in
gen_ai.usage.cache_read.input_tokensand, unlike Anthropic, are not added on top of the input figure: OpenAI already counts them inside it, and adding them would double-count. Cache creation is always zero, because OpenAI has no such concept.Two things specific to this handler
It is the only one of the six that reports the model which actually answered rather than the one requested, on both the root and the chat spans. OpenAI resolves an alias like
gpt-4oto a dated snapshot, and this handler has the resolved value to hand.Finish reasons are derived, not mapped. The Responses API has no
finish_reasonfield, so the shared mapping table does not apply and is deliberately not imported. The value comes from a closed three-way check: a function call in the output meanstool_calls, an incomplete status meanslengthorcontent_filterdepending on the reported cause, a completed status meansstop, and anything else writes no attribute. The function-call check comes first, because status alone reportscompletedfor a turn that stopped to call a tool.Other changes
The streaming path gets a
finally, so a consumer that breaks out of the iteration no longer leaves the root span unended and unexported, taking the whole run out of AI Config Monitoring along with thefeature_flagevent it carries.Breaking change
The span is renamed from
openai.responsetoinvoke_agent. Queries selecting on the old name will not match. Prompt and completion content is no longer on spans unless the caller passescapture_content=True.Where this sits
Needs the usage layer (#28) and the content layer (#29). Independent of the other five handler PRs; the stack orders them only because
gh stackis linear.Tests: 763 to 781.
Note
Overview
Replaces the single
openai.responsespan with the same tree the TypeScript SDK emits: aninvoke_agentroot (LaunchDarkly identity and run-level token totals), onechat {model}child per provider turn, andexecute_tool {name}siblings for each tool call. Span logic moves intospans.py; blocking and streaming paths share_run_model_turn/ equivalent streaming bookkeeping.Telemetry behavior changes: OpenAI
cached_tokensare recorded asgen_ai.usage.cache_read.input_tokenswithout inflating input totals. Finish reasons are derived from Responses output/status (function calls beforecompleted). Root and chat spans setgen_ai.response.modelto the model that actually answered. Prompt/completion and tool args/results are off by default; callers passcapture_content=Trueon the factory oropenai_messages()wrapper.Reliability: Usage is accumulated before content serialization so billed tokens survive failures. Streaming adds a
finallypath so early consumer exit still ends spans (abandoned vs ERROR), and in-flight chat/tool spans are closed onBaseExceptionpaths.Breaking: Span name/query targets change; content attributes require opt-in.
Reviewed by Cursor Bugbot for commit 654d77e. Bugbot is set up for automated code reviews on this repo. Configure here.