feat(langchain-messages)!: emit invoke_agent, chat and execute_tool spans by apucacao · Pull Request #34 · launchdarkly/python-ai-sdk

apucacao · 2026-08-11T20:13:26Z

Replaces one flat span per call with the tree the TypeScript SDK emits, for langchain-messages.

invoke_agent                     one per call, carries the LD identity and the run total
├── chat {model}                 one per model turn, with that turn's own tokens
└── execute_tool {name}          one per tool call, a sibling of chat

Both provider attributes were wrong, in different ways

gen_ai.system was the configured provider name lower-cased, so an Anthropic-backed config reported anthropic where the TypeScript SDK reports langchain. That key names the instrumentation, and for the two LangChain handlers the instrumentation is the framework. It is now the literal string.

gen_ai.provider.name did not exist here at all, and it is not a passthrough of the configured name. It names who served the model, and its semantic-convention enum has no langchain member, so it follows the client the handler actually instantiates: ChatAnthropic for a configured provider of anthropic, ChatOpenAI for everything else.

A Bedrock or Azure config therefore reports openai. That looks wrong and is right, because an OpenAI client is what made the request. There is a test pinning it, because a passthrough reads as obviously correct.

Other changes

Cached tokens are read from usage_metadata.input_token_details and reported per turn, without being added to the input figure, which LangChain already reports inclusive of them.
Finish reasons go through the shared LangChain helper, which reads the reason from generation_info or response_metadata depending on which vendor answered. This handler is one of the places where the same code path serves either vendor, so an untranslated passthrough is least defensible here.
The streaming path gets a finally. The per-chunk usage accumulation is a faithful port of the TypeScript, summing each field as chunks arrive rather than reading a single terminal figure.

Breaking change

The span is renamed from langchain.invoke to invoke_agent. Queries selecting on the old name will not match. gen_ai.system changes value, as above. Prompt and completion content is no longer on spans unless the caller passes capture_content=True.

Where this sits

Needs the usage layer (#28) and the content layer (#29). Independent of the other five handler PRs; the stack orders them only because gh stack is linear.

Tests: 799 to 824.

Note

Overview
Replaces one flat langchain.invoke / langchain.stream span per call with the same tree the TypeScript SDK emits: invoke_agent root (LD identity + run token totals), chat {model} per model turn, and execute_tool {name} tool spans as siblings under the root.

Provider attributes are corrected: gen_ai.system is always langchain (framework, not configured provider); gen_ai.provider.name is added and reflects who actually served the call (anthropic vs openai from ChatAnthropic / ChatOpenAI, not a config passthrough).

Telemetry now uses shared server helpers for usage (including cache breakdown from input_token_details without double-counting input), finish reasons via lang_chain_finish_reasons, and optional capture_content=True for prompts/completions/tool I/O (default off). Span lifecycle is hardened with finally / end_unfinished_spans so cancellation, stream abandonment, and serialization failures still end spans and preserve billed tokens on the root.

Adds spans.py for span construction; langchain_messages() pops capture_content so it configures the handler instead of breaking config().

^{Reviewed by Cursor Bugbot for commit 178891a. Bugbot is set up for automated code reviews on this repo. Configure here.}

apucacao · 2026-08-11T20:20:44Z

bugbot run

apucacao · 2026-08-11T20:44:57Z

bugbot run

apucacao · 2026-08-11T21:02:21Z

bugbot run

apucacao · 2026-08-11T21:19:33Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c3de18e. Configure here.}

apucacao · 2026-08-11T21:46:45Z

bugbot run

apucacao · 2026-08-12T17:23:33Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 3b4ac11. Configure here.}

apucacao · 2026-08-12T17:46:38Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 777418b. Configure here.}

apucacao · 2026-08-12T18:03:37Z

bugbot run

apucacao · 2026-08-12T18:08:41Z

bugbot run

apucacao · 2026-08-14T19:16:58Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit fd6c38e. Configure here.}

apucacao · 2026-08-14T19:31:00Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d9c07c9. Configure here.}

apucacao · 2026-08-14T19:40:40Z

bugbot run

…pans One flat span named langchain.invoke becomes the tree the TypeScript SDK emits: an invoke_agent root, one `chat {model}` child per model turn, one `execute_tool {name}` child per tool call. BREAKING CHANGE: the span this handler emits is renamed from `langchain.invoke` and `langchain.stream` to `invoke_agent`. Queries selecting on the old names will not match. Prompt and completion content is no longer on spans unless the caller passes capture_content=True. gen_ai.system changes value; see below. Both provider attributes were wrong, in different ways. gen_ai.system was the configured provider name lower-cased, so an Anthropic-backed config reported `anthropic` where the TypeScript SDK reports `langchain`. That key names the instrumentation, and for these two handlers the instrumentation is the framework. It is now the literal string. gen_ai.provider.name now exists here at all, and it is not a passthrough of the configured name. It names who served the model, and its semantic-convention enum has no langchain member, so it follows the client the handler actually instantiates: ChatAnthropic for a configured provider of anthropic, ChatOpenAI for everything else. A Bedrock or Azure config therefore reports openai, which looks wrong and is right, because an OpenAI client is what made the request. Cached tokens are now read from usage_metadata.input_token_details and reported per turn, without being added to the input figure, which LangChain already reports inclusive of them. Finish reasons go through the shared LangChain helper, which reads the reason from generation_info or response_metadata depending on which vendor answered and maps it onto the shared vocabulary. This handler is one of the places where the same code path serves either vendor, so an untranslated passthrough is least defensible here. The streaming path gets a finally. The per-chunk usage accumulation is a faithful port of the TypeScript, summing each field as chunks arrive rather than reading a single terminal figure. Tests: 59 to 79.

…ros as spend Two defects, both introduced by this port. The structured-output paths were fighting. Python already bound response_format for OpenAI when both tools and outputFormat were set, and broke out of the tool loop with the model's own reply. The port added the TypeScript handler's structured follow-up turn on top of that, so for OpenAI the loop produced a structured reply and then a second call threw it away and billed another turn. Neither SDK does both. The follow-up now runs only when response_format was not bound, which is what carries Anthropic and every other non-OpenAI provider, since binding response_format is an OpenAI-only mechanism. The streaming path marked every turn as having reported usage, including turns where no chunk carried any. That defeats the flag: a later failure or abandonment then wrote all-zero totals on the root and claimed the run cost nothing, which is a different claim from unknown and the one thing the flag exists to prevent. The blocking path gets this right for free, because lang_chain_span_usage returns None for a bag the provider never filled. Two tests, and the first needs two turns to be meaningful: a turn that dies mid-iteration never reaches the accumulator, so only a turn that completes without usage followed by one that fails can exercise it. My first attempt passed with the fix reverted, which is how I found that out. Found by Bugbot on #34.

… wrapper The wrapper never passed capture_content to the factory, so it stayed in kwargs and reached config(), which takes no such argument. A caller asking for content on spans got a TypeError rather than content. Lifted out alongside variables, which was already handled the same way and for the same reason: one configures the handler, the other belongs to the invocation, and config() accepts neither. This wrapper also takes llm, so the flag joins it on the factory call rather than replacing the argument list. Found by Bugbot on #33 against openai-agents. Five of the six wrappers had it; each is fixed in its own layer.

…nds its span The success-side content write and the span finish sat outside the try, so a raise while recording the result skipped both the finish and the failure path. The tool span was never ended, so the exporter never saw it: the run showed a root marked ERROR and no sign the tool had been called. Reachable rather than theoretical. Serialising a tool result raises TypeError whenever capture_content is on and the result is not JSON-serialisable, which is any object a handler happens to return. Inherited from the claude-messages handler this one was modelled on, which had it in the wrong place. The TypeScript handlers have always done this inside the try. Found by Bugbot on #34.

Both reachable through capture_content, where serialising any non-JSON-serialisable value raises TypeError. The output write and the span finish sat outside the guard that fails the chat span, so a raise there left it open with nothing able to recover it: the blocking path has no finally. Now inside the try. The streaming finally awaited the vendor generator's aclose() before touching any span. aclose() can raise, and doing it first took the whole teardown with it: the root never ended, never exported, and the run disappeared from AI Config Monitoring along with the feature_flag event that block exists to protect. Spans close first now, and the vendor teardown is contained, because its failure is not worth losing the trace over. Two tests, each failing on its own defect when reverted. Found by Bugbot on #34.

…ger reads Span construction moved to spans.py, which holds the real _HAS_OTEL. The handler kept its own copy, plus the two imports it needed, alive only by a noqa. Nothing read any of it. No tests aimed at this one, so only the dead code goes. handler._HAS_OTEL to False and believed they were exercising the install without the otel extra; the flag was unread, so they exercised nothing and passed either way. They now patch spans._HAS_OTEL, which is the flag start_root_span actually consults: with it patched, span creation returns None, and with it set it does not. Found by Bugbot on #32. Five of the six handlers carried the dead gate, and four had tests aimed at it.

…open The streaming finally closed the model span and the root, but the in-flight execute_tool span was held only by a local. except Exception does not see a CancelledError or a GeneratorExit, so a tool cancelled mid-flight left its span open and unexported: the trace showed a closed parent above a child that never arrived. Tracked in open_tool_span and abandoned in the finally, the same way the model span already was. The tracker is cleared on the two paths that end the span and deliberately not in a finally, because a finally would also clear it for the BaseException case, which is the one case where the outer finally is the only thing left to close it. Found by Bugbot on the openai-messages layer, and shared by four of the six handlers.

…guard The output content write and the span finish sat outside the try that fails the chat span, and this path has no finally that could recover it. A raise while serialising the parsed object left the span open and unexported, and dropped the turn from the run total even though the provider had already billed it. Reachable through capture_content with any parsed object json.dumps refuses. The usage is now accumulated straight after the provider returns, before anything that can raise, and every span write happens inside the guard. Two tests: the span still ends and reports ERROR, and the tokens survive. Found while checking whether the openai-messages defect Bugbot reported on #32 reached the other handlers. It reached three of them.

The tool loop added its usage only after the try that serialises completion content, so a content failure dropped a turn the provider had already billed. The structured turn already accumulated before any content work, which is what made the tool loop's ordering look accidental rather than considered. It was. Both now accumulate straight after the provider returns. The structured result path then serialised the parsed object with json.dumps before handing it to set_output_content_attributes, including when capture_content was off. That helper is a no-op without the flag and json.dumps is not, so a parsed object json.dumps refuses turned a successful run into a raised TypeError for a caller who had asked for no content at all. The work now sits behind the flag that decides whether anyone will read it. Two tests, each failing on its own defect when reverted. Found by Bugbot on #34.

…okens The content write and the span finish sat outside the try that fails the chat span, and the usage was accumulated after both. A raise while serialising completion content left the span for the finally to end as abandoned, which reads as a consumer who walked away rather than as the failure it was, and dropped a turn the provider had already billed. Both now sit inside a guard, and the accumulation happens first. Two tests: the span is failed rather than abandoned, and the tokens survive. Found by Bugbot on #34.

The tool loop's own return serialised the run output with json.dumps before handing it to set_output_content_attributes, including when capture_content was off. That helper is a no-op without the flag and json.dumps is not, so with tools and an outputFormat on a non-OpenAI provider, where the output is the parsed object, a value json.dumps refuses turned a successful run into a raised TypeError for a caller who had asked for no content at all. The outputFormat-only path was already fixed the same way. This is its sibling exit, which the earlier fix missed. Found by Bugbot on #34.

… too The prompt write ran before the try that fails the span it writes to. Serialising conversation content raises on anything that is not JSON-serialisable, so a raise there left the chat span open on the structured path, and on both root paths left the root open: never ended, never exported, so the run disappeared from AI Config Monitoring along with the feature_flag event it carries. The output writes were moved inside their guards earlier in this stack. The input writes were not, which is the same defect at the other end of the same span. Two tests, one per root path, each failing when the fix is reverted. Found by Bugbot on #34. Four other handlers share the shape and are fixed in their own layers.

The input write for each chat span ran before the try that fails it. On the blocking path a raise left the child span open and unexported while the root was failed, and there is no finally on that path to recover it. On the streaming path the raise reached the outer finally with open_model_span still set, so the span was ended as abandoned: a content failure that reads as a consumer walking away. The root's own input writes were moved inside their guards earlier in this stack, and the structured turn already kept its input write inside one. The per-turn writes in both loops were missed. Two tests, one per path, each failing when the fix is reverted. Found by Bugbot on this PR.

A timeout or a task.cancel() raises asyncio.CancelledError, which inherits from BaseException, so it walks past every except Exception this handler has. The blocking path ended its spans only from those clauses, so a cancelled run exported nothing at all. Not a wrong attribute: no span. The root carries the feature_flag event and every launchdarkly.* attribute, so the whole run vanished from AI Config Monitoring rather than showing as incomplete. Three finally blocks now own the ends the except clauses cannot reach: one around the structured-output turn, one around the tool loop for the chat and tool spans, one in the caller for the root. This handler has that third finally the other packages do not need, because outputFormat without tools routes through a separate structured turn with its own model span, and a cancellation mid ainvoke there stranded that span exactly the same way. This is the shape the streaming path has had since the earlier rounds, so both paths of this handler now agree. Open spans are tracked by clearing a local when a path ends one, rather than by asking the span. A mock span answers is_recording() truthily and the test suite here is built on mock spans, so asking would have made the finally fire a second end on every successful run. A cancelled root still reports the spend of the turns that completed, for the same reason the failure path does: those turns were billed. Spans are left at UNSET and marked launchdarkly.run.cancelled. Nothing failed, the caller went away. Two tests, driving a real task.cancel() against a provider call that never returns. Gutting either added finally in the tool loop or the root fails both. The third finally, around the structured turn, has no dedicated regression test in this commit, only ad hoc mutation proof, because it guards a path (outputFormat without tools) the two required tests do not exercise. Found by Bugbot on the langchain-messages layer, then found here by audit.

…abandoned A CancelledError never enters except Exception, so the streaming teardown always ran its abandonment path and marked launchdarkly.stream.abandoned. A consumer that stops reading did abandon the stream, and that word is right for it. A timeout did not: nothing chose to stop reading, the run was cancelled underneath the consumer. The blocking path in this handler already reports launchdarkly.run.cancelled for that, so the two paths disagreed about the same event. The existing test for a tool cancelled mid-flight asserted stream.abandoned, which is what the defect looked like from inside. It now asserts run.cancelled. The consumer-break test still asserts stream.abandoned and needed no change. No new attribute. Both keys already exist and are in the vocabulary lock. Found by Bugbot on the openai-agents layer, then found here by audit.

The write ran whatever the capture setting was. set_output_content_attributes is a no-op without capture, but building the SpanMessage and SpanMessagePart is not, so a caller who turned content capture off still paid to assemble a transcript nothing would read. Every equivalent site on the blocking path already carries this guard, with a comment saying why. This was the one that did not, so the file disagreed with itself. full_output_str stays outside the guard: the done event returns it whatever the setting is. Harmless today, because that string is always a string. It stops being harmless the moment the streaming path handles structured output, which is what the blocking path's comment is warning about. One test, asserting the call is not made at all rather than that it did nothing. Removing the guard fails it. Found by Bugbot on this PR.

apucacao · 2026-08-14T19:57:29Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 178891a. Configure here.}

apucacao mentioned this pull request Aug 11, 2026

feat(langchain-agents)!: emit invoke_agent, chat and execute_tool spans #35

Draft

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py Outdated

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from 6e89feb to 36f4e28 Compare August 11, 2026 20:43

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py Outdated

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from 36f4e28 to 9131182 Compare August 11, 2026 21:01

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py Outdated

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py Outdated

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from 9131182 to c3de18e Compare August 11, 2026 21:18

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py Outdated

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py Outdated

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from c3de18e to 3b4ac11 Compare August 12, 2026 17:23

cursor Bot reviewed Aug 12, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from 3b4ac11 to 777418b Compare August 12, 2026 17:45

cursor Bot reviewed Aug 12, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from 777418b to 115c02f Compare August 12, 2026 18:02

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from 115c02f to 9b5d9b2 Compare August 12, 2026 18:07

cursor Bot reviewed Aug 14, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from fd6c38e to d9c07c9 Compare August 14, 2026 19:30

cursor Bot reviewed Aug 14, 2026

View reviewed changes

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from d9c07c9 to 79d1afc Compare August 14, 2026 19:39

cursor Bot reviewed Aug 14, 2026

View reviewed changes

Comment thread packages/langchain-messages/src/launchdarkly_ai_langchain_messages/handler.py Outdated

apucacao added 16 commits August 14, 2026 15:55

apucacao force-pushed the ag/py-telemetry-langchain-messages branch from 79d1afc to 178891a Compare August 14, 2026 19:56

cursor Bot reviewed Aug 14, 2026

View reviewed changes

Uh oh!

Conversation

apucacao commented Aug 11, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Both provider attributes were wrong, in different ways

Other changes

Breaking change

Where this sits

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

Uh oh!

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

Uh oh!

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 11, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

apucacao commented Aug 12, 2026

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

Uh oh!

apucacao commented Aug 14, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

apucacao commented Aug 11, 2026 •

edited by cursor Bot

Loading