[bot] Merge master/72858ca7 into rel/dev - #1805
Merged
Merged
Conversation
…LP root spans observe() resolves the item's dataset and exports one experiment root span per (dataset item, run) over OTLP instead of creating a dataset-run item, so a run is assembled from the langfuse.experiment.* attributes gd-eval writes on its own span. The span is timed from the gen-ai turn it describes and carries the gen-ai trace id, latency and cost as observation metadata. score_safe writes each score to every destination the yielded ScoreTarget names, sending observationId only for the experiment span so the gen-ai write is the call every client already accepts. RunTraceContext carries the pinned window and the item's question so a skill's scoring block keeps calling ctx.observe(pt, run_idx), and the api.dataset_run_items compat shim exports the same span, raising LookupError for an item Langfuse does not know. The trace-version upsert goes away: v4 observations are immutable, so the model version moves onto gd-eval's span as langfuse.version. jira: trivial risk: low
…riment spans from every agentic kind Each of the eight agentic skill modules now hands its evaluated question to submit_trace_scoring as item_input, and passes the run's conversation id and a compact per-run result dict into ctx.observe as conversation_id and output, so experiment spans carry what was asked and what the run produced instead of only a trace id. Adds a parametrized AST guard in test_trace_linker.py asserting every submit_trace_scoring call carries item_input. jira: trivial risk: low
…s over OTLP LangfuseSink.log_item now exports one OTLP experiment root span per evaluated item and posts its four scores against that span's trace and observation id, replacing the old ingestion batch (trace-create plus score-create events) and the dataset-run-items call. The sink builds its HttpxLangfuseClient through a new keyword-only transport parameter so tests inject an httpx.MockTransport instead of patching httpx.Client. When a dataset item cannot be resolved, the span is still exported and scored, just without experiment attributes. jira: trivial risk: low
…st a fake server Six end-to-end tests drive the real client, sink and CLI against the in-process fake Langfuse server over HTTP, asserting the whole request sequence a run produces: poll /v2/observations for the conversation, look the dataset item up, export one experiment root span over OTLP, and score both that span and the gen-ai trace. Covers the inline agentic path, the batched path through run_agentic_items, a dataset item Langfuse does not know, the single-shot sink under the CLI, a refused span export and a rate-limited score. jira: trivial risk: low
…aset-run code paths Drops the dataset-run-items compat shim from the client and the legacy /traces, /ingestion and /dataset-run-items routes from the fake server, so nothing in the package names an endpoint Langfuse v4 does not serve. The CLI's local-dataset warning and the skip-switch test now describe the dataset-item lookup that actually runs. A BOOLEAN score is coerced to 1.0/0.0 by its dataType rather than its Python type, so the sink's int 1/0 serialises as a float, and the sink stamps LANGFUSE_TRACING_ENVIRONMENT and leads its tags with "gd-eval" exactly as the agentic path does. jira: trivial risk: low
Documents the run as one Langfuse experiment whose items are traces with an experiment-carrying root span, the four REST endpoints the package speaks and the 2026-11-16 Cloud cutover, and corrects the env vars to LANGFUSE_BASE_URL with LANGFUSE_HOST as the legacy alias. The local-dataset and skip-switch notes describe what actually happens now, and the AGENTS.md architecture row names every module under core/langfuse. jira: trivial risk: low
…den the Langfuse score path Brings back api.dataset_run_items.create on the httpx client, exporting one experiment root span for external callers on the dataset-run vocabulary. The score path answers a cancelled drain before it reaches HTTP rather than sleeping through create_score's retries, an unfiltered trace read pages at the API maximum and twice as deep, and observe() documents the ScoreTarget contract for callers that write scores with create_score directly. jira: trivial risk: low
feat(gooddata-eval): link runs as Langfuse experiments via OTLP root spans
yenkins-admin
requested review from
hkad98,
lupko and
pcerny
as code owners
September 10, 2026 09:32
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## rel/dev #1805 +/- ##
===========================================
+ Coverage 82.13% 82.27% +0.13%
===========================================
Files 282 282
Lines 20286 20326 +40
===========================================
+ Hits 16662 16723 +61
+ Misses 3624 3603 -21 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🚀 Automated PR to perform merge from master into rel/dev with changes up to 72858ca (created by https://github.com/gooddata/gooddata-python-sdk/actions/runs/34461130544).