Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions src/pages/docs/release-notes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,65 @@
description: "Latest Future AGI release notes covering new features, improvements, and bug fixes across datasets, evaluations, simulation, and observability products."
---

## Week of 2026-06-18

<div class="mb-12 pb-8 border-b border-[var(--color-border-subtle)] last:border-b-0">

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Custom Attribute Filter Dropdown Now Populates:** In some cases, the custom attribute dropdown in the dashboard was empty for projects with many unique span attributes. It now lists all available attributes.

- **Saved View Column Selections Now Persist:** In some cases, deselecting a column in a saved Observe view immediately snapped back to the saved state. Column visibility changes now stick for the session.

- **API Key Expiry Enforced Across All Gateway Components:** Expired API keys are now rejected consistently across all gateway entry points, including components that previously accepted synced keys past their expiration date.

</div>

## Week of 2026-06-11

<div class="mb-12 pb-8 border-b border-[var(--color-border-subtle)] last:border-b-0">

<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Few-Shot Examples for LLM Judge:** When configuring a custom LLM evaluator, you can now attach a dataset of input/output examples. The judge uses these as few-shot references during scoring, producing more consistent and calibrated results across your eval runs.

Check warning on line 26 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L26

Did you really mean 'eval'?

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Trace List Loads Reliably for Large Accounts:** In some cases, the trace list failed to load for accounts with a high number of distinct users. This has been resolved.

- **Annotation Filters in Eval Tasks Now Work for Voice Call Projects:** In some cases, eval tasks using annotation filters on voice call rows returned no results. Annotation filters now correctly match annotations across all project and row types.

Check warning on line 32 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L32

Did you really mean 'Eval'?

Check warning on line 32 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L32

Did you really mean 'eval'?

- **Trace View No Longer Crashes on Large Images:** In some cases, opening a trace containing a span with an embedded image larger than 50MB caused the page to fail to load. This no longer occurs.

- **Eval Task Filter Conditions Show Readable Column Names:** In some cases, eval task filter conditions displayed internal identifiers instead of the column's display name. Filters now show human-readable names.

Check warning on line 36 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L36

Did you really mean 'Eval'?

Check warning on line 36 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L36

Did you really mean 'eval'?

- **App No Longer Crashes With Browser Translation Enabled:** In some cases, using a browser's built-in translation feature (such as Chrome Translate or Edge Translate) caused a page crash. This no longer occurs.

- **Eval Template Deletion Cleans Up Dataset Columns:** Deleting an eval template now removes the associated eval columns and cells from your datasets automatically.

Check warning on line 40 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L40

Did you really mean 'Eval'?

Check warning on line 40 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L40

Did you really mean 'eval'?

Check warning on line 40 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L40

Did you really mean 'eval'?

</div>

## Week of 2026-06-04

<div class="mb-12 pb-8 border-b border-[var(--color-border-subtle)] last:border-b-0">

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Revamped Tracing Filters:** Filters across the Trace and Span views have been rebuilt with a more consistent and reliable foundation. Text-based filters now handle case differences correctly, and the filter picker accurately resolves metric names across all namespaces.

Check warning on line 50 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L50

Did you really mean 'namespaces'?

- **Call Recording on Error Feed Overview:** For simulation projects, the Error Feed cluster overview now shows the call recording player instead of the agent flow section. You can listen to the call directly while reviewing the error cluster without switching views.

- **Customer Agent Task Completion Evaluator:** A new built-in system evaluator is now available: customer_agent_task_completion. It checks whether your agent fully completed the assigned task in a customer interaction, returning a Pass or Fail result. It takes your agent's prompt and the full conversation as inputs. This is especially useful in Simulation, we recommend adding it to your simulation eval runs to automatically verify task completion across scenarios.

Check warning on line 54 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L54

Did you really mean 'customer_agent_task_completion'?

Check warning on line 54 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L54

Did you really mean 'eval'?

- **Pass/Fail Now Shown Correctly in Trace Eval Drawer:** Pass/Fail evaluations like PII were displayed as a percentage score in the trace eval drawer, which read as a confidence level rather than a verdict. They now render as Pass or Fail.

Check warning on line 56 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L56

Did you really mean 'Eval'?

Check warning on line 56 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L56

Did you really mean 'eval'?

- **Eval Save and Test Require Valid Template Variables:** The Save and Test buttons in an eval's instructions editor are now disabled until the instructions contain at least one valid template variable. A tooltip explains why the buttons are inactive, and the check applies to both the create and edit flows.

Check warning on line 58 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L58

Did you really mean 'Eval'?

Check warning on line 58 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L58

Did you really mean 'eval's'?

Check warning on line 58 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L58

Did you really mean 'tooltip'?

- **Required Eval Field Mappings No Longer Dropped:** In some cases, creating a system eval failed because required field mappings were silently removed from the payload during the setup flow. Required mappings are now preserved and validated before submission.

Check warning on line 60 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L60

Did you really mean 'Eval'?

Check warning on line 60 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L60

Did you really mean 'eval'?

- **Composite Eval Test Run on Tasks Fixed:** Running a test on a composite eval from the Eval Task view was failing, preventing you from verifying evals on a single test row before running them across all entries. This has been resolved.

Check warning on line 62 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L62

Did you really mean 'Eval'?

Check warning on line 62 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L62

Did you really mean 'eval'?

Check warning on line 62 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L62

Did you really mean 'Eval'?

Check warning on line 62 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L62

Did you really mean 'evals'?

- **Custom Code Eval Parameters Now Apply:** Parameters passed via the SDK when running custom code evals were being ignored for some cases. They now apply correctly.

Check warning on line 64 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L64

Did you really mean 'Eval'?

Check warning on line 64 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L64

Did you really mean 'evals'?

- **Legacy Observe Tabs and Charts UI Removed:** The old Charts UI and legacy tab bar were still appearing on Tracing tabs after the charts revamp. The outdated interface is now fully removed so only the updated UI is shown.

Expand All @@ -33,7 +71,7 @@

- **Save View Button Visible and Tab Names Truncate Cleanly:** The Save View button was nearly invisible in dark theme due to low contrast. Long view names and tab labels also overflowed. Both issues are fixed: the button is clearly visible and long names truncate at the boundary.

- **Agent Graph No Longer Shows a Blank Screen for Voice Bots:** Opening the Agent Graph for a voice bot trace showed a blank screen with no explanation. Voice projects now default to the appropriate graph view, and unsupported tabs show a tooltip explaining why they are unavailable.

Check warning on line 74 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L74

Did you really mean 'tooltip'?

- **Tracing Graph Full Screen Now Works:** The full-screen button on the trace agent graph and path views was not functioning. Both views now open in browser full screen correctly.

Expand All @@ -43,17 +81,17 @@

- **Task Status Updates Without a Page Refresh:** In some cases, task status in the list stayed stale until you refreshed the page. The task list now polls automatically while rows are in progress, so statuses update on their own.

- **Full Variable Names Visible on Hover in Mapping:** Variable names in the task screen's variable mapping column were truncated with no way to read the full name. Hovering over a column key now shows the full variable name in a tooltip.

Check warning on line 84 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L84

Did you really mean 'tooltip'?

- **Empty Dataset Cells No Longer Show as Objects in Eval Mapping:** In some cases, empty cells from a dataset appeared as a raw object in the eval variable mapping step instead of showing as blank. Empty cells are now displayed correctly.

Check warning on line 86 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L86

Did you really mean 'Eval'?

Check warning on line 86 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L86

Did you really mean 'eval'?

- **Evals Skip Instead of Failing When Required Attributes Are Missing:** When a span was missing a required mapped attribute, the eval was incorrectly marked as Failed. Evals are now skipped for those spans, keeping your pass and fail metrics accurate.

Check warning on line 88 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L88

Did you really mean 'Evals'?

Check warning on line 88 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L88

Did you really mean 'eval'?

Check warning on line 88 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L88

Did you really mean 'Evals'?

- **Removed Member No Longer Sees Indefinite Loading on Login:** In some cases, an account that had been removed from an organization saw a loading state persist indefinitely after attempting to log in, requiring a page refresh to see the correct message. The page now resolves correctly without a refresh.

- **Show More in Error Details Now Works:** In some cases, the Show More button in the error details section was not functioning. It now expands correctly. The error localizer also no longer runs for evals that already passed.

Check warning on line 92 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L92

Did you really mean 'localizer'?

Check warning on line 92 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L92

Did you really mean 'evals'?

- **Output Type Locked After Eval Creation:** Once an evaluation is created, the output type can no longer be changed. A tooltip now explains this directly in the interface so the restriction is clear.

Check warning on line 94 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L94

Did you really mean 'Eval'?

Check warning on line 94 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L94

Did you really mean 'tooltip'?

</div>

Expand All @@ -63,23 +101,23 @@

<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Perplexity Sonar Models Now Available for Evaluations:** You can now use Perplexity's full Sonar model family (sonar, sonar-pro, sonar-reasoning, sonar-reasoning-pro, and sonar-deep-research), including the Agent API for agentic workflows with built-in web search, when running evaluations. Contributed by the Perplexity team. [PR #650](https://github.com/future-agi/future-agi/pull/650).

Check warning on line 104 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L104

Did you really mean 'agentic'?

- **Use System Prompt as Context in Evals:** You can now leverage your agent's system prompt as context when running system and custom evals, giving you a more complete view of how your prompts affect model outputs.

Check warning on line 106 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L106

Did you really mean 'Evals'?

Check warning on line 106 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L106

Did you really mean 'evals'?

- **New API: Access Eval Task Data Across Two Axes:** Two new API endpoints are now live. You can access eval task results via API in two ways: a per-evaluator summary (pass rates, average scores, and choice distributions across the full task) and a per-span breakdown (each evaluator's result for every individual span). Both support optional date range filtering.

Check warning on line 108 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L108

Did you really mean 'Eval'?

Check warning on line 108 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L108

Did you really mean 'eval'?

Check warning on line 108 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L108

Did you really mean 'evaluator's'?

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Eval Results in Observe Now Display Correctly:** In some cases, eval results in Trace Observe were not displaying correctly. This has been resolved and results now appear as expected.

Check warning on line 112 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L112

Did you really mean 'Eval'?

Check warning on line 112 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L112

Did you really mean 'eval'?

- **Eval Type Now Saved Correctly When Creating New Evaluations:** When creating an LLM-as-judge or Code evaluation, the selected type was previously always saved as Agent. The correct eval type is now preserved in all cases.

Check warning on line 114 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L114

Did you really mean 'Eval'?

Check warning on line 114 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L114

Did you really mean 'eval'?

- **Eval Experience Improvements Across the Platform:** A set of improvements to the eval creation and review experience: linking directly to a specific eval version now opens on that version; long task error messages collapse to a one-line summary with a Show more toggle; results no longer show stale data when switching between dataset, tracing, and simulation panels; variable highlighting in the prompt editor reflects which variables are actually mapped; clicking an execution row now opens that specific run rather than always opening the latest; and the ground-truth embedding status now updates in realtime, with no page refresh needed.

Check warning on line 116 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L116

Did you really mean 'Eval'?

Check warning on line 116 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L116

Did you really mean 'eval'?

Check warning on line 116 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L116

Did you really mean 'eval'?

Check warning on line 116 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L116

Did you really mean 'realtime'?

- **Sessions View from the Users Tab No Longer Times Out:** When navigating to the Sessions view from the Users tab, the page could get stuck on a loading screen or time out. Sessions now loads reliably from that entry point.

- **Composite Evals No Longer Accept Other Composites as Children:** When building a composite evaluation, the child picker now only lists individual, non-composite evaluators. Previously, composite evals could be selected as children, which produced unexpected results.

Check warning on line 120 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L120

Did you really mean 'Evals'?

Check warning on line 120 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L120

Did you really mean 'evals'?

- **Usage and Billing Page Display Accuracy Improved:** Several display issues on the Usage and Billing page have been fixed: AI credits were showing incorrect units, time period labels on usage cards were inaccurate, chart axis labels showed duplicates or mixed formatting, and the pricing tier table now includes column headers and correct unit labels.

Expand All @@ -93,13 +131,13 @@

- **Nested Variable References Now Work in LLM-as-Judge Templates:** Variables that reference nested properties using dot notation were not rendering correctly in custom prompt evaluator templates. They now resolve and render as expected.

- **Fix with Falcon Now Only Appears on Failing Evals:** The Fix with Falcon option previously appeared on both passing and failing eval rows. It now only appears on evals that are failing, not on every row.

Check warning on line 134 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L134

Did you really mean 'Evals'?

Check warning on line 134 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L134

Did you really mean 'eval'?

Check warning on line 134 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L134

Did you really mean 'evals'?

- **Resuming a Completed Eval Task Now Shows a Clear Message:** In some cases, clicking Resume on a task that had already finished showed a raw error. It now shows a clear message indicating the task may have already completed.

Check warning on line 136 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L136

Did you really mean 'Eval'?

- **Instruction Validation Errors Now Visible When Creating Evals from Sessions:** When adding an LLM-as-judge evaluation from the Sessions view, validation errors on the instructions field were not shown, causing saves to silently fail. Error messages now appear inline as expected.

Check warning on line 138 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L138

Did you really mean 'Evals'?

- **Eval and Task List Chips Now Have Hover Feedback and a Stable Popover:** Chips in the Tasks and Evals list had no hover state, and the popover showing additional items closed as soon as the cursor moved toward it. Chips now darken on hover and the popover stays open as the cursor moves into it.

Check warning on line 140 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L140

Did you really mean 'Eval'?

Check warning on line 140 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L140

Did you really mean 'Evals'?

- **Task Usage Table Columns No Longer Get Cut Off:** The Task Usage table was clipping right-side columns. The table now scrolls horizontally so every column stays visible.

Expand All @@ -111,11 +149,11 @@

<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Composite Evals Now Work Across Trace and Sessions:** You can now run composite evaluations against traces and sessions, not just individual spans. This lets you measure performance across multi-step conversations and grouped interactions in a single evaluation run.

Check warning on line 152 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L152

Did you really mean 'Evals'?

- **Custom Evals Run With Partial Inputs:** Custom evaluations now proceed even when some input fields are missing from your trace data, returning results with a warning indicator instead of failing entirely. System evaluators continue to require all inputs before running.

Check warning on line 154 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L154

Did you really mean 'Evals'?

- **Custom Variable Paths in Eval Task Mappings:** When configuring an eval task, you can now type any custom attribute path from your trace data rather than choosing from a fixed list. This gives you full flexibility when mapping trace fields to eval inputs, including deeply nested attributes.

Check warning on line 156 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L156

Did you really mean 'Eval'?

Check warning on line 156 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L156

Did you really mean 'eval'?

Check warning on line 156 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L156

Did you really mean 'eval'?

- **Dynamic API Columns Support Nested Field Paths:** When configuring a dynamic API column, you can now use dot notation to reference nested fields in the API response (for example, result.score.value). This lets you pull specific values from deeply structured API responses without needing to flatten them first.

Expand All @@ -131,15 +169,15 @@

- **Voice Recordings Now Load Correctly from Error Feed:** Voice traces opened directly from the Error Feed were showing no recording even when one existed. Recordings now load correctly when viewing a voice trace from the Error Feed.

- **Errored Evals Now Show a Clear Error Indicator:** Evaluations that encountered an error were previously shown as a blank dash in the trace and voice drawers, making it hard to tell that something had gone wrong. They now show a clear Error indicator so the status is immediately visible.

Check warning on line 172 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L172

Did you really mean 'Errored'?

Check warning on line 172 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L172

Did you really mean 'Evals'?

- **Filter Picker Returns Correct Results for Matching Metric Names:** In some cases, when two metrics shared the same name across different namespaces, the filter property picker returned incorrect or missing results. The picker now returns the correct metric in all cases.

Check warning on line 174 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L174

Did you really mean 'namespaces'?

- **Eval Results Now Load for All Trace Evaluations:** In some cases, evaluation results for trace-level evaluations were not appearing in the details panel even after the eval completed successfully. These results now display correctly.

Check warning on line 176 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L176

Did you really mean 'Eval'?

Check warning on line 176 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L176

Did you really mean 'eval'?

- **Eval Versions Save and Restore Correctly:** In some cases, saving a new version of an evaluation template or restoring an older one could result in incorrect configuration being applied. Versioning now works reliably, and context settings persist correctly across edits.

Check warning on line 178 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L178

Did you really mean 'Eval'?

- **Eval Creation Saves Correctly When Adding Multiple Evals:** In some cases, the Save & Add button in the eval picker stayed disabled even after completing all required steps, affecting the simulation, Observe, and dataset flows. The button now enables correctly once all steps are done, so you can save and continue adding evaluations without getting stuck.

Check warning on line 180 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L180

Did you really mean 'Eval'?

Check warning on line 180 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L180

Did you really mean 'Evals'?

Check warning on line 180 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L180

Did you really mean 'eval'?

- **Large Traces Are Now Evaluated Completely:** Previously, evaluation prompts for very large traces were cut off well before the end of the trace content, causing evaluations to run on incomplete context. The limit has been significantly increased so that large traces are fully considered during evaluation.

Expand All @@ -157,7 +195,7 @@

- **Tag Input in Trace Detail Now Responds to Clicks on the Enter Icon:** In the trace detail view, clicking the Enter icon in the tag input field now correctly adds the tag. Previously, only pressing the keyboard Enter key would work.

- **Adding Evals with Number Inputs Now Works Reliably:** In some cases, adding an evaluation that included a number input field was causing unexpected errors. These errors no longer occur.

Check warning on line 198 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L198

Did you really mean 'Evals'?

</div>

Expand All @@ -169,41 +207,41 @@

- **Self-Hosted Install:** Setting up Future AGI on your own machine is now straightforward. Clone the repo, cd into the folder, and run bin/install on macOS or Linux. You need Docker, Docker Compose, and at least 8 GB of RAM. That's it.

- **Expanded Context Injection for Evals:** When configuring an eval, you can now choose exactly which context to inject as separate options: span metadata, trace IDs, session data, or call transcripts and recordings. If you already use variables in your eval, you can map context to them as before. If you do not, you can skip that step entirely. When running evals on sessions, the injected context includes depth into the underlying traces and spans, so you can see exactly where gaps occurred. When building an eval, the right context type is pre-selected automatically based on what you are evaluating, so there is less manual setup.

Check warning on line 210 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L210

Did you really mean 'Evals'?

Check warning on line 210 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L210

Did you really mean 'eval'?

Check warning on line 210 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L210

Did you really mean 'eval'?

Check warning on line 210 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L210

Did you really mean 'evals'?

Check warning on line 210 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L210

Did you really mean 'eval'?

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Task Page Filters Apply to Eval Variable Mapping:** Filters you set on the task page now carry through when mapping eval variables. The right traces, spans, and sessions are already scoped for you, so there is no need to search for them manually.

Check warning on line 214 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L214

Did you really mean 'Eval'?

Check warning on line 214 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L214

Did you really mean 'eval'?

- **Image Evals Now Accept URLs:** Image-based evals now accept public HTTP/HTTPS URLs and signed S3 links as inputs. Pass the URL as a string directly in the input field. No file upload or base64 encoding needed. The platform fetches and processes the image server-side before running the eval.

Check warning on line 216 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L216

Did you really mean 'Evals'?

Check warning on line 216 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L216

Did you really mean 'evals'?

Check warning on line 216 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L216

Did you really mean 'eval'?

- **Code Evals More Reliable:** Built-in code evals now run in a consistent execution environment. Eval descriptions have also been updated to accurately reflect current behavior.

Check warning on line 218 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L218

Did you really mean 'Evals'?

Check warning on line 218 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L218

Did you really mean 'evals'?

Check warning on line 218 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L218

Did you really mean 'Eval'?

- **Built-In Validators Improved:** Ten built-in validators have been updated for better accuracy. Email, HTML, SQL, URL, and XML validators now handle a wider range of inputs correctly. Scoring metrics including diff, kappa, word-level error rate, and Meteor score all produce more precise results.

Check warning on line 220 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L220

Did you really mean 'Validators'?

Check warning on line 220 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L220

Did you really mean 'validators'?

Check warning on line 220 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L220

Did you really mean 'validators'?

- **Eval Scores Are Consistent Regardless of Input Formatting:** Eval scores no longer vary based on incidental whitespace in inputs. All inputs are normalized before scoring, and comparing two identical empty values now returns a perfect match.

Check warning on line 222 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L222

Did you really mean 'Eval'?

Check warning on line 222 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L222

Did you really mean 'Eval'?

- **Optional Eval Fields Now Have Sensible Defaults:** Code evals with optional numeric configuration fields now run with their default behavior when those fields are left blank. No configuration is needed unless you want to override the defaults.

Check warning on line 224 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L224

Did you really mean 'Eval'?

Check warning on line 224 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L224

Did you really mean 'evals'?

- **Structured Output Compatibility Improved:** Evals that use LLM as a judge were returning empty results for some nested schema shapes, particularly with models that do not fully support structured output. Both cases are now handled gracefully.

Check warning on line 226 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L226

Did you really mean 'Evals'?

- **Continuous Evals Now Run Reliably at Scale:** Always-on evals with sampling configured now process incoming data consistently over time, regardless of total volume seen so far.

Check warning on line 228 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L228

Did you really mean 'Evals'?

Check warning on line 228 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L228

Did you really mean 'evals'?

- **Task Submission Error Handling Improved:** If an eval configuration fails to save inside the Tasks wizard, you now see a clear error message immediately and can fix it before submitting. The wizard keeps your inputs intact.

Check warning on line 230 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L230

Did you really mean 'eval'?

- **Saved Eval Settings Preserved on Re-edit:** Opening the edit view on a staged eval in Tasks was resetting the model selection and error localizer toggle back to defaults. Both settings are now correctly restored when you reopen an eval for editing.

Check warning on line 232 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L232

Did you really mean 'Eval'?

Check warning on line 232 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L232

Did you really mean 'eval'?

Check warning on line 232 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L232

Did you really mean 'localizer'?

Check warning on line 232 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L232

Did you really mean 'eval'?

- **Session List Loads Faster:** The session list now loads more quickly, so you spend less time waiting.

- **Playground Handles URL Inputs Reliably:** The Playground now processes URL inputs correctly without becoming unresponsive. In some cases, entering a URL as an input would cause the interface to stop responding until the page was refreshed.

- **Observe Task Evals Now Validate Recording URLs:** Task evals in Observe now verify that recording URLs from your provider's webhook are reachable before running. If a URL is inaccessible, you get a clear error message so you can fix it quickly and get accurate results.

Check warning on line 238 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L238

Did you really mean 'Evals'?

Check warning on line 238 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L238

Did you really mean 'evals'?

- **Dot-Notation Now Supports All Nesting Patterns:** You can now use any variable notation style in eval prompts, including dot notation and deeply nested references.

Check warning on line 240 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L240

Did you really mean 'eval'?

- **Only Published Evals Appear in the Eval Drawer:** Draft eval templates created during building or testing no longer show up in the eval selection drawer. Only published evals are visible there.

Check warning on line 242 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L242

Did you really mean 'Evals'?

Check warning on line 242 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L242

Did you really mean 'Eval'?

Check warning on line 242 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L242

Did you really mean 'eval'?

Check warning on line 242 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L242

Did you really mean 'eval'?

Check warning on line 242 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L242

Did you really mean 'evals'?

- **Error Localizer Only Runs When Needed:** The error localizer now skips evals that already passed. It only runs when there is actually something to investigate.

Check warning on line 244 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L244

Did you really mean 'Localizer'?

Check warning on line 244 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L244

Did you really mean 'localizer'?

Check warning on line 244 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L244

Did you really mean 'evals'?

- **Dataset Column Deletion Is Faster:** Deleting columns from a dataset is now more efficient, especially for larger datasets.

Expand All @@ -215,17 +253,17 @@

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Improved Reliability for Voice Observability evals:** Traces, replays, and evals for voice calls now stay fully accessible long after a call ends. Vapi and Retell recording URLs rotate and expire on their own schedules, which causes playback to silently break on older calls. FutureAGI now stores a durable copy of every external recording at ingestion time, so your observability data and eval runs are no longer dependent on provider URL availability.

Check warning on line 256 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L256

Did you really mean 'evals'?

Check warning on line 256 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L256

Did you really mean 'evals'?

Check warning on line 256 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L256

Did you really mean 'Vapi'?

Check warning on line 256 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L256

Did you really mean 'eval'?

- **Error Feed Now Works for Voice Simulation:** Eval-source clusters on VAPI and simulations were not rendering correctly. The Pattern Summary, Trends KPIs, and trace drawer all needed updates to support these project types. All three are now fixed, and clicking a voice trace now opens the voice call drawer as expected.

Check warning on line 258 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L258

Did you really mean 'KPIs'?

- **Datasets: Select-All State Resets When Switching Datasets:** Switching datasets or tabs was preserving the previous selection state, causing incorrect behavior in delete, duplicate, and copy actions. Selection now resets cleanly on every dataset switch.

- **Trace Attribute Drawer: Long Values Are Expandable and Rows Are Easier to Scan:** Long string values in the span attributes drawer were clipped with no way to see the full content. Values are now click-to-expand, and dividers between rows make it easier to tell where one attribute ends and the next begins.

- **Eval List Shows Correct Default Version:** The evals list now correctly shows the current default version for each template instead of always showing V1.

Check warning on line 264 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L264

Did you really mean 'Eval'?

Check warning on line 264 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L264

Did you really mean 'evals'?

- **Zero Eval Scores Now Render:** Eval score rendering was treating a score of 0 as empty. Dataset grids, eval logs, and datapoint drawers now correctly display zero scores.

Check warning on line 266 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L266

Did you really mean 'Eval'?

Check warning on line 266 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L266

Did you really mean 'Eval'?

Check warning on line 266 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L266

Did you really mean 'eval'?

Check warning on line 266 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L266

Did you really mean 'datapoint'?

- **j/k Navigation Shortcuts No Longer Swallow Text Input:** The j and k row navigation shortcuts were intercepting keystrokes globally, blocking you from typing those letters into comment fields and text inputs in the detail panel. These shortcuts now correctly yield to focused text inputs.

Expand All @@ -233,7 +271,7 @@

- **Workspace Invite Fixed for Existing Users:** In few cases, existing org members invited to a new workspace were not receiving the invitation email and could not see the new workspace in their list. The invite flow now correctly sends the email and grants access uniformly.

- **Eval "Created By" Now Shows Organization Name for Legacy Evals:** Evals without creator metadata were showing "User" in the Created By column. They now fall back to the organization display name, and filtering by creator also matches on organization name.

Check warning on line 274 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L274

Did you really mean 'Eval'?

Check warning on line 274 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L274

Did you really mean 'Evals'?

Check warning on line 274 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L274

Did you really mean 'Evals'?

</div>

Expand All @@ -245,7 +283,7 @@

- **Jinja2 Template Support in Prompt Editor and Agent Playground:** You can now write prompts using Jinja2 syntax `{% if %}`, `{% for %}`, filters, and other Jinja2 constructs alongside the existing Mustache `{{ }}` format. A new Template Format dropdown lets you switch between Mustache and Jinja2 in the prompt workbench, run prompt view, and agent playground node forms. The backend renders Jinja2 templates safely, and variable extraction for the inputs panel works correctly in both modes.

- **Annotation and Eval Metrics as Columns When Adding to Dataset:** When adding traces to a dataset from the trace drawer, you can now include annotation scores and eval metric results as dataset columns. Select the metrics you want during the "Add to Dataset" flow and they'll be carried over as column values on each row, letting you capture quality signals directly in the dataset without a separate export step.

Check warning on line 286 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L286

Did you really mean 'Eval'?

Check warning on line 286 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L286

Did you really mean 'eval'?

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

Expand All @@ -253,7 +291,7 @@

- **Voice Call Talk-Time Split Fixed:** Talk-time percentages in the voice call drawer were showing a blank for certain call types. Per-role totals now calculate correctly across all transcript formats.

- **Prompt Workbench Eval Delete Fixed:** Deleting an evaluation run in the Prompt Workbench now succeeds for all types of prompts. Previously, for specific cases it would show a failure toast and leave the eval in the list.

Check warning on line 294 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L294

Did you really mean 'Eval'?

Check warning on line 294 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L294

Did you really mean 'eval'?

- **Annotation Queue CSV Export Now Works:** You can now export annotation queue data as CSV directly from the analytics tab. The Export CSV button is fully functional and ready to use.

Expand All @@ -275,19 +313,19 @@

- **Error Feed:** A new feed that puts all your AI errors, hallucinations, and pipeline issues in one place. Issues are scanned and scored automatically as new traces come in, and you can run a Deep Analysis on any issue to get a ranked list of likely causes and suggested fixes. For each issue, you get a quick summary of what went wrong, examples of traces that failed compared to ones that worked, an agent flow diagram, and a trend chart. You can triage each issue by setting its status, severity, and assignee, create a Linear ticket in one click, and if a fixed issue comes back, it is automatically flagged as a regression. The trend chart also marks when each release went out, so you can spot the release that likely caused the problem.

- **Observe Revamp:** The Tracing experience has been rebuilt around saved views, stronger search, and a more connected layout. Filter the list using natural language with Ask AI, or build queries with Basic and Query modes. Save custom views (filters, columns, sort, density) and switch between them, with compacted layouts for quicker scanning. Search across traces, spans, and agent flow, and view the full agent / graph flow to understand how your agent is moving between steps. You can run evals or add traces to a dataset right from the list. Navigation is now consistent across Trace, Sessions, and Users: prev/next, group by, view trace, view sessions, and replay all route to the right place, and filters carry over between tabs.

Check warning on line 316 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L316

Did you really mean 'evals'?

- **Experiments Revamp:** The Experiments flow has been rebuilt from inside a dataset. You can now name your experiment, pick its type (LLM, TTS, STT, or Image), and add the prompts and agents you want to compare (each with version selection and the option to stack multiple models). Running experiments on agents is fully supported. Add evaluations from the same picker as the Evals page, and optionally pick a column from your dataset to compare results against. The Experiments list shows status, model count, and eval count per run, so you can monitor progress, edit experiments, rerun, or add more evaluations anytime.

Check warning on line 318 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L318

Did you really mean 'Evals'?

Check warning on line 318 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L318

Did you really mean 'eval'?

- **Evaluations Revamp:** We rebuilt the Evals page. Variable mapping is much easier now, with the relevant data points shown right next to the field you are mapping. There is a new test area on the same page where you can try your eval on custom inputs, a dataset, a real trace, or a simulation before saving. You can also bundle multiple evals together (composite evals), and write evals in Python or JavaScript. Evals can now connect to external tools and the internet to enhance their judgements, and you can configure how detailed the explanation should be. The list has filter chips for use cases like RAG, Hallucination, Conversation, Code, PDF, Text, and Safety, plus trend charts and error rates per eval.

Check warning on line 320 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L320

Did you really mean 'Evals'?

Check warning on line 320 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L320

Did you really mean 'eval'?

Check warning on line 320 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L320

Did you really mean 'evals'?

Check warning on line 320 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L320

Did you really mean 'evals'?

Check warning on line 320 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L320

Did you really mean 'evals'?

Check warning on line 320 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L320

Did you really mean 'Evals'?

Check warning on line 320 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L320

Did you really mean 'eval'?

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **AWS Bedrock added to Agent Command Center Gateway:** AWS Bedrock is now available as a provider in the Agent Command Center Gateway, alongside OpenAI, Anthropic, Google (Gemini), Azure OpenAI, Cohere, Groq, Together AI, Fireworks AI, Mistral AI, and Custom/Self-hosted.

Check warning on line 324 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L324

Did you really mean 'Groq'?

- **Tighter guardrails:** Guardrail configuration shows the full keyword blocklist setup, and guardrails reliably trigger when a score crosses the limit you set.

- **Command Center is more reliable across providers:** A set of provider-specific behaviours have been tightened up: GPT-5 routing, multimodal Gemini handling, full-length session IDs, cleaner auth errors for invalid keys, default cost tracking, cache TTL retention, and immediate webhook delivery logs.

Check warning on line 328 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L328

Did you really mean 'multimodal'?

</div>

Expand Down Expand Up @@ -337,15 +375,15 @@

<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Dashboards:** Create custom dashboards to track agent performance across eval scores, system metrics, cost, and experiment progress in 1 central place. All the data that was previously scattered across datasets, eval logs, Observe, and experiments is now queryable in one place. Add widgets, filter by agent or time range, and compare performance over time.

Check warning on line 378 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L378

Did you really mean 'eval'?

Check warning on line 378 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L378

Did you really mean 'eval'?

Check warning on line 378 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L378

Did you really mean 'queryable'?

- **Falcon AI:** A context-aware AI assistant embedded in the platform. It picks up the context of whatever page you are on, so you can ask questions or trigger actions directly against your current data. Supports trace debugging, simulation, eval creation, dataset building.

Check warning on line 380 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L380

Did you really mean 'eval'?

- **MCP Server:** FutureAGI now ships an MCP server that you can connect to your IDE. Supported clients include Cursor, Claude Code, VS Code, Claude Desktop, and Windsurf. Once connected, your coding assistant has access to your evaluations, datasets, experiments, traces, and prompts. You can also configure which tool groups are available to connected clients from the settings page.

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Eval not added when using a different column for mapping in run optimization:** Fixed a bug where selecting a different column for eval mapping during run optimization would silently drop the eval instead of adding it.

Check warning on line 386 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L386

Did you really mean 'Eval'?

Check warning on line 386 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L386

Did you really mean 'eval'?

Check warning on line 386 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L386

Did you really mean 'eval'?

- **Annotation queue status not updating correctly:** Fixed the queue status state flow where an already active queue still showed the "Activate" option, allowing it to be activated multiple times.

Expand Down Expand Up @@ -373,17 +411,17 @@

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Skip reasons now shown for evals and CSAT in voice simulate:** Evals and CSAT are now automatically skipped for calls where there was no meaningful conversation or the audio was under 5 seconds. A skip reason is shown directly in the cell so you always know why a particular eval was not scored.

Check warning on line 414 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L414

Did you really mean 'evals'?

Check warning on line 414 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L414

Did you really mean 'Evals'?

Check warning on line 414 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L414

Did you really mean 'eval'?

- **Audio and image output types in Prompt Workbench:** Prompt Workbench now supports audio and image as output types when running prompts. This makes it straightforward to test prompts that generate voice or visual outputs directly from the Workbench.

- **Custom eval names now work correctly across workspaces:** Custom eval lookups via the SDK are now scoped to the correct workspace, fixing an error that some users were seeing when the same eval name existed in multiple workspaces.

Check warning on line 418 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L418

Did you really mean 'eval'?

Check warning on line 418 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L418

Did you really mean 'eval'?

Check warning on line 418 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L418

Did you really mean 'lookups'?

Check warning on line 418 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L418

Did you really mean 'eval'?

- **Full eval explanation now visible in test view:** The explanation output box in the eval test view now grows automatically to show the complete reasoning. Previously the text was getting cut off.

Check warning on line 420 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L420

Did you really mean 'eval'?

Check warning on line 420 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L420

Did you really mean 'eval'?

- **Dataset name now fills in automatically when uploading a file:** When you upload a CSV or JSON file to create a dataset, the name field is now pre-filled using the filename with special characters removed. You can still edit it freely after.

- **Clearer errors when a model does not support your input type:** When running evals with a model that has modality restrictions, you now see a specific message explaining exactly what is not supported instead of a generic error.

Check warning on line 424 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L424

Did you really mean 'evals'?

</div>

Expand Down Expand Up @@ -436,7 +474,7 @@
<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Observability Traces Show Evaluation Data:**
Observability now lets users filter traces by evaluation data. With the **Show Traces with Evals** toggle, users can view only interactions that have been evaluated without manually scrolling through all traces.

Check warning on line 477 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L477

Did you really mean 'Evals'?

- **Workspace Admins Can Access Keys:**
Workspace administrators can now view keys, eliminating dependency on workspace owners for key retrieval and enabling better key management.
Expand All @@ -445,7 +483,7 @@
When clicking **Create Scenarios** from an agent definition, the scenario generation form now pre-fills with agent details, reducing manual data entry and speeding up scenario creation.

- **Reasoning Model Support:**
Added support for reasoning models with nullable **runprompt** parameters, enabling advanced reasoning workflows and more flexible parameter configurations.

Check warning on line 486 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L486

Did you really mean 'nullable'?

Check warning on line 486 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L486

Did you really mean 'runprompt'?

- **Better Scenario Naming:**
Improved naming conventions for auto-generated scenarios, making it easier to identify and distinguish between different test scenarios in large test suites.
Expand All @@ -472,7 +510,7 @@
Made the session history header sticky when there is only one item to view, removing unnecessary tab navigation and simplifying the single-session review experience.

- **Improved Error Handling for API Keys and Prompt Execution:**
Enhanced platform-wide error handling. When API keys are missing or misconfigured, users now see clear, actionable guidance directing them to update their settings. Prompt execution errors also provide specific, helpful details instead of generic messages, making issues easier to understand and resolve quickly.

Check warning on line 513 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L513

Did you really mean 'misconfigured'?

- **Voice Observability Project Deletion Fixed:**
Resolved error handling during voice observability project deletion attempts, ensuring projects delete cleanly or provide clear feedback on deletion status.
Expand All @@ -489,7 +527,7 @@
<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Image Output Support in Datasets and Prompt Workbench:**
Users can now generate and view image outputs directly in Dataset Run Prompt and Prompt Workbench when working with image models. This enables complete multimodal workflows for testing and experimenting with models that generate visual content.

Check warning on line 530 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L530

Did you really mean 'multimodal'?

- **Multiple Image Upload Support in Datasets:**
Users can now upload multiple images to a single dataset column using comma-separated values in JSON or CSV files. This enables more flexible data handling for image-based evaluations and experiments, with full support for accessing and leveraging images in prompt sections across run prompt and experiment workflows.
Expand All @@ -506,7 +544,7 @@
Synthetic data generation performance has been optimized, significantly reducing the time required to create and populate dataset rows. This streamlines dataset creation workflows and enables faster iteration during testing and development.

- **Enhanced Dataset Upload Handling:**
Improved column type detection and validation during JSON and CSV uploads. The system now better handles JSON objects, arrays, empty lists, numeric and boolean values, and datetime formats, resulting in more accurate data inference and fewer upload errors.

Check warning on line 547 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L547

Did you really mean 'datetime'?

- **More Natural Chat Simulation Personas:**
Chat simulation personas now generate more natural, human-like conversations. Personas avoid overly formal patterns (such as repeated **thank you** responses) and produce more realistic conversational flows that better reflect real user interactions.
Expand All @@ -533,7 +571,7 @@
Enhanced HTML parsing and rendering to ensure prompt outputs display with correct formatting and spacing.

- **Better Error Messaging in Error Localizer:**
Error Localizer now provides more actionable and accurate error messages when evaluation failures occur, helping teams diagnose and resolve issues more quickly.

Check warning on line 574 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L574

Did you really mean 'Localizer'?

- **Clearer Optimization Parameters Display:**
Optimization parameters configured before running Fix My Agent are now visible on the results page, providing full transparency into the settings used for each optimization run.
Expand Down Expand Up @@ -642,7 +680,7 @@
Trace tables are now significantly faster with smoother scrolling and improved alignment, enabling quick and comfortable analysis of large volumes of trace data at scale.

- **PDF & Document Preview Across the Platform:**
Uploaded PDFs and documents can now be previewed directly across datasets and experiments, allowing instant verification of file contents without downloading and reducing errors and rework.

Check warning on line 683 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L683

Did you really mean 'PDFs'?

- **Enhanced Audio Player Experience:**
The audio player now loads audio only when the play button is clicked. This reduces table load time, removes lag in audio-heavy views, and makes reviewing voice conversations faster and smoother.
Expand All @@ -662,10 +700,10 @@
You can now add custom instructions while creating scenarios. These inputs influence scenario generation, giving you better control over how scenarios are created.

- **Observe Table Performance Improvements:**
Observe tables are now more stable and performant for large datasets. Simplified table cells improve scrolling, rendering speed, and overall readability.

Check warning on line 703 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L703

Did you really mean 'performant'?

- **Enhanced Eval Mapping with Prompt and Knowledge Base Inputs:**
Eval mapping now supports both prompt-related columns and Knowledge Bases as selectable inputs. This makes evaluation setup clearer, reduces configuration confusion, and enables more accurate, context-aware evaluations across the platform.

Check warning on line 706 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L706

Did you really mean 'Eval'?

- **Fetch Agent Definition from Providers:**
Agent definitions including prompts and description can now be fetched directly from providers like VAPI or Retell using API key and assistant ID. This reduces manual configuration and keeps agent setups in sync.
Expand All @@ -685,8 +723,8 @@
<div class="mb-12 pb-8 border-b border-[var(--color-border-subtle)] last:border-b-0">

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>
- **Filters for Evals in Dataset Summary:**

Check warning on line 726 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L726

Did you really mean 'Evals'?
You can now filter Dataset Summary by specific evaluations. This helps you focus only on relevant evals, and summary charts update automatically based on the selected filters.

Check warning on line 727 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L727

Did you really mean 'evals'?

- **Default Prompt Tokens Update Based on Model Selection:**
In Prompt Workbench, default token limits now update automatically when you change the model. This avoids token mismatch issues and removes the need for manual corrections.
Expand Down Expand Up @@ -741,7 +779,7 @@

<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Smarter Debugging with Actionable Simulation Insights (Fixmyagent):**

Check warning on line 782 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L782

Did you really mean 'Fixmyagent'?
Simulation results now deliver intelligent, context-aware suggestions to resolve both agent-level and infrastructure issues. Developers can quickly identify problems across prompts, model configurations, and runtime setups, with targeted recommendations for faster resolution. Users can also filter simulation calls to view only those with valid suggestions, enabling more focused debugging and faster optimization.

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>
Expand All @@ -757,7 +795,7 @@
<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Documentation Links Added Across Observe:**
Introduced direct documentation links across LLM Tracing, Sessions, Evals & Tasks, Alerts, and Users. Added a tooltip for Scheduled Runs in Evals & Tasks to improve clarity and onboarding.

Check warning on line 798 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L798

Did you really mean 'Evals'?

Check warning on line 798 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L798

Did you really mean 'tooltip'?

Check warning on line 798 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L798

Did you really mean 'Evals'?

</div>
## Week of 2025-12-01
Expand All @@ -773,7 +811,7 @@
Improved the primary graph dropdown for easier metric switching and refined error handling in observation evaluations to deliver clearer and more accurate failure reporting.

- **Prompt Workbench Improvements:**
Prompt Workbench now provides a smoother experience with live WebSocket streaming in Improve Prompt and fixes for Groq model execution. Additional UI refinements include smoother tab interactions, restored metadata visibility, and resolved overflow issues.

Check warning on line 814 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L814

Did you really mean 'Groq'?

- **Fixed Processing of Audio Type:**
Resolved inconsistent parsing of audio URLs that caused errors during audio rendering and experiment execution. Audio inputs now load and process reliably across all workflows.
Expand Down Expand Up @@ -803,7 +841,7 @@
Experiments now load significantly faster during re-runs, reducing wait times and improving responsiveness across iterations.

- **Enhancements in Observe:**
Observe received multiple usability, stability, and backend improvements to deliver a more consistent experience across traces, sessions, and analytics. Updates include sticky filters, clearer pagination, improved table layouts, refined metadata visibility, streamlined pricing logic, improved JSON and payload handling, corrected evaluation log counts, more accurate session ordering, and several data consistency fixes. LLM tracing also now includes clearer copies and tooltips for improved understanding of model transitions and reasoning.

Check warning on line 844 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L844

Did you really mean 'tooltips'?

- **Filters Freezing UI in Observe:**
Fixed an issue where applying filters caused the Observe interface to freeze.
Expand All @@ -826,7 +864,7 @@
<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Support for Custom Voices in Run Prompt and Experiments:**
Developers can now use custom voices from Eleven Labs and Cartesia, enabling fine-grained control over voice style, brand identity, and experiment fidelity.

Check warning on line 867 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L867

Did you really mean 'Cartesia'?

</div>

Expand Down Expand Up @@ -859,7 +897,7 @@
- **Dataset Audio Evaluations Not Working:**
Fixed an issue where dataset audio evaluations would time out for large audio files. Evaluation throughput is now stable across large datasets.

- **Fix Redundant Eval Mapping Issue in Run Test:**

Check warning on line 900 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L900

Did you really mean 'Eval'?
Corrected redundant or inconsistent evaluation mappings to ensure inputs and outputs in Run Test match the expected configuration.

</div>
Expand All @@ -872,11 +910,11 @@

- **Show Reasoning Column in Simulate:** A reasoning column has been added to simulation results, allowing users to view the logic behind evaluation outcomes. This helps teams better interpret model decisions and debug unexpected behaviors.

- **TraceAI Livekit SDK Release:** Support added for tracing Livekit-based agents, enabling visibility into audio events and voice interactions for improved debugging and analysis.

Check warning on line 913 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L913

Did you really mean 'Livekit'?
<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Workbench UI: Hover Tooltip Additions:**

Check warning on line 916 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L916

Did you really mean 'Tooltip'?
Hover-based tooltips have been added across the Workbench interface, providing contextual guidance and reducing confusion while navigating or editing prompts.

Check warning on line 917 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L917

Did you really mean 'tooltips'?

- **General Bug Fixes in Simulate and Observe:**
Resolved several platform stability issues, including validation errors that blocked evaluation configurations from being saved, inconsistent filter behavior in prototype and project views caused by incorrect parameter formatting, and pagination problems on the User Dashboard resulting in more consistent and reliable performance across the platform
Expand All @@ -894,7 +932,7 @@
<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **New TTS Model Integrations for Run Prompt and Experiments:**
Added support for Cartesia, Hume, Neuphonics, and LMNT TTS models, expanding the range of available voices and synthesis characteristics.

Check warning on line 935 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L935

Did you really mean 'Cartesia'?

Check warning on line 935 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L935

Did you really mean 'Neuphonics'?

- **Enhanced Simulation Behaviors and Realism:**
Simulation output now features more natural persona logic, frustration modeling, improved background noise handling, and smoother conversational transitions for more realistic interactions.
Expand Down Expand Up @@ -928,7 +966,7 @@
The Usage Summary experience has been fully redesigned to provide detailed visibility into workspace-level activity. All API call logs across Traces, Observe, Simulation, and Error Analysis now include workspace attribution. A new cumulative usage API provides long-term consumption insights with improved cost and count tracking for financial clarity.

- **New Agent Definition UX with Multi-Step Flow:**
The Agent Definition workflow has been rebuilt into a guided three-step setup—Basic Information, Configuration, and Behaviour. The updated layout improves discoverability, adds a contextual resource panel, and introduces row-level table actions.

Check warning on line 969 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L969

Did you really mean 'discoverability'?

- **Prompt Workbench Revamp:**
The Workbench UI has been redesigned to simplify prompt version management and improve collaboration. Prompt versions now follow a commit-based history model, making it easier to review, compare, and maintain consistency across experiments.
Expand Down Expand Up @@ -1022,7 +1060,7 @@
<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Add Rows in Simulate Scenarios:** Scenario tables can now be expanded with maximum flexibility. Rows can be added manually for precision control, generated intelligently using AI for rapid test case creation, or imported directly from existing datasets to leverage historical data. This enhancement streamlines scenario building and dramatically reduces setup time for complex simulations.
- **Run Evaluations for Completed Test Runs:** New evaluations can now be executed on already completed test runs without rerunning entire simulations, delivering significant time and cost savings. Users can select desired test runs via checkboxes, click Run Evals, and choose specific evaluations to execute. This targeted approach enables efficient resource utilization, faster iteration on evaluation metrics, and flexible experimentation with different criteria.

Check warning on line 1063 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1063

Did you really mean 'Evals'?
- **Agent Definition Version Selection:** Specific Agent Definition Versions can now be selected when creating new test runs and directly from the test run details page. This enhancement provides greater control over testing workflows and ensures reproducibility across experiments, making version comparison seamless and reliable.

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>
Expand All @@ -1037,15 +1075,15 @@
<div class="mt-6 mb-3 text-lg font-semibold">What's New</div>
<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Voice Observability Through Vapi Integration:** Voice interactions are now fully observable within the platform. Assistant call logs from Vapi, including voice simulations, are automatically captured and displayed in your Observe project alongside other project data, enabling comprehensive monitoring and analysis of voice-based interactions.

Check warning on line 1078 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1078

Did you really mean 'Vapi'?

Check warning on line 1078 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1078

Did you really mean 'Vapi'?

- **Eval Groups in Experiment and Optimization:** Evaluation groups can now be configured, created, and applied directly within Experiment and Optimization workflows. This integrated approach reduces workflow friction and accelerates the evaluation setup process.

Check warning on line 1080 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1080

Did you really mean 'Eval'?

<div class="mt-6 mb-3 text-lg font-semibold">Bugs/Improvements</div>

- **Media Visualization in Eval Playground:** Media columns now render actual image and audio content instead of raw URL strings, providing complete context and improved clarity in evaluation results.

Check warning on line 1084 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1084

Did you really mean 'Eval'?

- **Accelerated Learning & Improved Accessibility:** Implemented a View Docs button across all major modules to streamline access to relevant documentation. Additionally, specific documentation links have been added directly to individual Evals, enabling quicker understanding and more efficient usage.

Check warning on line 1086 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1086

Did you really mean 'Evals'?

- **Contextual Flow Analysis Display:** The interface has been streamlined by removing flow analysis views from dataset-based scenarios where they are not applicable, resulting in a cleaner and more intuitive user experience.

Expand All @@ -1061,7 +1099,7 @@

<div class="mt-6 mb-3 text-lg font-semibold">Features</div>

- **Simulate via SDK:** You can now simulate realistic, ultra-low-latency customer calls against your deployed LiveKit agents directly through the SDK. This update enables fully local testing without external dependencies, automatically records high-fidelity WAVs and transcripts over the WebRTC stream, and integrates with AI Evaluation for end-to-end performance evaluation. Developers gain full ownership and flexibility—with self-hosted control, customizable ASR, TTS, and model configurations—while cutting simulation costs by roughly 60–70%.

Check warning on line 1102 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1102

Did you really mean 'WAVs'?

- **Selective Test Rerun in Simulate:** Users now have precise control over simulation testing with the ability to rerun individual calls. You can choose to rerun the complete call with evaluations or re-execute evaluations independently, enabling targeted debugging and validation without requiring full test restarts.

Expand All @@ -1077,7 +1115,7 @@
​​
- **Evaluation Group Management:** Users can now configure and create evaluation groups directly from datasets and simulate, streamlining evaluation setup and saving time.

- **Default evals group:** Access preconfigured evaluation groups for use cases like RAG, computer vision, etc., and save time in evaluation setup.

Check warning on line 1118 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1118

Did you really mean 'evals'?

Check warning on line 1118 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1118

Did you really mean 'preconfigured'?

- **Advanced Simulation Management:** Test executions now auto-refresh with real-time data, giving users instant visibility into ongoing runs. Users can stop simulations at any point to prevent unnecessary calls and costs. Enhanced features include Visual Workflow Tracing to pinpoint agent deviations, Real-Time Test Control to efficiently manage test execution, and Comprehensive Performance Metrics (latency, interruption response time, etc.) for precise agent evaluation and optimization.

Expand Down Expand Up @@ -1107,7 +1145,7 @@

- **Large Dataset Upload Stability:** Improved dataset upload experience by resolving loading issues for large CSV/JSON files, enhancing stability and user visibility.

- **Custom Evaluation Editing Fixes:** Resolved bugs in the Evals Playground to ensure smoother and more reliable editing of custom evaluations.

Check warning on line 1148 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1148

Did you really mean 'Evals'?
- **Group Evaluation UI/UX Improvements:** Refined the user interface and experience when editing group evaluations, making the process more intuitive and consistent.

</div>
Expand All @@ -1119,14 +1157,14 @@
<div class="mt-6 mb-3 text-lg font-semibold">What's New</div>

**Features**
- **Advanced Evaluation Group Management:** Streamline your evaluation workflows with comprehensive CRUD operations for evaluation groups. Create, view, edit, and delete evaluation groups seamlessly, then apply them directly to tasks and prompts for consistent scoring across your AI applications. Enhanced with intelligent popovers that display eval input details, LLM/Knowledge Base dependencies, and linked evaluations during the grouping process.

Check warning on line 1160 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1160

Did you really mean 'eval'?
- **Enhanced Call Management & Audio Controls:** Manage your voice AI testing with the completely revamped Call Details Drawer that displays associated scenarios for each test run. Features a sophisticated multi-channel audio player for separate visualization and playback of assistant and customer audio streams.
- **Flexible Call Recording Downloads:** Export call recordings in multiple formats (Caller Audio, Agent Audio, Mono Audio, Stereo Audio) to match your analysis workflow requirements. Coupled with granular audio field selection in evaluations for precise control over which conversation segments to score and analyze.

**Bugs/Improvements**
- **Enhanced Collaboration Features:** Boost team productivity with collaborator support in prompts, allowing you to add and view team members working on specific prompts. Track prompt ownership with visible Created By fields and organize your work more efficiently with sorting capabilities for sample folders, prompts, and prompt templates.
- **Annotation & Prompt Import Fixes in Dataset:** Enhanced annotation workflows by preventing empty label view selections and resolving prompt overflow issues in Run Experiment interfaces.
- **Filter Issues for Evals Selection:** Bug fix for eval type filters on evaluations drawer across the platform.

Check warning on line 1167 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1167

Did you really mean 'Evals'?

Check warning on line 1167 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1167

Did you really mean 'eval'?

</div>

Expand All @@ -1142,10 +1180,10 @@
- **Enhanced Plans & Pricing Experience:** Navigate pricing options effortlessly with our completely redesigned pricing page featuring interactive plan comparison cards, a dynamic price calculator, and detailed plan breakdowns. The new design provides clear visibility into feature tiers and helps you make informed decisions about your subscription.

**Bugs/Improvements**
- **Enhanced Observability & Dashboard Accuracy:** Resolved filtering issues for User ID across User Details Dashboard and Observe sections. Improved project selector clarity in Observe Eval Task Drawer and fixed workspace-level OTEL trace creation issues for more reliable monitoring.

Check warning on line 1183 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1183

Did you really mean 'Eval'?
- **UI/UX Enhancements:** Streamlined simulation flow interfaces for better user experience and standardized decimal precision across the platform (displaying 2 decimal places for all numeric values).
- **Enhanced Data Visibility in Dataset Summary:** Understand exactly how many data points contributed to your summary results and evaluation metrics, helping with complete transparency.
- **Code Snippet for Running Evals via SDK:** Copy-paste ready terminal commands to run any evaluation without manual configuration by leveraging code snippet on the evals playground.

Check warning on line 1186 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1186

Did you really mean 'Evals'?

Check warning on line 1186 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1186

Did you really mean 'evals'?
- **Unified Design System:** Experience consistent interactions across the platform with our custom DatePicker component, ensuring a polished and cohesive user experience throughout your workflow.

</div>
Expand All @@ -1157,11 +1195,11 @@
<div class="mt-6 mb-3 text-lg font-semibold">What's New</div>

**Features**
- **Comprehensive Annotation Quality Dashboard:** Monitor annotation quality at scale with our centralized analytics dashboard. Track key metrics including annotator agreement rates, completion times, and advanced quality scores (cosine similarity, Pearson correlation, Fleiss' kappa) to ensure your training data meets the highest standards.

Check warning on line 1198 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1198

Did you really mean 'Fleiss'?
- **Enterprise-Grade Multi-Workspace Security:** Deploy with confidence using our complete RBAC framework. Create isolated workspaces, manage team members with full CRUD capabilities (edit, deactivate, resend invitations), and implement role-based access controls that scale with your organization's security requirements.
- **Advanced Observability with Feed Insights:** Gain unprecedented visibility into agent performance with the new Feed Insights tab in the Observe section. Identify failed stages, affected spans, view error cluster events, track user counts, and analyze trend data over time for rapid issue diagnosis and agent optimization.
- **Intelligent Onboarding Navigation:** Experience streamlined onboarding with our redesigned sidebar that prominently highlights the 'Get Started' section until all 7 onboarding steps are completed. This ensures new users follow a structured path to success before transitioning to the regular navigation experience.
- **No Config Evals – Agent Compass for AI Teams:** AI agent developers often struggle to identify performance bottlenecks and system failures across complex execution flows. Traditional evaluation methods and system metrics offer only fragmented, span-level visibility—leaving teams blind to the bigger picture. As a result, diagnosing latency spikes, inefficient prompts, or tool-call failures becomes a time-consuming, manual process. Without actionable, trace-level insights, performance optimization turns reactive, error-prone, and expensive.

Check warning on line 1202 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1202

Did you really mean 'Evals'?

**Bugs/Improvements**
- **Improved Observability Reliability:** Enhanced backend resilience for incomplete span creation scenarios and fixed issues when OpenTelemetry exports fail partially, ensuring complete trace visibility.
Expand All @@ -1174,7 +1212,7 @@
#### What's New

**Features**
- **Add Rows in Evals Tab of Prompt Workbench:** Instantly add new rows with variable values in the evaluations screen, allowing you to generate outputs and evaluate without returning to the Prompt Workbench homepage.

Check warning on line 1215 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1215

Did you really mean 'Evals'?
- **Trace Linked to Prompt Workbench:** View comprehensive performance metrics (latency, cost, tokens, evaluation metrics) for each prompt version linked to traces (and spans) across development, staging, and production environments via the Metrics section in Prompt Workbench.
- **Critical Issue Detection & Mitigation Advice on Datasets:** Get actionable, AI-powered insights with recommendations to improve your agent's performance and accelerate your path to production.
- **Access FAGI from AWS Marketplace:** Sign up or sign in to the FAGI platform via AWS Marketplace and leverage AWS contracts and billing to work with FAGI.
Expand All @@ -1190,7 +1228,7 @@
- **Support for Video in Trace Detail Screen:** Added support for viewing videos in the Trace Details screen.
- **Fixed Scroll Issue in Agent Description Box (Simulation):** Enabled scroll functionality via mouse in the agent description box within the simulation module.
- **Error Handling on Simulation Page:** Improved error handling for low credit balances on the simulation homepage to enhance user experience.
- **Credit Utilization for Error Localizer:** Added visibility of credit utilization for the error localizer in the usage summary screen.

Check warning on line 1231 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1231

Did you really mean 'Localizer'?

Check warning on line 1231 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1231

Did you really mean 'localizer'?
</div>

## Week of 2025-08-19
Expand All @@ -1200,17 +1238,17 @@

**Features**
- **Comparison Summary:** Compare evaluations and prompt summaries of two different datasets now with detailed graphs and scores.
- **Function Evals:** Enable adding and editing function-type custom evals from the list of evals supported by Future AGI.

Check warning on line 1241 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1241

Did you really mean 'Evals'?

Check warning on line 1241 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1241

Did you really mean 'evals'?

Check warning on line 1241 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1241

Did you really mean 'evals'?
- **Edit Synthetic Dataset:** Edit existing synthetic datasets directly or create a new version from changes.
- **Document Column Support in Dataset:** New document column type to upload/store files in cells (TXT, DOC, DOCX, PDF).
- **User Tab in Dashboard and Observe:** Searchable, filterable user list and detailed user view with metrics, interactive charts, synced time filters, and traces/sessions tabs.
- **Displaying the Timestamp Column in Trace/Spans:** Added Start Time and End Time columns in Observe → LLM Tracing and Prototype → All Runs → Run Details.
- **Configure Labels:** Configure system and custom labels per prompt version in Prompt Management.
- **Async Evals via SDK:** Run evaluation asynchronously for long-running evaluations or larger datasets.

Check warning on line 1247 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1247

Did you really mean 'Evals'?

**Bugs/Improvements**
- SDK Codes: Update the SDK codes for columns and rows on create dataset, add rows, and landing dataset page.
- Fixed the editable issue in custom evals form: Incorrect config was displayed on evals page for function evals.

Check warning on line 1251 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1251

Did you really mean 'evals'?

Check warning on line 1251 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1251

Did you really mean 'evals'?

Check warning on line 1251 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1251

Did you really mean 'evals'?
- The bottom section for trace detail drawer disappeared: Dragging the bottom section caused the entire bottom area to disappear; behavior corrected.
- UI screen optimization for different screen sizes.
- Bug fixes for updates summary screen - color, text, and font alignment.
Expand Down Expand Up @@ -1241,12 +1279,12 @@

**Features**
- **Voice Simulation:** New testing infrastructure that deploys AI agents to conduct real conversations with your voice systems, analyzing actual audio, not just transcripts.
- **Edit Evals Config:** Now edit the config (prompt/criteria) for your custom evals via evals playground, but with the restriction of no variable addition.

Check warning on line 1282 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1282

Did you really mean 'Evals'?

Check warning on line 1282 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1282

Did you really mean 'evals'?

Check warning on line 1282 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1282

Did you really mean 'evals'?

**Bugs/Improvements**
- Bug fix for dynamic column creation via Weviate.

Check warning on line 1285 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1285

Did you really mean 'Weviate'?
- Reduced dependencies for TraceAI packages (HTTPS & GRPC).
- Automated eval refinement: Retune your evals in evals playground by providing feedback.

Check warning on line 1287 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1287

Did you really mean 'eval'?

Check warning on line 1287 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1287

Did you really mean 'evals'?

Check warning on line 1287 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1287

Did you really mean 'evals'?
- Markdown now available as a default option for improved readability.
- Support for video (traces and spans) in Observe project.
</div>
Expand All @@ -1257,9 +1295,9 @@
#### What's New

**Features**
- **Edit, Duplicate, and Delete Custom Evals:** Now duplicate, edit, or delete evaluations if they are not in use anymore or logic is outdated.

Check warning on line 1298 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1298

Did you really mean 'Evals'?
- **Bulk Annotation/User Feedback:** Bulk annotate your observe traces with user feedback directly using API or SDK.
- **JSON View for Evals Log:** Access evals log data in JSON format in evals playground.

Check warning on line 1300 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1300

Did you really mean 'Evals'?

Check warning on line 1300 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1300

Did you really mean 'evals'?

Check warning on line 1300 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1300

Did you really mean 'evals'?

**Bugs/Improvements**
- Span name visibility in traces for Observe and Prototype.
Expand All @@ -1285,14 +1323,14 @@
- Fixed audio icon.
- Error handling for network connection issues.
- Bug fixes for prompt workbench versioning issues.
- Changed the color mapping for deterministic type evals.

Check warning on line 1326 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1326

Did you really mean 'evals'?
- Updated loaders for evals playground.

Check warning on line 1327 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1327

Did you really mean 'evals'?
- Pagination fix in Observe.
- Added clear functionality in add to dataset column mapping fields in Observe.
- Clear graph property when Observe changes; fixed thumbs down icon not rendering.
- Generate variable bug fix in prompt workbench.
- Experiment page break on content tab switch.
- Fixed the created_at 30-day filter on evals log section.

Check warning on line 1333 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1333

Did you really mean 'created_at'?

Check warning on line 1333 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1333

Did you really mean 'evals'?
</div>

## Week of 2025-07-14
Expand All @@ -1301,7 +1339,7 @@
#### What's New

**Bugs/Improvements**
- Prevented overscroll in X direction for entire platform.

Check warning on line 1342 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1342

Did you really mean 'overscroll'?
- Glitch after refreshing while generating sample data.
- Error message update for doc uploads and save button status for doc upload.
- Variable auto-population issue in compare prompt for multiple versions.
Expand Down Expand Up @@ -1338,10 +1376,10 @@
**Bugs/Improvements**
- Improved error handling in prompt versioning.
- Removed unnecessary keys from evaluation outputs.
- Better handling of required keys to column names in add_evaluation in dataset.

Check warning on line 1379 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1379

Did you really mean 'add_evaluation'?
- Removed TraceAI code from FutureAGI SDK - experiment rerun fix.
- SSO login issues.
- Eval ranking fixes.

Check warning on line 1382 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1382

Did you really mean 'Eval'?
- Fixed sizing and view issue in dataset when row size is adjusted.
- Fixed sidebar item not showing active style when child page is active globally.
- Edit integer type has red background in edit field.
Expand All @@ -1361,25 +1399,25 @@
- Prototype / Configure project.
- Trace details view for Observe/Prototype.
- Allow search in dataset.
- Run insights view - evals (deployed without the error modal part).

Check warning on line 1402 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1402

Did you really mean 'evals'?
- Improved user flow for synthetic data creation with "best practices" for each input.
- Add to dataset flow from Prototype.
- API for Gmail account signup.

Check warning on line 1405 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1405

Did you really mean 'signup'?
- Enabling search within data.
- First-time user experience walkthrough for newly onboarded users.

Check warning on line 1407 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1407

Did you really mean 'walkthrough'?

Check warning on line 1407 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1407

Did you really mean 'onboarded'?
- Quick filters for annotations view in Prototype and Observe.
- Compare runs in Prototype.
- Diff view for compare dataset.
- Enhancement of Observe and Prototype.
- Addition of new evals for audio - conversational and completeness evals.

Check warning on line 1412 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1412

Did you really mean 'evals'?

Check warning on line 1412 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1412

Did you really mean 'evals'?

**Bugs/Improvements**
- New choice for Tone Eval if none of the choices are suitable.

Check warning on line 1415 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1415

Did you really mean 'Eval'?
- Bug on experiment view.
- UI/UX bugs - knowledge base and audio support for evals.

Check warning on line 1417 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1417

Did you really mean 'evals'?
- Required input field column detail not coming on Audio Quality evals.

Check warning on line 1418 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1418

Did you really mean 'evals'?
- UX changes for loader of plan screen.
- Changed the color and the percentage of the eval chips in experiment.

Check warning on line 1420 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1420

Did you really mean 'eval'?
</div>

## Week of 2025-04-21
Expand All @@ -1396,11 +1434,11 @@
**Bugs/Improvements**
- Rate limit hit better UI.
- Audio and knowledge base bug fixes.
- Improved wrong evals view.

Check warning on line 1437 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1437

Did you really mean 'evals'?
- Fixes in compare dataset.
- Changed the logo URL.
- Filter issue fixed in Prototype.
- Rate limit error message to upgrade the plan.
- Experiment optimization under datasets to work faster.
- Huggingface error handling for different datasets.

Check warning on line 1443 in src/pages/docs/release-notes.mdx

View check run for this annotation

Mintlify / Mintlify Validation (futureagi) - vale-spellcheck

src/pages/docs/release-notes.mdx#L1443

Did you really mean 'Huggingface'?
</div>
Loading