feat: shared JSONL/CSV dataset compatibility with promptfoo

[christso]

Summary

Ensure AgentV's dataset loading is compatible with promptfoo's file:// test data format, so the same JSONL and CSV files work in both tools without modification.

Motivation

The lowest-effort integration point between AgentV and promptfoo is shared test data files. If both tools can read the same JSONL/CSV datasets, users get data portability without any config conversion tooling.

Research reference: integration-assessment-promptfoo-braintrust.md

promptfoo Dataset Format

promptfoo loads external test data via file:// references:

# promptfooconfig.yaml
tests: file://tests/cases.jsonl

JSONL format:

{"query": "What is 2+2?", "expected": "4", "category": "math"}
{"query": "Capital of France?", "expected": "Paris", "category": "geography"}

CSV format:

query,expected,category
"What is 2+2?","4","math"
"Capital of France?","Paris","geography"

Columns map to vars in promptfoo's test cases.

AgentV Dataset Format

# EVAL.yaml
datasets:
  - source: ./tests/cases.jsonl
    format: jsonl

AgentV expects:

{"input": "What is 2+2?", "expected_output": "4", "metadata": {"category": "math"}}

Compatibility Gap

The field names differ:

• promptfoo uses arbitrary column names mapped via vars
• AgentV uses input / expected_output / metadata

Proposed Solution

Add a column_mapping option to AgentV's dataset loader:

datasets:
  - source: ./tests/cases.jsonl
    format: jsonl
    column_mapping:
      input: query           # Map "query" column to AgentV's "input"
      expected_output: expected  # Map "expected" column to "expected_output"
      metadata.category: category  # Map "category" to metadata

This allows the same JSONL/CSV file to be used by both tools without modification.

Acceptance Criteria

• Dataset loader accepts column_mapping for field remapping
• Supports JSONL and CSV formats
• Falls back to default field names (input, expected_output) when no mapping specified
• Dot notation for metadata fields (metadata.category)
• Works with existing file:// or relative path references
• Documentation with promptfoo compatibility example

Effort Estimate

1-2 days