Skip to content

refactor: standardize complete performance reports - #277

Merged
acgetchell merged 1 commit into
mainfrom
refactor/276-standard-reports
Oct 10, 2026
Merged

acgetchell merged 1 commit into
mainfrom
refactor/276-standard-reports

Conversation

@acgetchell

@acgetchell acgetchell commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Complete performance runs now use the shared research-repo-tools renderer for scratch and published reports. Configuration in tooling/performance-report.toml selects the title and docs/performance.md destination, replacing the local full-report renderer and duplicate publication assembly.

  • Keep scientific eligibility checks, the dimension plot, and historical regeneration support local.
  • Preserve the existing v0.4.6 report byte-for-byte in the archive and retain immutable complete-run history. No automatic pruning or new benchmark measurements.
  • Correct documentation for the shared mean/median report layout and retained JSON provenance, and force LF in the generated Cargo shell fixture so native Windows text output cannot break its shell syntax.

This uses the published research-repo-tools 0.1.8 APIs. It prepares for #276; v0.1.9 adoption and the remaining shared capabilities stay open there.

Validation: just check python-ci passed on macOS, including all 371 Python tests, type checking, lint/config checks, documentation checks, and spelling. An isolated check of the generated Cargo fixture reproduced a shell syntax failure with Windows-style CRLF output and succeeded with the repaired LF output. This is a macOS model; native Linux and Windows results remain pending on this PR.

Refs #276

Summary by CodeRabbit

  • Documentation

    • Updated the benchmarking and release guides to explain the shared performance-report workflow, complete-run results, and retained evidence.
    • Added an archived comparison for v0.4.6 versus v0.4.5 and clarified how historical reports differ from current reports.
    • Updated report links, locations, and regeneration guidance across the documentation.
  • Bug Fixes

    • Markdown checks and formatting now exclude the current performance report, preserving its published format.

- Use shared rendering and configured publication for complete-run reports.
- Preserve dimension plots, historical report bytes, and immutable evidence.
- Align report guidance with retained provenance and make the Cargo shell
  fixture use LF on Windows.

Refs #276
@acgetchell acgetchell added this to the v0.4.7 milestone Oct 10, 2026
@acgetchell acgetchell added refactor Code restructuring without behavior change reproducibility Reproducibility and artifact packaging python Pull requests that update Python code labels Oct 10, 2026
@coderabbitai

coderabbitai Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: acgetchell/la-stack/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 74325925-fb4b-4c9b-83db-6e8a2eebd849

📥 Commits

Reviewing files that changed from the base of the PR and between f3eb43d and 1f33f14.


📒 Files selected for processing (12)
  • docs/BENCHMARKING.md
  • docs/RELEASING.md
  • docs/archive/performance/README.md
  • docs/archive/performance/v0.4.6-vs-v0.4.5.md
  • docs/code_organization.md
  • docs/performance/README.md
  • justfile
  • scripts/README.md
  • scripts/archive_performance.py
  • scripts/performance_runs.py
  • scripts/tests/test_performance_workflow.py
  • tooling/performance-report.toml

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.



📝 Walkthrough

Walkthrough

The performance workflow now publishes shared complete-run reports at a configured current path, while legacy comparison reports remain archived in historical format. Promotion validates and publishes shared report plans. Documentation and tests describe the report configuration, provenance, repeated runs, and Markdown exclusions.

Changes

Performance reporting

Layer / File(s) Summary
Shared report contract and configuration
tooling/performance-report.toml, docs/BENCHMARKING.md, docs/code_organization.md, scripts/README.md
The configured current report path is docs/performance.md. Documentation describes tables organized by statistic and named series, per-series attribution, and provenance retained in the evidence JSON. It assigns complete-run report rendering to the shared package.
Complete-run promotion and validation
scripts/archive_performance.py, scripts/performance_runs.py, scripts/tests/test_performance_workflow.py, justfile
Scratch reports now use render_run. Promotion validates release evidence, checks the selected run identity, and publishes the shared report plan. Tests cover configurable report paths, repeated runs, legacy-artifact collision behavior, and unchanged published outputs. Markdown check and format recipes exclude docs/performance.md.
Legacy report preservation
docs/archive/performance/*, docs/performance/README.md, docs/RELEASING.md, docs/BENCHMARKING.md
The archive index now includes the v0.4.6-versus-v0.4.5 report. Documentation distinguishes historical snapshots from shared complete-run history and directs release reviews to the shared report and immutable evidence.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant ReleaseWorkflow
  participant archive_performance
  participant ReportConfiguration
  participant SharedRunReportPlan
  ReleaseWorkflow->>archive_performance: promote complete-run evidence
  archive_performance->>archive_performance: validate evidence and match selected run identity
  archive_performance->>ReportConfiguration: read configured report path and title
  archive_performance->>SharedRunReportPlan: publish shared report and history plan
Loading

Merge Risk: ⚪ Minimal · up to 1f33f

This change moves complete performance report publishing to the shared tooling and keeps the legacy reports archived. No concrete merge-blocking risk is evident. Native Linux and Windows test results are still pending.

Pre-merge checks | Passed 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly summarizes the main change: standardizing complete performance report rendering and publication through shared APIs and configuration.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR


🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Oct 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.11%. Comparing base (f3eb43d) to head (1f33f14).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #277      +/-   ##
==========================================
- Coverage   98.12%   98.11%   -0.02%     
==========================================
  Files          14       14              
  Lines        7112     7112              
==========================================
- Hits         6979     6978       -1     
- Misses        133      134       +1     
Flag Coverage Δ
unittests 98.11% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@acgetchell
acgetchell merged commit d12bdb4 into main Oct 10, 2026
19 checks passed
@acgetchell
acgetchell deleted the refactor/276-standard-reports branch October 10, 2026 04:13
@acgetchell acgetchell modified the milestones: v0.4.7, v0.5.0 Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests that update Python code refactor Code restructuring without behavior change reproducibility Reproducibility and artifact packaging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant