v4.12.1

Release Notes

In this release, we've focused on improving the local user experience.
We've revamped the output of the cli to provide tailed logs while the benchmarks are being executed, and polished a bit the display of results after your run.

We also added support to run multiple modes in a single command invocation, both for local and CI runs!
For instance, that means you can now do
codspeed run --mode simulation --mode memory --mode walltime -- cargo codspeed run

Last but not least, you can now define entrypoints in your project config

# codspeed.yml
# List of commands that are run when running `codspeed run` without further arguments
benchmarks:
  - name: My ls command
    # Mutually exclusive with entrypoint
    # Expects a command that will run a benchmark for an arbitrary command
    exec: ls -al /usr/bin
   
  - name: My pytest-codspeed benchmark
    # Mutually exclusive with exec
    # Expects a command that will run a benchmark program that is using a codspeed integration
    entrypoint: pytest --codspeed src

Fix a bug where memory profiling with codspeed exec for binaries that use statically-linked allocators (e.g. jemalloc in Rust). Previously, we reported incorrect memory usage because it didn't attach to the statically linked allocators.

Details

🚀 Features

• Set NODE_OPTIONS to get perf map from node by @GuillaumeLagrange in #267
• Disable node instrospection for exec-harness by @GuillaumeLagrange
• Support CODSPEED_MEMTRACK_BINARIES for static allocator discovery by @not-matthias
• Gracefully handle logs from the runner while rolling buffer is active by @GuillaumeLagrange
• Add --show-full-output to bypass the new rolling buffer by @GuillaumeLagrange
• Add a spinner while waiting for results by @GuillaumeLagrange
• Clean up group headers and emoji usage, make the executor title not dim after run by @GuillaumeLagrange
• Add rolling buffer to display logs from executors by @GuillaumeLagrange
• Polish impact reporting and upload feedback by @art049
• Add color-coded metrics to benchmark result tables by @art049
• Enhance local logger with richer visual hierarchy by @art049
• Style the ASCII banner with CodSpeed orange by @GuillaumeLagrange
• Enforce CodSpeed CLI as single source of truth for all measurements by @art049
• Add optimize and setup-harness skills by @art049
• Add the cursor plugin by @art049
• Check if user is logged in before doing a local run by @GuillaumeLagrange in #263
• Skip respository resolve with --skip-upload by @GuillaumeLagrange
• Use an enum and fix schema for entry/entrypoint by @GuillaumeLagrange in #260
• Accept a mix of entrypoint and exec targets in project config by @GuillaumeLagrange
• Introduce OrchestratorConfig and ExecutorConfig by @GuillaumeLagrange
• Unify run_part_id suffix computation between ci providers and local by @GuillaumeLagrange
• Add the claude plugin by @art049
• Add local_data to upload metadata by @GuillaumeLagrange
• Use repositoryOverview resolver rather than repository by @GuillaumeLagrange

🐛 Bug Fixes

• Use fp unwinding mode when running go test by @GuillaumeLagrange in #269
• Improve error message when no benchmarks are found by @GuillaumeLagrange in #264

🏗️ Refactor

• Extract shared test helpers for compile and track by @not-matthias
• Pass extra env via config instead of unsafe set_var by @not-matthias
• Remove the closure argument to poll in favor of polling options by @GuillaumeLagrange
• Move EXEC_HARNESS_VERSION and EXEC_HARNESS_COMMAND to orchestrator mod by @GuillaumeLagrange
• Flatten orchestrator execution loop over (command, mode) pairs by @GuillaumeLagrange
• Introduce Orchestrator to support multi-mode execution by @GuillaumeLagrange in #259
• Consolidate poll_results into upload module by @GuillaumeLagrange

📚 Documentation

• docs: add documentation about multiple modes in a single run by @GuillaumeLagrange in #195

🧪 Testing

• Add spawn wrapper integration test for static allocator discovery by @not-matthias in #266

⚙️ Internals

• Bump linux-perf-data to use upstream version by @GuillaumeLagrange in #271
• Remove lazy_static in favor of LazyLock by @GuillaumeLagrange in #265
• Rename skill directories with codspeed prefix by @art049
• Move architecture docs out of this repo by @GuillaumeLagrange in #262
• Typo in plugin description by @art049
• Bump memtrack version
• chore: bump runner version to 4.12.0 by @github-actions[bot] in #194
• chore: bump runner version to 4.12.1 by @github-actions[bot] in #196

Install codspeed-runner 4.12.1

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.12.1/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.12.1

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

Full Changelog: v4.11.1...v4.12.1

v4.11.1

Release Notes

🐛 Bug Fixes

• fix: stop using curl --fail-with-body and remove jq dependency by @GuillaumeLagrange in #193

⚙️ Internals

• chore: bump runner version to 4.11.1 by @github-actions[bot] in #191
• Pin all actions (#250) by @art049 in #250
• Bump nix to 0.31.1 (#247) by @3v1n0 in #247

Install codspeed-runner 4.11.1

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.11.1/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.11.1

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

What's Changed

Full Changelog: v4.11.0...v4.11.1

v4.11.0

Release Notes

🚀 Features

• feat: add SHA256 hash verification for the installer script by @art049 in #189
• Collect debug info by pid by @GuillaumeLagrange in #240
• Skip tests requiring sudo if GITHUB_ACTIONS is not set by @GuillaumeLagrange
• Deduplicate symbol maps, unwind_data and debug info by @GuillaumeLagrange
• Use bench_pids filters when harvesting symbols from perf file by @GuillaumeLagrange
• Add new fields to perf metadata by @GuillaumeLagrange in #246
• Always try to attach to libc-compatible API by @not-matthias
• Support tcmalloc by @not-matthias
• Support more standard allocation APIs by @not-matthias
• Add tracegrind support as an alternative simulation tool by @art049 in #236

🐛 Bug Fixes

• Re-add perf map colection removed by mistake by @GuillaumeLagrange in #251

🏗️ Refactor

• Use common lib paths for all allocators by @not-matthias

⚙️ Internals

• chore: bump runner version to 4.11.0 by @github-actions[bot] in #190
• Bump exec-harness version
• Remove outdated allocator check by @not-matthias in #241
• Add tcmalloc test by @not-matthias

Install codspeed-runner 4.11.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.11.0/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.11.0

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

Full Changelog: v4.10.6...v4.11.0

v4.10.6

Release Notes

🚀 Features

• Use codspeed.io installer for cached and authenticated requests by @adriencaccia in #187
• Force necessary flags for go run in introspected go by @GuillaumeLagrange in #238
• Bump the stack size for python by @GuillaumeLagrange
• Make python generate perf maps in exec-harness analysis by @GuillaumeLagrange
• Harvest python perf maps after parsing perf data by @GuillaumeLagrange
• Allow memory instrument with exec-harness's integration hook by @GuillaumeLagrange
• Use the new instrument-hooks bindings rather than codspeed core by @GuillaumeLagrange

🐛 Bug Fixes

• Panic if integration doesn't support memory profiling by @not-matthias in #239
• Allow memtrack caching by resetting .cargo permissions by @not-matthias in #237
• Search build directories in sub-folders by @not-matthias
• Handle working-directory option by @not-matthias
• Improve simulation script support and detect subprocesses by @GuillaumeLagrange

🏗️ Refactor

• Move common perf-map utility outside of valgrind helpers by @GuillaumeLagrange

⚙️ Internals

• chore: bump runner version to 4.10.6 by @github-actions[bot] in #188
• Bump memtrack version
• Add debug logs for harvested bench pids by @GuillaumeLagrange
• Update wording in build.rs by @GuillaumeLagrange in #235
• Use tokio::fs when harvesting perf maps from pids by @GuillaumeLagrange
• Import instrument-hooks bindings from codspeed-rust by @GuillaumeLagrange
• Move small snapshot files out of Git LFS by @art049

Install codspeed-runner 4.10.6

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.10.6/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.10.6

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

Full Changelog: v4.10.5...v4.10.6

v4.10.5

Release Notes

🚀 Features

• Support codspeed-node memory profiling by @not-matthias in #228

🐛 Bug Fixes

• Check benchmark URIs rather than memtrack events by @not-matthias in #230
• Do not error out if no head report is present after local run by @GuillaumeLagrange in #233

⚙️ Internals

• chore: bump runner version to 4.10.5 by @github-actions[bot] in #186

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.10.5/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.10.5

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

Full Changelog: v4.10.4...v4.10.5

v4.10.4

Release Notes

🚀 Features

• Check for repository existence before run in local runs by @GuillaumeLagrange in #229
• Support pytest-codspeed memory profiling by @not-matthias in #225
• Attach to different jemalloc prefixed functions by @not-matthias
• Support tracking libstdc++ by @not-matthias
• Track posix_memalign by @not-matthias

🐛 Bug Fixes

• Check non-default allocators first by @not-matthias
• Reset owernship by @not-matthias

🏗️ Refactor

• Use standard allocation API across allocators by @not-matthias in #212

⚙️ Internals

• chore: bump runner version to 4.10.3 by @github-actions[bot] in #184
• chore: bump runner version to 4.10.4 by @github-actions[bot] in #185
• Use snapshots for c tests by @not-matthias
• Only run tests in CI by @not-matthias
• Use single thread for tests to avoid opening too many files by @not-matthias
• Add cpp allocator test by @not-matthias
• Add rust allocator tests by @not-matthias
• Allow tracking statically linked allocators in tests by @not-matthias

Install codspeed-runner 4.10.4

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.10.4/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.10.4

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

Full Changelog: v4.10.2...v4.10.4

v4.10.2

Release Notes

🚀 Features

• Improve efficiency of parse_perf_file by @GuillaumeLagrange
• Parse the perf file memory mappings by @GuillaumeLagrange

🐛 Bug Fixes

• Remove very spammy trace logs by @GuillaumeLagrange in #224

⚙️ Internals

• chore: bump runner version to 4.10.2 by @github-actions[bot] in #183
• Remove useless conversions. by @GuillaumeLagrange

Install codspeed-runner 4.10.2

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.10.2/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.10.2

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

Full Changelog: v4.10.1...v4.10.2

v4.10.1

Release Notes

🚀 Features

• Implement codspeed use to set mode for shell session by @GuillaumeLagrange in #221
• Show codspeed banner in CI when exec_harness is used by @GuillaumeLagrange
• Improve the result table display (#217) by @art049 in #217

🐛 Bug Fixes

• Initialize tracker before spawning sub-process by @not-matthias

🏗️ Refactor

• Restructure event handling with a common header and improve event parsing by @not-matthias
• Create a module for shared structs between exec and run by @GuillaumeLagrange
• Remove logic from the run module by @GuillaumeLagrange
• Move the cli related module inside a cli module by @GuillaumeLagrange

⚙️ Internals

• chore: bump runner version to 4.10.1 by @github-actions[bot] in #182
• Bump memtrack version
• Turn warning about overwritten unwind_data into debug logs by @GuillaumeLagrange in #223
• Fix cargo check on macOs by @adriencaccia
• Remove codspeed use without argument in favor of codspeed show by @GuillaumeLagrange in #222
• Add clang-format pre-commit hook by @not-matthias in #219
• Save both old and new addr for realloc event by @not-matthias

Install codspeed-runner 4.10.1

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/codspeed/releases/download/v4.10.1/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.10.1

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

Full Changelog: v4.10.0...v4.10.1

v4.10.0

Release Notes

🚀 Features

• Add pre-commit check to sync schema with code by @GuillaumeLagrange
• Add binary to generate project config schema by @GuillaumeLagrange
• Add command aliases by @art049 in #216

🐛 Bug Fixes

• Rename targets to benchmarks in the project_config to match spec by @GuillaumeLagrange in #220
• Use child process instead of pid/perf fifo for FIFO health check by @not-matthias in #201
• Make FIFO cancel-safe by @not-matthias
• Dont run health check in hot path by @not-matthias
• Update warning message format for go runner installation by @not-matthias in #218
• Tweaked exec-harness build for gcc15+ by @art049

🏗️ Refactor

• Move the whole runner logic under lib.rs to add other binaries by @GuillaumeLagrange

⚙️ Internals

• Add cancel-safety test for FIFO by @not-matthias
• Remove cargo check from pre-commit by @GuillaumeLagrange in #215
• Remove runner_mode from project config by @GuillaumeLagrange
• Improve readme by @art049 in #214

Full Runner Changelog: https://github.com/CodSpeedHQ/codspeed/blob/main/CHANGELOG.md

v4.8.2

Release Notes

🐛 Bug Fixes

• Check whether artifact contains any elements by @not-matthias in #204

⚙️ Internals

• chore: bump runner version to 4.8.2 by @github-actions[bot] in #174

Install codspeed-runner 4.8.2

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/CodSpeedHQ/runner/releases/download/v4.8.2/codspeed-runner-installer.sh | sh

Download codspeed-runner 4.8.2

File	Platform	Checksum
codspeed-runner-aarch64-unknown-linux-musl.tar.gz	ARM64 MUSL Linux	checksum
codspeed-runner-x86_64-unknown-linux-musl.tar.gz	x64 MUSL Linux	checksum

Full Runner Changelog: https://github.com/CodSpeedHQ/runner/blob/main/CHANGELOG.md

Full Changelog: v4.8.1...v4.8.2