Skip to content

Record reproducible LDBC measurements and optimization gaps - #875

Draft
frankmcsherry wants to merge 3 commits into
master-nextfrom
codex/ddir-ldbc-gaps
Draft

frankmcsherry wants to merge 3 commits into
master-nextfrom
codex/ddir-ldbc-gaps

Conversation

@frankmcsherry

Copy link
Copy Markdown
Member

The LDBC suite needs a reproducible performance baseline and a record of optimization opportunities that another session can refresh. Add CURRENT.md with measurements of all 41 queries on both backends, GAPS.md with bounded investigation tasks, and exact data/build/run instructions in DATA.md, REFRESH.md, and the build receipt.

The baseline uses unmodified master-next 229508dd on an M4 mini (16 GiB, four workers): three isolated SF0.003 catalogue trials, three concurrent IS1/IS3/IC11/BI11/BI18 trials, and one all-query tiny check. Seven complete compressed reports (~1.8 MB), their checksums, resolved Cargo.lock, and resource receipts are included. Query definitions, the timed runner, engine code and dependency pins are unchanged.

readout.py reports complete recorded cycles, setup and final retirement separately, and summarizes compatible repeated trials. compare.py also accepts compressed reports. The recipes document the fixed hash seed required by the current default parameter selection. Anyone can replace the readout and revise a gap assessment using new validated evidence; no growing historical archive or session-specific query driver is required.

These are small-data lifecycle measurements, not official LDBC scores, SF1 capacity measurements, or 30-binding maintenance throughput. Each BI query has one standing binding, and the small update stream leaves many results unchanged. The gap register distinguishes observed code behavior from hypotheses about elapsed time.

Validation: all seven runs passed (8,818 same-plan answer checks including setup/warmup); repeated-trial comparisons passed for both timing recipes; the eight Python harness/catalogue tests passed; report checksums and documentation links verified. All runs stayed below the 2-GiB combined process-group cap with zero swap use. The concurrent panel's median full cycle was 10.093 ms on Vec and 11.777 ms on Corgi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant