diff --git a/.cargo/config.toml b/.cargo/config.toml index 934b5126..0374e829 100644 --- a/.cargo/config.toml +++ b/.cargo/config.toml @@ -1,5 +1,25 @@ +# leanVM proves over a binary field tower, whose multiplication is a carryless +# multiply. Both entries below have to enable it, or the prover falls back to +# scalar bit-twiddling: `+pclmulqdq` on x86, and on aarch64 the `aes` feature, +# which is what LLVM gates PMULL behind. +# +# The aarch64 entry is not symmetry for its own sake. `aarch64-unknown-linux-gnu` +# defaults to `neon` alone, while `aarch64-apple-darwin` defaults to a CPU that +# already has `aes`, so a Linux container on an Apple-silicon host was the one +# build with no carryless multiply. Measured on an M4 Max, one aggregation proof +# went from ~30 s without this to well under a second with it, which is the +# difference between a local devnet that cannot fill a slot and one that can. + [target.x86_64-unknown-linux-gnu] rustflags = [ "-Ctarget-cpu=x86-64-v3", "-Ctarget-feature=+avx2,+sse2,+ssse3,+sse4.1,+sse4.2,+bmi1,+lzcnt,+pclmulqdq", ] + +[target.aarch64-unknown-linux-gnu] +rustflags = [ + # Baseline armv8.2 (the floor for Apple silicon and Neoverse N1/V1), plus the + # crypto extensions carrying PMULL and SHA-2. + "-Ctarget-cpu=neoverse-n1", + "-Ctarget-feature=+aes,+sha2,+neon", +] diff --git a/CLAUDE.md b/CLAUDE.md index 7ce0bb6c..9b38ac6e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,7 +30,7 @@ crates/ └─ src/metrics.rs # State transition timing + counters common/ ├─ types/ # Core types (State, Block, Attestation, Checkpoint) - ├─ crypto/ # XMSS aggregation (leansig wrapper) + ├─ crypto/ # XMSS sign/verify + aggregation (leanVM wrapper) ├─ metrics/ # Prometheus re-exports, TimingGuard, gather utilities └─ test-fixtures/ # Spec-fixture loading (prod dep of rpc's Hive test driver) net/ @@ -262,9 +262,37 @@ actual_slot = finalized_slot + 1 + relative_index **XMSS (eXtended Merkle Signature Scheme):** - Post-quantum signature scheme -- 52-byte public keys, 2536-byte signatures (`SIGNATURE_SIZE` in `common/types/src/signature.rs`) +- Wire sizes: `PUBLIC_KEY_SIZE` (`common/types/src/state.rs`) and `SIGNATURE_SIZE` + (`common/types/src/attestation.rs`), static-asserted against leanVM's scheme + constants in `common/crypto/src/signature.rs` - Epoch-based to prevent reuse -- Aggregation via leanVM (previously leanMultisig) for efficiency +- Signing, verification, and aggregation all come from leanVM, which internalized + XMSS in its own `xmss` crate; there is no external leanSig dependency +- BLAKE2s over binary fields, not Poseidon over KoalaBear: a leanVM `main` bump + across that rewrite invalidates every genesis key and every stored proof, even + when the wire sizes happen to match +- `ethlambda_crypto::init_leanvm(use_arena)` must run once at startup, before any + proving or proof decoding. `--prover-arena` opts into leanVM's bump arena, + which recycles the prover's large buffers across proofs instead of re-faulting + them, so its pages stay resident for the node's lifetime +- `ethlambda keygen` generates genesis validator keys through the same + `ValidatorSecretKey` the node loads them with, so a key set cannot be built + against a different leanVM than the client reading it. Keys are only usable by + a client on the matching revision, and no file size changes when the scheme + does, so the manifest records `leanvm_rev`. See [`docs/keygen.md`](docs/keygen.md) + +**Aggregation shape (one leanVM `AggregateSignature`, grouped by epoch):** +- Type-1 and Type-2 are the same object: one `XmssGroup` per slot, carrying the + one message signed at it and that group's sorted, deduplicated keys +- **A slot carries one message.** Two distinct `AttestationData` at one slot + cannot share an aggregate; `ethlambda_crypto::ConflictingMessages` says so, and + nothing below it can work around the constraint +- **The binding is off the wire.** `to_bytes_without_pubkeys()` carries neither + the keys nor the `(slot, message)` pairs, so every decode rebuilds the whole + signer set from a `SignerSet` per claim. A wrong set, message or slot decodes + fine and fails inside the SNARK verifier, so there is no cheap binding check +- Narrowing replaces splitting: re-aggregate the parent with a `declare` naming + the group to keep (`split_type_2_by_message`) **Signature Aggregation (Two-Phase):** 1. **Gossip signatures**: Fresh XMSS from network → aggregate via leanVM @@ -309,7 +337,7 @@ one port is supported and not a misconfiguration. See [`docs/rpc.md`](docs/rpc.m GENESIS_TIME: 1770407233 MILLISECONDS_PER_SLOT: 4000 # optional, defaults to DEFAULT_MILLISECONDS_PER_SLOT GENESIS_VALIDATORS: - - attestation_pubkey: "cd323f232b34ab26d6db7402c886e74ca81cfd3a..." # 52-byte XMSS pubkeys (hex) + - attestation_pubkey: "cd323f232b34ab26d6db7402c886e74ca81cfd3a..." # XMSS pubkeys, hex, PUBLIC_KEY_SIZE bytes proposal_pubkey: "b7b0f72e24801b02bda64073cb4de6699a416b37..." ``` - Validator indices are assigned sequentially (0, 1, 2, ...) based on array order @@ -400,7 +428,9 @@ behavior. ## External Dependencies **Critical:** -- `leansig`: XMSS signatures (leanEthereum project) +- `leanvm`: XMSS signatures and recursive aggregation, taken from leanVM's facade + crate (which re-exports `xmss`, `rec_aggregation` and its `rand`) and pinned to + one `main` revision (leanEthereum project) - `libssz` / `libssz-derive` / `libssz-types`: SSZ serialization - `libssz-merkle`: Merkle tree hashing (`hash_tree_root()`) - `spawned-concurrency`: Actor model diff --git a/Cargo.lock b/Cargo.lock index dacda8d5..61184c1e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -172,7 +172,7 @@ version = "1.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -183,7 +183,7 @@ checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" dependencies = [ "anstyle", "once_cell_polyfill", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -647,25 +647,6 @@ dependencies = [ "tracing", ] -[[package]] -name = "backend" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "mt-air", - "mt-fiat-shamir", - "mt-field", - "mt-koala-bear", - "mt-poly", - "mt-sumcheck", - "mt-symetric", - "mt-utils", - "mt-whir", - "parallel", - "tracing", - "zk-alloc", -] - [[package]] name = "backtrace" version = "0.3.76" @@ -721,16 +702,25 @@ version = "0.6.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "230c5f1ca6a325a32553f8640d31ac9b49f2411e901e427570154868b46da4f7" +[[package]] +name = "bincode" +version = "1.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad" +dependencies = [ + "serde", +] + [[package]] name = "bindgen" version = "0.72.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "993776b509cfb49c750f11b8f07a46fa23e0a1386ffc01fb1e7d343efc387895" dependencies = [ - "bitflags", + "bitflags 2.11.1", "cexpr", "clang-sys", - "itertools 0.10.5", + "itertools 0.12.1", "proc-macro2", "quote", "regex", @@ -770,6 +760,12 @@ dependencies = [ "hex-conservative", ] +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + [[package]] name = "bitflags" version = "2.11.1" @@ -1134,12 +1130,6 @@ version = "0.9.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" -[[package]] -name = "const-oid" -version = "0.10.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" - [[package]] name = "const-str" version = "0.4.3" @@ -1492,7 +1482,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ccc2776f0c61eca1ca32528f85548abd1a4be8fb53d1b21c013e4f18da1e7090" dependencies = [ "data-encoding", - "syn 1.0.109", + "syn 2.0.117", ] [[package]] @@ -1507,13 +1497,44 @@ dependencies = [ "walkdir", ] +[[package]] +name = "defmt" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e2953bfe4f93bbd20cc71198842756f77d161884c99ebbabc41d80231ded88d1" +dependencies = [ + "bitflags 1.3.2", + "defmt-macros", +] + +[[package]] +name = "defmt-macros" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bad9c72e7ca2137e0dc3813245a0d282fd6daad32fd800af018306a9169b5fe8" +dependencies = [ + "defmt-parser", + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "defmt-parser" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10d60334b3b2e7c9d91ef8150abfb6fa4c1c39ebbcf4a81c2e346aad939fee3e" +dependencies = [ + "thiserror 2.0.18", +] + [[package]] name = "der" version = "0.7.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" dependencies = [ - "const-oid 0.9.6", + "const-oid", "pem-rfc7468", "zeroize", ] @@ -1645,7 +1666,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ "block-buffer 0.10.4", - "const-oid 0.9.6", + "const-oid", "crypto-common 0.1.7", "subtle", ] @@ -1657,7 +1678,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" dependencies = [ "block-buffer 0.12.0", - "const-oid 0.10.2", "crypto-common 0.2.2", ] @@ -1667,7 +1687,7 @@ version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38" dependencies = [ - "bitflags", + "bitflags 2.11.1", "block2", "libc", "objc2", @@ -1848,7 +1868,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.52.0", ] [[package]] @@ -1934,6 +1954,7 @@ dependencies = [ "serde", "serde_json", "serde_yaml_ng", + "tempfile", "thiserror 2.0.18", "tikv-jemallocator", "tokio", @@ -1957,10 +1978,8 @@ dependencies = [ "ethlambda-test-fixtures", "ethlambda-types", "hex", - "leansig", "libssz", "libssz-types", - "rand 0.10.1", "rayon", "serde", "spawned-concurrency", @@ -1976,11 +1995,10 @@ version = "0.1.0" dependencies = [ "ethlambda-types", "hex", - "lean-multisig", - "leansig", - "leansig_wrapper", - "rand 0.10.1", + "leanvm", + "postcard", "thiserror 2.0.18", + "tracing", ] [[package]] @@ -2082,12 +2100,10 @@ version = "0.1.0" dependencies = [ "ethlambda-crypto", "ethlambda-types", - "leansig", "libssz", "libssz-derive", "libssz-types", "lru", - "rand 0.10.1", "rocksdb", "tempfile", "thiserror 2.0.18", @@ -2367,7 +2383,7 @@ dependencies = [ "rkyv", "rustc-hash", "serde", - "spin 0.9.8", + "spin", "thiserror 2.0.18", ] @@ -2487,6 +2503,16 @@ version = "0.2.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" +[[package]] +name = "fiat_shamir" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" +dependencies = [ + "parallel", + "primitives", + "serde", +] + [[package]] name = "find-msvc-tools" version = "0.1.9" @@ -2515,6 +2541,18 @@ dependencies = [ "miniz_oxide", ] +[[package]] +name = "flock" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" +dependencies = [ + "fiat_shamir", + "parallel", + "pcs", + "primitives", + "zk_alloc", +] + [[package]] name = "fnv" version = "1.0.7" @@ -2762,7 +2800,7 @@ version = "0.20.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7b88256088d75a56f8ecfa070513a775dd9107f6530ef14919dac831af9cfe2b" dependencies = [ - "bitflags", + "bitflags 2.11.1", "libc", "libgit2-sys", "log", @@ -2889,7 +2927,7 @@ dependencies = [ "hash32", "rustc_version 0.4.1", "serde", - "spin 0.9.8", + "spin", "stable_deref_trait", ] @@ -3116,7 +3154,7 @@ dependencies = [ "libc", "percent-encoding", "pin-project-lite", - "socket2 0.5.10", + "socket2 0.6.3", "tokio", "tower-service", "tracing", @@ -3134,7 +3172,7 @@ dependencies = [ "js-sys", "log", "wasm-bindgen", - "windows-core 0.62.2", + "windows-core 0.57.0", ] [[package]] @@ -3362,25 +3400,6 @@ dependencies = [ "syn 2.0.117", ] -[[package]] -name = "include_dir" -version = "0.7.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "923d117408f1e49d914f1a379a309cffe4f18c05cf4e3d12e613a15fc81bd0dd" -dependencies = [ - "include_dir_macros", -] - -[[package]] -name = "include_dir_macros" -version = "0.7.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7cab85a7ed0bd5f0e76d93846e0147172bed2e2d3f859bcc33a8d9699cad1a75" -dependencies = [ - "proc-macro2", - "quote", -] - [[package]] name = "indenter" version = "0.3.4" @@ -3475,6 +3494,15 @@ dependencies = [ "either", ] +[[package]] +name = "itertools" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba291022dbbd398a455acf126c1e341954079855bc60dfdda641363bd6922569" +dependencies = [ + "either", +] + [[package]] name = "itertools" version = "0.13.0" @@ -3516,6 +3544,59 @@ dependencies = [ "tracing", ] +[[package]] +name = "jiff" +version = "0.2.35" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "668b7183bd07af9a4885f5c35b0cc5c83c4607a913c16b7e17291832910d2dcc" +dependencies = [ + "defmt", + "jiff-core", + "jiff-static", + "jiff-tzdb-platform", + "log", + "portable-atomic", + "portable-atomic-util", + "serde_core", + "windows-link", +] + +[[package]] +name = "jiff-core" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7feca88439efe53da3754500c1851dedf3cb36c524dd5cf8225cc0794de95d09" +dependencies = [ + "defmt", +] + +[[package]] +name = "jiff-static" +version = "0.2.35" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a69dcb3a21cfb32ce1cd056169337ca284af0766dd766e7878819b251a49204" +dependencies = [ + "jiff-core", + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "jiff-tzdb" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "142bd39932ad231f10513df9ab62661fead8719872150b7ad02a2df79f4e141e" + +[[package]] +name = "jiff-tzdb-platform" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "875a5a69ac2bab1a891711cf5eccbec1ce0341ea805560dcd90b7a2e925132e8" +dependencies = [ + "jiff-tzdb", +] + [[package]] name = "jobserver" version = "0.1.34" @@ -3630,126 +3711,42 @@ version = "1.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" -[[package]] -name = "lean-multisig" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "backend", - "clap", - "lean_vm", - "leansig_wrapper", - "libc", - "rand 0.10.1", - "rec_aggregation", - "serde_json", - "sub_protocols", - "system-info", - "utils", - "zk-alloc", -] - [[package]] name = "lean_compiler" version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" dependencies = [ - "backend", - "include_dir", "lean_vm", - "pest", - "pest_derive", - "rand 0.10.1", - "sub_protocols", - "tracing", - "utils", -] - -[[package]] -name = "lean_prover" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "backend", - "itertools 0.14.0", - "lean_compiler", - "lean_vm", - "pest", - "pest_derive", - "rand 0.10.1", - "serde", - "sub_protocols", - "tracing", - "utils", + "primitives", ] [[package]] name = "lean_vm" version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" dependencies = [ - "backend", - "itertools 0.14.0", - "leansig_wrapper", - "pest", - "pest_derive", - "rand 0.10.1", - "serde", + "fiat_shamir", + "flock", + "parallel", + "pcs", + "primitives", "tracing", - "utils", + "zk_alloc", ] [[package]] -name = "leansig" +name = "leanvm" version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanSig?branch=devnet4#15cbdd43ec8525aa43fea2f42cafc5ed366084ae" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" dependencies = [ - "dashmap", - "ethereum_ssz", - "num-bigint", - "num-traits", - "p3-baby-bear", - "p3-field", - "p3-koala-bear", - "p3-symmetric", - "rand 0.10.1", - "rayon", - "serde", - "sha3 0.10.9", - "thiserror 2.0.18", -] - -[[package]] -name = "leansig_fast_keygen" -version = "0.1.0" -source = "git+https://github.com/TomWambsgans/leanSig?branch=devnet4-fast-keygen#0fa9e19b8946ef50a34f3d50d82918b98bcfa4a5" -dependencies = [ - "dashmap", - "ethereum_ssz", - "num-bigint", - "num-traits", - "p3-baby-bear", - "p3-field", - "p3-koala-bear", - "p3-symmetric", - "rand 0.10.1", - "rayon", - "serde", - "sha3 0.10.9", - "thiserror 2.0.18", -] - -[[package]] -name = "leansig_wrapper" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "backend", - "ethereum_ssz", - "leansig", - "leansig_fast_keygen", - "p3-field", - "rand 0.10.1", + "clap", + "lean_vm", + "primitives", + "rand 0.9.4", + "rec_aggregation", + "sphincs", + "xmss", + "zk_alloc", ] [[package]] @@ -4599,15 +4596,6 @@ dependencies = [ "libc", ] -[[package]] -name = "lz4_flex" -version = "0.13.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7ef0d4ed8669f8f8826eb00dc878084aa8f253506c4fd5e8f58f5bce72ddb97e" -dependencies = [ - "twox-hash", -] - [[package]] name = "malachite" version = "0.6.1" @@ -4759,127 +4747,6 @@ dependencies = [ "uuid", ] -[[package]] -name = "mt-air" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "mt-field", - "mt-poly", -] - -[[package]] -name = "mt-fiat-shamir" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "mt-field", - "mt-koala-bear", - "mt-symetric", - "mt-utils", - "parallel", - "serde", - "tracing", -] - -[[package]] -name = "mt-field" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "itertools 0.14.0", - "mt-utils", - "num-bigint", - "parallel", - "paste", - "rand 0.10.1", - "serde", - "tracing", -] - -[[package]] -name = "mt-koala-bear" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "itertools 0.14.0", - "mt-field", - "mt-utils", - "num-bigint", - "paste", - "rand 0.10.1", - "serde", - "tracing", -] - -[[package]] -name = "mt-poly" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "itertools 0.14.0", - "mt-field", - "mt-utils", - "parallel", - "rand 0.10.1", - "serde", - "system-info", - "zk-alloc", -] - -[[package]] -name = "mt-sumcheck" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "mt-air", - "mt-fiat-shamir", - "mt-field", - "mt-poly", - "parallel", - "tracing", - "zk-alloc", -] - -[[package]] -name = "mt-symetric" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "mt-field", - "mt-koala-bear", - "parallel", - "zk-alloc", -] - -[[package]] -name = "mt-utils" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "serde", -] - -[[package]] -name = "mt-whir" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "itertools 0.14.0", - "mt-fiat-shamir", - "mt-field", - "mt-koala-bear", - "mt-poly", - "mt-sumcheck", - "mt-symetric", - "mt-utils", - "parallel", - "rand 0.10.1", - "system-info", - "tracing", - "zk-alloc", -] - [[package]] name = "multiaddr" version = "0.18.2" @@ -4969,7 +4836,7 @@ version = "0.28.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4ce3636fa715e988114552619582b530481fd5ef176a1e5c1bf024077c2c9445" dependencies = [ - "bitflags", + "bitflags 2.11.1", "libc", "log", "netlink-packet-core", @@ -5008,7 +4875,7 @@ version = "0.30.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6" dependencies = [ - "bitflags", + "bitflags 2.11.1", "cfg-if", "cfg_aliases", "libc", @@ -5020,7 +4887,7 @@ version = "0.31.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d" dependencies = [ - "bitflags", + "bitflags 2.11.1", "cfg-if", "cfg_aliases", "libc", @@ -5057,7 +4924,7 @@ version = "0.50.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -5184,16 +5051,6 @@ version = "4.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33" -[[package]] -name = "objc2-foundation" -version = "0.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272" -dependencies = [ - "bitflags", - "objc2", -] - [[package]] name = "object" version = "0.37.3" @@ -5252,180 +5109,12 @@ dependencies = [ "sha2", ] -[[package]] -name = "p3-baby-bear" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "p3-challenger", - "p3-field", - "p3-mds", - "p3-monty-31", - "p3-poseidon1", - "p3-poseidon2", - "p3-symmetric", - "rand 0.10.1", -] - -[[package]] -name = "p3-challenger" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "p3-field", - "p3-maybe-rayon", - "p3-monty-31", - "p3-symmetric", - "p3-util", - "tracing", -] - -[[package]] -name = "p3-dft" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "itertools 0.14.0", - "p3-field", - "p3-matrix", - "p3-maybe-rayon", - "p3-util", - "spin 0.10.0", - "tracing", -] - -[[package]] -name = "p3-field" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "itertools 0.14.0", - "num-bigint", - "p3-maybe-rayon", - "p3-util", - "paste", - "rand 0.10.1", - "serde", - "tracing", -] - -[[package]] -name = "p3-koala-bear" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "p3-challenger", - "p3-field", - "p3-mds", - "p3-monty-31", - "p3-poseidon1", - "p3-poseidon2", - "p3-symmetric", - "rand 0.10.1", -] - -[[package]] -name = "p3-matrix" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "itertools 0.14.0", - "p3-field", - "p3-maybe-rayon", - "p3-util", - "rand 0.10.1", - "serde", - "tracing", -] - -[[package]] -name = "p3-maybe-rayon" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" - -[[package]] -name = "p3-mds" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "p3-dft", - "p3-field", - "p3-symmetric", - "p3-util", - "rand 0.10.1", -] - -[[package]] -name = "p3-monty-31" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "itertools 0.14.0", - "num-bigint", - "p3-dft", - "p3-field", - "p3-matrix", - "p3-maybe-rayon", - "p3-mds", - "p3-poseidon1", - "p3-poseidon2", - "p3-symmetric", - "p3-util", - "paste", - "rand 0.10.1", - "serde", - "spin 0.10.0", - "tracing", -] - -[[package]] -name = "p3-poseidon1" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "p3-field", - "p3-symmetric", - "rand 0.10.1", -] - -[[package]] -name = "p3-poseidon2" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "p3-field", - "p3-mds", - "p3-symmetric", - "p3-util", - "rand 0.10.1", -] - -[[package]] -name = "p3-symmetric" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "itertools 0.14.0", - "p3-field", - "p3-util", - "serde", -] - -[[package]] -name = "p3-util" -version = "0.5.1" -source = "git+https://github.com/Plonky3/Plonky3.git#3f67d136c71bec40f180c85d0bb2b654acddef22" -dependencies = [ - "serde", - "transpose", -] - [[package]] name = "parallel" version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" dependencies = [ - "system-info", + "libc", ] [[package]] @@ -5491,6 +5180,19 @@ version = "1.0.15" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a" +[[package]] +name = "pcs" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" +dependencies = [ + "fiat_shamir", + "parallel", + "primitives", + "serde", + "tracing", + "zk_alloc", +] + [[package]] name = "pem" version = "3.0.6" @@ -5526,39 +5228,6 @@ dependencies = [ "ucd-trie", ] -[[package]] -name = "pest_derive" -version = "2.8.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "11f486f1ea21e6c10ed15d5a7c77165d0ee443402f0780849d1768e7d9d6fe77" -dependencies = [ - "pest", - "pest_generator", -] - -[[package]] -name = "pest_generator" -version = "2.8.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8040c4647b13b210a963c1ed407c1ff4fdfa01c31d6d2a098218702e6664f94f" -dependencies = [ - "pest", - "pest_meta", - "proc-macro2", - "quote", - "syn 2.0.117", -] - -[[package]] -name = "pest_meta" -version = "2.8.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "89815c69d36021a140146f26659a81d6c2afa33d216d736dd4be5381a7362220" -dependencies = [ - "pest", - "sha2", -] - [[package]] name = "pin-project" version = "1.1.13" @@ -5644,6 +5313,15 @@ version = "1.13.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49" +[[package]] +name = "portable-atomic-util" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a106d1259c23fac8e543272398ae0e3c0b8d33c88ed73d0cc71b0f1d902618" +dependencies = [ + "portable-atomic", +] + [[package]] name = "postcard" version = "1.1.3" @@ -5739,6 +5417,19 @@ dependencies = [ "uint 0.10.0", ] +[[package]] +name = "primitives" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" +dependencies = [ + "libc", + "parallel", + "serde", + "tracing-forest", + "tracing-subscriber", + "zk_alloc", +] + [[package]] name = "proc-macro-crate" version = "3.5.0" @@ -5803,7 +5494,7 @@ checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744" dependencies = [ "bit-set", "bit-vec", - "bitflags", + "bitflags 2.11.1", "num-traits", "rand 0.9.4", "rand_chacha 0.9.0", @@ -5831,7 +5522,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "27c6023962132f4b30eb4c172c91ce92d933da334c59c23cddee82358ddafb0b" dependencies = [ "anyhow", - "itertools 0.10.5", + "itertools 0.12.1", "proc-macro2", "quote", "syn 2.0.117", @@ -5927,7 +5618,7 @@ dependencies = [ "quinn-udp", "rustc-hash", "rustls", - "socket2 0.5.10", + "socket2 0.6.3", "thiserror 2.0.18", "tokio", "tracing", @@ -5964,7 +5655,7 @@ dependencies = [ "cfg_aliases", "libc", "once_cell", - "socket2 0.5.10", + "socket2 0.6.3", "tracing", "windows-sys 0.60.2", ] @@ -6178,25 +5869,20 @@ dependencies = [ [[package]] name = "rec_aggregation" version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" dependencies = [ - "backend", - "include_dir", + "bincode", + "flock", "lean_compiler", - "lean_prover", "lean_vm", - "leansig_wrapper", - "lz4_flex", - "objc2", - "objc2-foundation", - "postcard", - "rand 0.10.1", + "parallel", + "pcs", + "primitives", + "rand 0.9.4", "serde", - "sha3 0.11.0", - "sub_protocols", + "sphincs", "tracing", - "utils", - "zk-alloc", + "xmss", ] [[package]] @@ -6205,27 +5891,27 @@ version = "0.5.18" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" dependencies = [ - "bitflags", + "bitflags 2.11.1", ] [[package]] name = "ref-cast" -version = "1.0.26" +version = "1.0.27" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "216e8f773d7923bcba9ceb86a86c93cabb3903a11872fc3f138c49630e50b96d" +checksum = "7e440fb4e4b4147295338efb76001ab9e4efc0e5839df2c47fc5ac2381d365c3" dependencies = [ "ref-cast-impl", ] [[package]] name = "ref-cast-impl" -version = "1.0.26" +version = "1.0.27" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2c9283685feec7d69af75fb0e858d5e7378f33fe4fc699383b2916ab9273e03c" +checksum = "92ecd8964f8453721699a1ed72037b0db49ce2f5a5138486ee89bed6f67cdf3a" dependencies = [ "proc-macro2", "quote", - "syn 3.0.3", + "syn 3.0.4", ] [[package]] @@ -6515,11 +6201,11 @@ version = "1.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" dependencies = [ - "bitflags", + "bitflags 2.11.1", "errno", "libc", "linux-raw-sys", - "windows-sys 0.61.2", + "windows-sys 0.52.0", ] [[package]] @@ -6803,9 +6489,9 @@ dependencies = [ [[package]] name = "serde_with" -version = "3.21.0" +version = "3.22.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "76a5c54c7310e7b8b9577c286d7e399ddd876c3e12b3ed917a8aabc4b96e9e8c" +checksum = "ee78f1fbe43ac4a0e47aadb3dbd357b69eb0d3793e948624cd03dd2750ab1c0a" dependencies = [ "base64", "bs58", @@ -6813,6 +6499,7 @@ dependencies = [ "hex", "indexmap 1.9.3", "indexmap 2.14.0", + "jiff", "schemars 0.9.0", "schemars 1.2.2", "serde_core", @@ -6823,9 +6510,9 @@ dependencies = [ [[package]] name = "serde_with_macros" -version = "3.21.0" +version = "3.22.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "84d57bc0c8b9a17920c178daa6bb924850d54a9c97ab45194bb8c17ad66bb660" +checksum = "8705578779c2b6bd90d84d66eb2e206b708b1a4d7b9f17641b293545bf1c7e46" dependencies = [ "darling 0.23.0", "proc-macro2", @@ -7003,7 +6690,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3a766e1110788c36f4fa1c2b71b387a7815aa65f88ce0229841826633d93723e" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -7061,19 +6748,21 @@ dependencies = [ ] [[package]] -name = "spin" -version = "0.9.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6980e8d7511241f8acf4aebddbb1ff938df5eebe98691418c4468d0b72a96a67" +name = "sphincs" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" dependencies = [ - "lock_api", + "parallel", + "primitives", + "rand 0.9.4", + "serde", ] [[package]] name = "spin" -version = "0.10.0" +version = "0.9.8" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d5fe4ccb98d9c292d56fec89a5e07da7fc4cf0dc11e156b41793132775d3e591" +checksum = "6980e8d7511241f8acf4aebddbb1ff938df5eebe98691418c4468d0b72a96a67" dependencies = [ "lock_api", ] @@ -7106,12 +6795,6 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f446288b699d66d0fd2e30d1cfe7869194312524b3b9252594868ed26ef056a" -[[package]] -name = "strength_reduce" -version = "0.2.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fe895eb47f22e2ddd4dabc02bce419d2e643c8e3b585c78158b349195bc24d82" - [[package]] name = "strsim" version = "0.11.1" @@ -7139,17 +6822,6 @@ dependencies = [ "syn 2.0.117", ] -[[package]] -name = "sub_protocols" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "backend", - "lean_vm", - "tracing", - "utils", -] - [[package]] name = "subtle" version = "2.6.1" @@ -7180,9 +6852,9 @@ dependencies = [ [[package]] name = "syn" -version = "3.0.3" +version = "3.0.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3" +checksum = "e6275cddf4610d1775e6d1fe9469b2e77d0f39fd98fb7450901b821e0c53649f" dependencies = [ "proc-macro2", "quote", @@ -7229,7 +6901,7 @@ version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a13f3d0daba03132c0aa9767f98351b3488edc2c100cda2d2ec2b04f3d8d3c8b" dependencies = [ - "bitflags", + "bitflags 2.11.1", "core-foundation", "system-configuration-sys", ] @@ -7244,14 +6916,6 @@ dependencies = [ "libc", ] -[[package]] -name = "system-info" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "libc", -] - [[package]] name = "tagptr" version = "0.2.0" @@ -7274,7 +6938,7 @@ dependencies = [ "getrandom 0.4.2", "once_cell", "rustix", - "windows-sys 0.61.2", + "windows-sys 0.52.0", ] [[package]] @@ -7559,7 +7223,7 @@ version = "0.6.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840" dependencies = [ - "bitflags", + "bitflags 2.11.1", "bytes", "futures-util", "http", @@ -7658,28 +7322,12 @@ dependencies = [ "tracing-log", ] -[[package]] -name = "transpose" -version = "0.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1ad61aed86bc3faea4300c7aee358b4c6d0c8d6ccc36524c96e4c92ccf26e77e" -dependencies = [ - "num-integer", - "strength_reduce", -] - [[package]] name = "try-lock" version = "0.2.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" -[[package]] -name = "twox-hash" -version = "2.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9ea3136b675547379c4bd395ca6b938e5ad3c3d20fad76e7fe85f9e0d011419c" - [[package]] name = "typenum" version = "1.20.0" @@ -7792,17 +7440,6 @@ version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" -[[package]] -name = "utils" -version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" -dependencies = [ - "backend", - "tracing", - "tracing-forest", - "tracing-subscriber", -] - [[package]] name = "uuid" version = "1.23.1" @@ -8018,7 +7655,7 @@ version = "0.244.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe" dependencies = [ - "bitflags", + "bitflags 2.11.1", "hashbrown 0.15.5", "indexmap 2.14.0", "semver 1.0.28", @@ -8100,7 +7737,7 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.52.0", ] [[package]] @@ -8512,7 +8149,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2" dependencies = [ "anyhow", - "bitflags", + "bitflags 2.11.1", "indexmap 2.14.0", "log", "serde", @@ -8601,6 +8238,18 @@ dependencies = [ "xml-rs", ] +[[package]] +name = "xmss" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" +dependencies = [ + "ethereum_ssz", + "parallel", + "primitives", + "rand 0.9.4", + "serde", +] + [[package]] name = "yamux" version = "0.12.1" @@ -8759,13 +8408,11 @@ dependencies = [ ] [[package]] -name = "zk-alloc" +name = "zk_alloc" version = "0.1.0" -source = "git+https://github.com/leanEthereum/leanVM.git?rev=e2592df#e2592df4e30fdddbbf8ae26a333116c68cec7026" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=5a4f55c1138759f43f78483a7b70fde973e4a1ee#5a4f55c1138759f43f78483a7b70fde973e4a1ee" dependencies = [ "libc", - "parallel", - "system-info", ] [[package]] diff --git a/Cargo.toml b/Cargo.toml index 1d41c5a7..eef00613 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -81,8 +81,15 @@ prometheus = "0.14" clap = { version = "4.3", features = ["derive", "env"] } -# XMSS signatures -leansig = { git = "https://github.com/leanEthereum/leanSig", branch = "devnet4" } +# XMSS signatures + recursive aggregation, through leanVM's own facade crate: +# it re-exports the aggregation API, the `xmss` module (keys, signing, and the +# SSZ codec for the two wire types) and the `rand` that signing draws from, so +# the whole crypto stack arrives as one dependency at one revision. +# Pinned to a `main` commit for reproducible builds; bump the rev to track main. +leanvm = { git = "https://github.com/leanEthereum/leanVM.git", rev = "5a4f55c1138759f43f78483a7b70fde973e4a1ee" } + +# Secret-key (de)serialization for the leanVM xmss key format. +postcard = { version = "1.1.3", features = ["alloc"] } # SSZ implementation libssz = "0.3.0" diff --git a/bin/ethlambda/Cargo.toml b/bin/ethlambda/Cargo.toml index 591490ca..a22f0447 100644 --- a/bin/ethlambda/Cargo.toml +++ b/bin/ethlambda/Cargo.toml @@ -56,6 +56,8 @@ libc.workspace = true # tests would otherwise wait out the real retry backoff. Dev-only, so the # feature never reaches the shipped binary. tokio = { workspace = true, features = ["test-util"] } +# Keygen writes a directory of key files, so its tests need a throwaway one. +tempfile = "3" [build-dependencies] vergen-git2.workspace = true diff --git a/bin/ethlambda/build.rs b/bin/ethlambda/build.rs index aea5f542..78f8534d 100644 --- a/bin/ethlambda/build.rs +++ b/bin/ethlambda/build.rs @@ -2,11 +2,11 @@ use std::path::PathBuf; use vergen_git2::{Emitter, Git2Builder, RustcBuilder}; -/// Crate names whose resolved git revision is embedded in the binary, one per -/// upstream crypto repository: `leansig` for leanSig, `lean-multisig` for -/// leanVM (the direct dependency `ethlambda-crypto` builds against). -const LEANSIG_PACKAGE: &str = "leansig"; -const LEANVM_PACKAGE: &str = "lean-multisig"; +/// Crate name whose resolved git revision is embedded in the binary. +/// leanVM owns the whole crypto stack (it internalized XMSS), and every crate +/// ethlambda takes from it resolves to one revision, so this single pin +/// identifies the crypto the binary was built against. +const LEANVM_PACKAGE: &str = "leanvm"; fn main() -> Result<(), Box> { let git2 = Git2Builder::default().branch(true).sha(true).build()?; @@ -25,25 +25,16 @@ fn main() -> Result<(), Box> { Ok(()) } -/// Embed the resolved leanSig and leanVM git revisions from the workspace -/// Cargo.lock. +/// Embed the resolved leanVM git revision from the workspace Cargo.lock. /// -/// The crypto dependencies are pinned upstream (leansig to a moving branch, -/// leanVM to a rev), so a `cargo update` or a rev bump changes the measured -/// crypto with little or no ethlambda diff; benchmark reports embed these -/// revisions to keep results interpretable across lock bumps. +/// leanVM is pinned to a rev upstream, so a `cargo update` or a rev bump +/// changes the measured crypto with little or no ethlambda diff; benchmark +/// reports embed the revision to keep results interpretable across lock bumps. fn emit_crypto_revs() { - let revs = lockfile_git_revs(); - for (package, env_var) in [ - (LEANSIG_PACKAGE, "ETHLAMBDA_LEANSIG_REV"), - (LEANVM_PACKAGE, "ETHLAMBDA_LEANVM_REV"), - ] { - let rev = revs - .as_ref() - .and_then(|revs| revs.get(package).cloned()) - .unwrap_or_else(|| "unknown".to_string()); - println!("cargo:rustc-env={env_var}={rev}"); - } + let rev = lockfile_git_revs() + .and_then(|revs| revs.get(LEANVM_PACKAGE).cloned()) + .unwrap_or_else(|| "unknown".to_string()); + println!("cargo:rustc-env=ETHLAMBDA_LEANVM_REV={rev}"); if let Some(lockfile) = workspace_lockfile() { println!("cargo:rerun-if-changed={}", lockfile.display()); } @@ -77,7 +68,7 @@ fn lockfile_git_revs() -> Option> { source = Some(value.trim_matches('"').to_string()); } } - // source = "git+https://github.com/leanEthereum/leanSig?branch=devnet4#" + // source = "git+https://github.com/leanEthereum/leanVM.git?rev=#" let (Some(name), Some(source)) = (name, source) else { continue; }; diff --git a/bin/ethlambda/src/benchmark/corpus.rs b/bin/ethlambda/src/benchmark/corpus.rs index bc007c54..6136d19a 100644 --- a/bin/ethlambda/src/benchmark/corpus.rs +++ b/bin/ethlambda/src/benchmark/corpus.rs @@ -9,7 +9,7 @@ use ethlambda_types::{ attestation::{AggregationBits, HashedAttestationData}, block::SingleMessageAggregate, constants::DEFAULT_MILLISECONDS_PER_SLOT, - state::{State, Validator, ValidatorPubkeyBytes}, + state::{PUBLIC_KEY_SIZE, State, Validator, ValidatorPubkeyBytes}, }; /// Fixed genesis time for synthetic runs. The harness derives every tick @@ -124,7 +124,7 @@ fn splitmix64(state: &mut u64) -> u64 { } fn synthetic_pubkey(rng_state: &mut u64) -> ValidatorPubkeyBytes { - let mut bytes = [0u8; 52]; + let mut bytes = [0u8; PUBLIC_KEY_SIZE]; for chunk in bytes.chunks_mut(8) { let word = splitmix64(rng_state).to_le_bytes(); chunk.copy_from_slice(&word[..chunk.len()]); diff --git a/bin/ethlambda/src/benchmark/report.rs b/bin/ethlambda/src/benchmark/report.rs index 34e8014c..b1d14c4c 100644 --- a/bin/ethlambda/src/benchmark/report.rs +++ b/bin/ethlambda/src/benchmark/report.rs @@ -41,11 +41,9 @@ pub(crate) struct Sample { #[derive(Debug, Serialize)] pub(crate) struct Environment { pub client_version: &'static str, - /// Resolved leansig git revision from Cargo.lock. leansig is pinned to a - /// moving branch, so results are not comparable across revisions. - pub leansig_rev: &'static str, - /// Resolved leanVM git revision from Cargo.lock. leanVM does the signature - /// aggregation, so a rev bump moves the measured crypto too. + /// Resolved leanVM git revision from Cargo.lock. leanVM owns the whole + /// signature stack (XMSS and aggregation), so a rev bump moves the + /// measured crypto and results are not comparable across revisions. pub leanvm_rev: &'static str, pub os: &'static str, pub arch: &'static str, @@ -56,7 +54,6 @@ impl Environment { pub(crate) fn collect() -> Self { Self { client_version: version::CLIENT_VERSION, - leansig_rev: env!("ETHLAMBDA_LEANSIG_REV"), leanvm_rev: env!("ETHLAMBDA_LEANVM_REV"), os: std::env::consts::OS, arch: std::env::consts::ARCH, @@ -185,13 +182,8 @@ impl Report { ); let _ = writeln!( out, - " {} leansig={} leanvm={} os={} arch={} threads={}", - env.client_version, - env.leansig_rev, - env.leanvm_rev, - env.os, - env.arch, - env.available_parallelism + " {} leanvm={} os={} arch={} threads={}", + env.client_version, env.leanvm_rev, env.os, env.arch, env.available_parallelism ); let _ = writeln!(out); diff --git a/bin/ethlambda/src/checkpoint_sync.rs b/bin/ethlambda/src/checkpoint_sync.rs index e18ef8b6..535dc631 100644 --- a/bin/ethlambda/src/checkpoint_sync.rs +++ b/bin/ethlambda/src/checkpoint_sync.rs @@ -374,16 +374,16 @@ mod tests { fn create_test_validator() -> Validator { Validator { - attestation_pubkey: [1u8; 52], - proposal_pubkey: [11u8; 52], + attestation_pubkey: [1u8; 32], + proposal_pubkey: [11u8; 32], index: 0, } } fn create_different_validator() -> Validator { Validator { - attestation_pubkey: [2u8; 52], - proposal_pubkey: [22u8; 52], + attestation_pubkey: [2u8; 32], + proposal_pubkey: [22u8; 32], index: 0, } } @@ -391,8 +391,8 @@ mod tests { fn create_validators_with_indices(count: usize) -> Vec { (0..count) .map(|i| Validator { - attestation_pubkey: [i as u8 + 1; 52], - proposal_pubkey: [i as u8 + 101; 52], + attestation_pubkey: [i as u8 + 1; 32], + proposal_pubkey: [i as u8 + 101; 32], index: i as u64, }) .collect() diff --git a/bin/ethlambda/src/cli.rs b/bin/ethlambda/src/cli.rs index aa5418d0..f65ae10c 100644 --- a/bin/ethlambda/src/cli.rs +++ b/bin/ethlambda/src/cli.rs @@ -110,6 +110,19 @@ pub(crate) struct NodeOptions { /// coverage. #[arg(long, default_value = "false")] pub(crate) enable_proposer_aggregation: bool, + /// Prove on leanVM's bump arena instead of the system allocator. + /// + /// Buys proving throughput with memory: the arena's slabs stay faulted in + /// across proofs (a phase reset abandons their contents, it does not return + /// the pages), and on Linux it also stops glibc trimming its own heap. RSS + /// therefore ratchets up to the process's allocation high-water mark and + /// stays there for the lifetime of the node. Off by default so a long-lived + /// node keeps bounded memory; worth enabling on hosts with memory to spare + /// where proving latency is the constraint. + /// + /// Read once at startup: the allocator is fixed before the first proof. + #[arg(long, default_value = "false")] + pub(crate) prover_arena: bool, /// Maximum number of distinct attestations to pack when building a block. /// /// Bounds how many distinct `AttestationData` entries the proposer includes diff --git a/bin/ethlambda/src/command.rs b/bin/ethlambda/src/command.rs index 4483a974..85a61e7a 100644 --- a/bin/ethlambda/src/command.rs +++ b/bin/ethlambda/src/command.rs @@ -14,14 +14,25 @@ use clap::Parser; use crate::benchmark::BenchmarkOptions; use crate::cli::NodeOptions; +use crate::keygen::KeygenOptions; use crate::version; /// Tokens that already say what to run, so no default is inserted ahead of /// them. `help` is clap's own generated sub-command (`ethlambda help node`). -const EXPLICIT: &[&str] = &[NODE, BENCHMARK, "help", "-h", "--help", "-V", "--version"]; +const EXPLICIT: &[&str] = &[ + NODE, + BENCHMARK, + KEYGEN, + "help", + "-h", + "--help", + "-V", + "--version", +]; const NODE: &str = "node"; const BENCHMARK: &str = "benchmark"; +const KEYGEN: &str = "keygen"; #[derive(Debug, clap::Parser)] #[command( @@ -61,6 +72,8 @@ pub(crate) enum Command { Node(NodeOptions), /// Benchmark block building offline against a controlled workload. Benchmark(BenchmarkOptions), + /// Generate validator XMSS keys for a genesis. + Keygen(KeygenOptions), } /// Parse the process arguments, exiting the way clap does on a parse error, @@ -259,6 +272,72 @@ mod tests { assert_eq!(printed[0], printed[2]); } + /// Every sub-command token has to be in `EXPLICIT`, or the default is + /// inserted ahead of it and `ethlambda keygen ...` parses as + /// `ethlambda node keygen ...`, which fails on a stray positional. + #[test] + fn sub_command_tokens_do_not_get_the_default_inserted() { + for token in [NODE, BENCHMARK, KEYGEN] { + let args = [OsString::from("ethlambda"), OsString::from(token)]; + assert_eq!( + default_subcommand(&args), + None, + "`{token}` must be recognised as a sub-command" + ); + } + } + + #[test] + fn keygen_parses_as_its_own_sub_command() { + let args = [ + "ethlambda", + KEYGEN, + "--num-validators", + "4", + "--output-dir", + "keys", + ]; + match try_parse_from(args.iter().map(OsString::from)).expect("keygen parses") { + Command::Keygen(_) => {} + other => panic!("expected a keygen invocation, got {other:?}"), + } + } + + /// The output directory is the only thing keygen cannot guess, so a bare + /// `keygen --output-dir` has to parse. The default it lands on is asserted + /// through `Debug`, as the node options are above, rather than by widening + /// the parser's fields for a test. + #[test] + fn keygen_defaults_to_a_single_validator() { + let args = ["ethlambda", KEYGEN, "--output-dir", "keys"]; + match try_parse_from(args.iter().map(OsString::from)).expect("keygen parses") { + Command::Keygen(options) => { + assert!( + format!("{options:?}").contains("num_validators: 1"), + "{options:?}" + ); + } + other => panic!("expected a keygen invocation, got {other:?}"), + } + } + + /// `--num-validators 0` would write a manifest with no validators in it, + /// which a genesis generator then sums to a zero validator count. + #[test] + fn keygen_rejects_an_empty_validator_set() { + let args = [ + "ethlambda", + KEYGEN, + "--num-validators", + "0", + "--output-dir", + "keys", + ]; + let err = try_parse_from(args.iter().map(OsString::from)) + .expect_err("zero validators must be rejected"); + assert_eq!(err.kind(), ErrorKind::ValueValidation); + } + #[test] fn help_lists_the_sub_commands() { // Listed by clap itself, because they are real sub-commands. diff --git a/bin/ethlambda/src/keygen.rs b/bin/ethlambda/src/keygen.rs new file mode 100644 index 00000000..acf4062f --- /dev/null +++ b/bin/ethlambda/src/keygen.rs @@ -0,0 +1,397 @@ +//! Validator key generation (`ethlambda keygen`). +//! +//! Writes the key set and manifest that `--hash-sig-keys-dir` reads, in the +//! layout `hash-sig-cli generate` produces, so `generate-genesis.sh` can call +//! this instead. Only the SSZ-and-postcard form is written: `hash-sig-cli` also +//! dumped each key as serde JSON, which its own help called legacy and nothing +//! reads. +//! +//! # Why the node generates its own keys +//! +//! An XMSS key file is only usable by a client built against the same scheme, +//! and the scheme lives in leanVM, which ethlambda pins. Nothing in the file +//! layout changes when the scheme does: leanVM's move from Poseidon over +//! KoalaBear to BLAKE2s over binary fields kept the public key at +//! [`PUBLIC_KEY_SIZE`] bytes and the secret key in postcard, so a key set from +//! the wrong revision passes every format check a genesis generator makes and +//! fails only once a signature is verified. +//! +//! Generating here removes the second pin that has to be kept in step: these +//! keys come from the same `ethlambda-crypto` types the node loads them with, +//! so the two cannot disagree. The manifest also records the leanVM revision +//! the binary was built against, which is the one field that cannot drift. + +use std::fs; +use std::io::Write as _; +use std::path::{Path, PathBuf}; + +use ethlambda_crypto::signature::{ValidatorSecretKey, scheme}; +use ethlambda_types::state::PUBLIC_KEY_SIZE; +use eyre::{Context as _, bail}; +use tracing::info; + +#[derive(Debug, clap::Args)] +pub(crate) struct KeygenOptions { + /// Number of validators to generate a key pair for. + /// + /// Each validator gets two independent keys, an attester and a proposer, so + /// that it can sign an attestation and a block in the same slot. The + /// default generates one validator's pair, which is the shape for + /// inspecting a key or replacing a single node's. + #[arg(long, default_value_t = 1, value_parser = clap::value_parser!(u32).range(1..))] + num_validators: u32, + + /// Log2 of the slots each key can sign at, counted from slot 0. + /// + /// The default matches what the devnets generate. A key cannot sign past + /// its range, so this is the network's lifetime: at the default cadence + /// 2^18 slots is about 12 days. + #[arg(long, default_value_t = 18, value_parser = clap::value_parser!(u32).range(1..=32))] + log_num_active_epochs: u32, + + /// Directory to write the keys and manifest into. Created if absent. + #[arg(long)] + output_dir: PathBuf, + + /// Write `validator-keys-manifest.yaml` alongside the keys. + #[arg(long, default_value_t = true, action = clap::ArgAction::Set)] + create_manifest: bool, + + /// Name each validator by the first and last three bytes of its proposer + /// public key rather than by index, for key sets split across hosts. + #[arg(long)] + distributed: bool, + + /// Overwrite key files already in `--output-dir`. + /// + /// Off by default: a key is one-time-use material, and silently replacing a + /// set that validators are already signing with would make every one of + /// them sign twice at the same slot under a key someone else now holds. + #[arg(long)] + force: bool, +} + +/// One validator's generated key pair, as the manifest records it. +struct Validator { + /// `validator_`, or the pubkey-derived name under `--distributed`. + name: String, + proposer_pubkey: Vec, + attester_pubkey: Vec, +} + +pub(crate) fn run(options: KeygenOptions) -> eyre::Result<()> { + let KeygenOptions { + num_validators, + log_num_active_epochs, + output_dir, + create_manifest, + distributed, + force, + } = options; + + // Inclusive, so `2^n` slots is `0..=2^n - 1`. The exclusive reading is an + // easy off-by-one to make here and gives every key one epoch too many. + let last_slot = (1u64 << log_num_active_epochs) - 1; + let last_slot = u32::try_from(last_slot) + .wrap_err_with(|| format!("2^{log_num_active_epochs} slots exceeds the XMSS lifetime"))?; + let slots = 0..=last_slot; + + fs::create_dir_all(&output_dir) + .wrap_err_with(|| format!("failed to create {}", output_dir.display()))?; + + info!( + num_validators, + log_num_active_epochs, + signable_slots = last_slot as u64 + 1, + scheme = scheme::NAME, + output_dir = %output_dir.display(), + "Generating validator keys" + ); + + let mut validators = Vec::with_capacity(num_validators as usize); + for index in 0..num_validators { + // Two keys per validator, each with its own secret material: a shared + // key would spend one slot's leaf on whichever duty signed first. + let proposer = ValidatorSecretKey::generate(slots.clone()) + .wrap_err_with(|| format!("proposer key generation failed for validator {index}"))?; + let attester = ValidatorSecretKey::generate(slots.clone()) + .wrap_err_with(|| format!("attester key generation failed for validator {index}"))?; + + let proposer_pubkey = proposer.public_key().to_bytes(); + let attester_pubkey = attester.public_key().to_bytes(); + let name = validator_name(index, distributed, &proposer_pubkey); + + write_key_pair(&output_dir, &name, "proposer", &proposer, force)?; + write_key_pair(&output_dir, &name, "attester", &attester, force)?; + + info!(index, name, "Generated validator key pair"); + validators.push(Validator { + name, + proposer_pubkey, + attester_pubkey, + }); + } + + if create_manifest { + let path = write_manifest(&output_dir, log_num_active_epochs, distributed, &validators)?; + info!(manifest = %path.display(), "Wrote validator key manifest"); + } + + Ok(()) +} + +/// `validator_`, or `validator--` of the proposer public key. +/// +/// The distributed form lets key directories from separate hosts be merged +/// without their indices colliding, at the cost of an ordering the genesis +/// generator has to take from the manifest rather than from the file names. +fn validator_name(index: u32, distributed: bool, proposer_pubkey: &[u8]) -> String { + if !distributed { + return format!("validator_{index}"); + } + let (first, last) = proposer_pubkey.split_at(3); + format!( + "validator-{}-{}", + hex::encode(first), + hex::encode(&last[last.len() - 3..]) + ) +} + +/// Write one role's `_pk.ssz` and `_sk.ssz`. +/// +/// The secret key is postcard rather than SSZ, which has no encoding for it; +/// the extension is kept so the genesis tooling needs no special case. +fn write_key_pair( + output_dir: &Path, + name: &str, + role: &str, + key: &ValidatorSecretKey, + force: bool, +) -> eyre::Result<()> { + let public = key.public_key().to_bytes(); + debug_assert_eq!(public.len(), PUBLIC_KEY_SIZE); + let secret = key + .to_bytes() + .map_err(|err| eyre::eyre!("failed to encode the {role} secret key: {err}"))?; + + write_new( + &output_dir.join(format!("{name}_{role}_key_pk.ssz")), + &public, + force, + )?; + write_new( + &output_dir.join(format!("{name}_{role}_key_sk.ssz")), + &secret, + force, + ) +} + +/// Write `path`, refusing to replace an existing file unless `force`. +fn write_new(path: &Path, bytes: &[u8], force: bool) -> eyre::Result<()> { + if !force && path.exists() { + bail!( + "{} already exists; pass --force to replace the key set, \ + but only once no validator is still signing with it", + path.display() + ); + } + fs::write(path, bytes).wrap_err_with(|| format!("failed to write {}", path.display())) +} + +/// Write the manifest the genesis generator reads the public keys back from. +/// +/// Hand-rolled rather than serialized from a struct, to hold the field order +/// and the blank lines `hash-sig-cli` produces: a diff between two key sets' +/// manifests is worth keeping readable. +fn write_manifest( + output_dir: &Path, + log_num_active_epochs: u32, + distributed: bool, + validators: &[Validator], +) -> eyre::Result { + let path = output_dir.join("validator-keys-manifest.yaml"); + let mut out = Vec::new(); + + writeln!(out, "# Hash-Signature Validator Keys Manifest")?; + writeln!(out, "# Generated by ethlambda keygen\n")?; + writeln!(out, "key_scheme: {}", scheme::NAME)?; + writeln!(out, "hash_function: {}", scheme::HASH_FUNCTION)?; + writeln!(out, "encoding: {}", scheme::ENCODING)?; + writeln!(out, "pubkey_bytes: {}", scheme::PUBLIC_KEY_BYTES)?; + writeln!(out, "lifetime: {}", scheme::LIFETIME)?; + // The scheme fields above are all derived from parameters that outlived the + // last scheme change, so this is what actually identifies the key format. + writeln!(out, "leanvm_rev: {}", crate::version::LEANVM_REV)?; + writeln!(out, "log_num_active_epochs: {log_num_active_epochs}")?; + writeln!(out, "num_active_epochs: {}", 1u64 << log_num_active_epochs)?; + writeln!(out, "num_validators: {}\n", validators.len())?; + writeln!(out, "validators:")?; + + for (index, validator) in validators.iter().enumerate() { + // The distributed layout names validators by key rather than position, + // so an index would be a second, conflicting identity. + if !distributed { + writeln!(out, " - index: {index}")?; + } + let lead = if distributed { " - " } else { " " }; + writeln!( + out, + "{lead}proposer_key_pubkey_hex: 0x{}", + hex::encode(&validator.proposer_pubkey) + )?; + writeln!( + out, + " proposer_key_privkey_file: {}_proposer_key_sk.ssz", + validator.name + )?; + writeln!( + out, + " attester_key_pubkey_hex: 0x{}", + hex::encode(&validator.attester_pubkey) + )?; + writeln!( + out, + " attester_key_privkey_file: {}_attester_key_sk.ssz", + validator.name + )?; + writeln!(out)?; + } + + fs::write(&path, out).wrap_err_with(|| format!("failed to write {}", path.display()))?; + Ok(path) +} + +#[cfg(test)] +mod tests { + use ethlambda_crypto::signature::ValidatorPublicKey; + + use super::*; + + /// Read a public key back out of a generated `_pk.ssz`, which is what the + /// node's own validator registry does with it. + fn read_public_key(path: &Path) -> eyre::Result { + let bytes = + fs::read(path).wrap_err_with(|| format!("failed to read {}", path.display()))?; + ValidatorPublicKey::from_bytes(&bytes) + .map_err(|err| eyre::eyre!("{} is not a public key: {err}", path.display())) + } + + fn options(dir: &Path, num_validators: u32) -> KeygenOptions { + KeygenOptions { + num_validators, + // The smallest range that still exercises the subtree split, so the + // tests stay fast: key generation cost grows with the range. + log_num_active_epochs: 4, + output_dir: dir.to_path_buf(), + create_manifest: true, + distributed: false, + force: false, + } + } + + #[test] + fn indexed_names_are_positional() { + assert_eq!(validator_name(0, false, &[0xab; 32]), "validator_0"); + assert_eq!(validator_name(17, false, &[0xab; 32]), "validator_17"); + } + + /// The distributed name has to come from the key's own bytes, so two hosts + /// generating independently cannot collide. + #[test] + fn distributed_names_come_from_the_proposer_key() { + let mut pubkey = [0u8; PUBLIC_KEY_SIZE]; + pubkey[..3].copy_from_slice(&[0x01, 0x02, 0x03]); + pubkey[PUBLIC_KEY_SIZE - 3..].copy_from_slice(&[0x0a, 0x0b, 0x0c]); + + let name = validator_name(0, true, &pubkey); + assert_eq!(name, "validator-010203-0a0b0c"); + // The index is not in it, so the same key names the same validator + // wherever it was generated. + assert_eq!(name, validator_name(9, true, &pubkey)); + } + + /// The whole point of generating here: what comes out has to be what the + /// node's key loader reads back, both keys and manifest. + #[test] + #[ignore = "slow: XMSS key generation"] + fn generated_set_is_loadable_and_matches_its_manifest() { + let dir = tempfile::tempdir().expect("tempdir"); + run(options(dir.path(), 2)).expect("keygen succeeds"); + + let manifest = fs::read_to_string(dir.path().join("validator-keys-manifest.yaml")) + .expect("manifest written"); + assert!(manifest.contains("num_validators: 2"), "{manifest}"); + assert!(manifest.contains("num_active_epochs: 16"), "{manifest}"); + assert!( + manifest.contains(&format!("pubkey_bytes: {PUBLIC_KEY_SIZE}")), + "{manifest}" + ); + + for index in 0..2 { + for role in ["proposer", "attester"] { + let name = format!("validator_{index}"); + let public = read_public_key(&dir.path().join(format!("{name}_{role}_key_pk.ssz"))) + .expect("public key parses"); + let secret = fs::read(dir.path().join(format!("{name}_{role}_key_sk.ssz"))) + .expect("secret key written"); + let secret = ValidatorSecretKey::from_bytes(&secret).expect("secret key parses"); + + // The pair has to belong together, and the manifest has to + // name the same public key the file holds. + assert_eq!(secret.public_key().to_bytes(), public.to_bytes()); + assert_eq!(secret.signable_slots(), 0..=15); + let hex = format!("0x{}", hex::encode(public.to_bytes())); + assert!( + manifest.contains(&hex), + "{role} {index} missing from manifest" + ); + } + } + } + + /// Two validators must not share key material, however close together they + /// were generated. + #[test] + #[ignore = "slow: XMSS key generation"] + fn every_generated_key_is_distinct() { + let dir = tempfile::tempdir().expect("tempdir"); + run(options(dir.path(), 2)).expect("keygen succeeds"); + + let mut seen = std::collections::HashSet::new(); + for index in 0..2 { + for role in ["proposer", "attester"] { + let path = dir + .path() + .join(format!("validator_{index}_{role}_key_pk.ssz")); + let public = read_public_key(&path).expect("public key parses"); + assert!( + seen.insert(public.to_bytes()), + "validator_{index} {role} repeats another key" + ); + } + } + } + + /// Replacing a live key set would make every validator in it sign twice at + /// the same slot, so it takes an explicit `--force`. + #[test] + #[ignore = "slow: XMSS key generation"] + fn an_existing_key_set_is_not_overwritten_by_default() { + let dir = tempfile::tempdir().expect("tempdir"); + run(options(dir.path(), 1)).expect("first keygen succeeds"); + let before = fs::read(dir.path().join("validator_0_proposer_key_sk.ssz")).expect("written"); + + let err = run(options(dir.path(), 1)).expect_err("a second run must refuse"); + assert!(err.to_string().contains("--force"), "{err}"); + + let after = fs::read(dir.path().join("validator_0_proposer_key_sk.ssz")).expect("intact"); + assert_eq!(before, after, "the refused run must not have written"); + + let mut forced = options(dir.path(), 1); + forced.force = true; + run(forced).expect("--force replaces the set"); + let replaced = + fs::read(dir.path().join("validator_0_proposer_key_sk.ssz")).expect("written"); + assert_ne!(before, replaced, "--force must generate fresh material"); + } +} diff --git a/bin/ethlambda/src/main.rs b/bin/ethlambda/src/main.rs index 48611489..70cb5017 100644 --- a/bin/ethlambda/src/main.rs +++ b/bin/ethlambda/src/main.rs @@ -3,6 +3,7 @@ mod checkpoint_sync; mod cli; mod command; mod fd_limit; +mod keygen; mod version; // Jemalloc causes programs to deadlock during process startup under Shadow. @@ -82,6 +83,12 @@ fn main() -> eyre::Result<()> { init_benchmark_logging()?; benchmark::run(options) } + // Key generation is synchronous, CPU-bound work whose product is files, + // so it runs on this thread too and leaves stdout alone. + Command::Keygen(options) => { + init_keygen_logging()?; + keygen::run(options) + } } } @@ -110,6 +117,22 @@ fn init_benchmark_logging() -> eyre::Result<()> { .wrap_err("failed to set global tracing subscriber") } +/// Keygen logging: INFO and above, on stderr. Key generation takes minutes for a +/// large validator set, so progress has to be visible; stderr keeps stdout free +/// for a machine-readable summary to claim later. +fn init_keygen_logging() -> eyre::Result<()> { + let filter = EnvFilter::builder() + .with_default_directive(tracing::Level::INFO.into()) + .from_env_lossy(); + let subscriber = Registry::default().with( + tracing_subscriber::fmt::layer() + .with_writer(std::io::stderr) + .with_filter(filter), + ); + tracing::subscriber::set_global_default(subscriber) + .wrap_err("failed to set global tracing subscriber") +} + // Shadow single-steps execution in a discrete-event simulation, so the default // multi-threaded runtime's worker threads add only scheduling noise, never // parallelism. Use a single-threaded runtime under Shadow. This is an @@ -122,6 +145,14 @@ async fn run_node(options: NodeOptions) -> eyre::Result<()> { #[cfg(feature = "shadow-integration")] init_shadow_cost(&options.shadow); + // Compiles the aggregation bytecode and fixes the prover's allocator. Ahead of the + // test-driver branch below, which verifies signatures, and of every consensus path. + info!( + arena = options.prover_arena, + "Initializing leanVM prover and verifier" + ); + ethlambda_crypto::init_leanvm(options.prover_arena); + // Initialize metrics ethlambda_blockchain::metrics::init(); ethlambda_blockchain::metrics::set_node_info("ethlambda", version::CLIENT_VERSION); @@ -534,8 +565,8 @@ fn read_bootnodes(bootnodes_path: impl AsRef) -> eyre::Result Vec { (0..n) .map(|i| Validator { - attestation_pubkey: [i as u8; 52], - proposal_pubkey: [i as u8; 52], + attestation_pubkey: [i as u8; 32], + proposal_pubkey: [i as u8; 32], index: i as u64, }) .collect() } - /// A cheap-but-real XMSS signature (tiny lifetime, cached) for tests that - /// only need `ValidatorSignature::from_bytes` to succeed. `resolve_job` - /// never checks signature validity, only that it clones and carries a - /// resolvable id — mirrors `ethlambda_storage::store::tests::make_dummy_sig`. + /// A structurally valid XMSS signature for tests that only need + /// `ValidatorSignature::from_bytes` to succeed. `resolve_job` never checks + /// signature validity, only that it clones and carries a resolvable id — + /// mirrors `ethlambda_storage::store::tests::make_dummy_sig`. An all-zero + /// blob decodes as a valid (unverifiable) signature. fn dummy_sig() -> ValidatorSignature { - use ethlambda_crypto::signature::LeanSignatureScheme; - use leansig::{serialization::Serializable, signature::SignatureScheme}; - use rand::{SeedableRng, rngs::StdRng}; - - static CACHED_SIG: std::sync::LazyLock> = std::sync::LazyLock::new(|| { - let mut rng = StdRng::seed_from_u64(42); - let lifetime = 1 << 5; // small for speed - let (_pk, sk) = LeanSignatureScheme::key_gen(&mut rng, 0, lifetime); - let sig = LeanSignatureScheme::sign(&sk, 0, &[0u8; 32]).unwrap(); - sig.to_bytes() - }); - - ValidatorSignature::from_bytes(&CACHED_SIG).expect("cached test signature") + use ethlambda_types::attestation::SIGNATURE_SIZE; + ValidatorSignature::from_bytes(&vec![0u8; SIGNATURE_SIZE]) + .expect("all-zero test signature decodes") } /// A `HashedAttestationData` over default (all-zero) data for `resolve_job` diff --git a/crates/blockchain/src/block_builder.rs b/crates/blockchain/src/block_builder.rs index f7f4f25e..59fbb1ec 100644 --- a/crates/blockchain/src/block_builder.rs +++ b/crates/blockchain/src/block_builder.rs @@ -987,8 +987,8 @@ mod tests { let validators: Vec<_> = (0..NUM_VALIDATORS) .map(|i| ethlambda_types::state::Validator { - attestation_pubkey: [i as u8; 52], - proposal_pubkey: [i as u8; 52], + attestation_pubkey: [i as u8; 32], + proposal_pubkey: [i as u8; 32], index: i as u64, }) .collect(); @@ -1101,7 +1101,7 @@ mod tests { ); // Substitute a worst-case-size proof to model what `propose_block` - // would attach. The actual SNARK can't be built without lean-multisig, + // would attach. The actual SNARK can't be built without leanVM, // but the size cap (`ByteList512KiB`) bounds the worst case. let _ = signatures; let proof = MultiMessageAggregate::new( @@ -1147,8 +1147,8 @@ mod tests { let validators: Vec<_> = (0..NUM_VALIDATORS) .map(|i| ethlambda_types::state::Validator { - attestation_pubkey: [i as u8; 52], - proposal_pubkey: [i as u8; 52], + attestation_pubkey: [i as u8; 32], + proposal_pubkey: [i as u8; 32], index: i as u64, }) .collect(); @@ -1276,8 +1276,8 @@ mod tests { let validators: Vec<_> = (0..NUM_VALIDATORS) .map(|i| ethlambda_types::state::Validator { - attestation_pubkey: [i as u8; 52], - proposal_pubkey: [i as u8; 52], + attestation_pubkey: [i as u8; 32], + proposal_pubkey: [i as u8; 32], index: i as u64, }) .collect(); @@ -1580,8 +1580,8 @@ mod tests { let validators: Vec<_> = (0..NUM_VALIDATORS) .map(|i| ethlambda_types::state::Validator { - attestation_pubkey: [i as u8; 52], - proposal_pubkey: [i as u8; 52], + attestation_pubkey: [i as u8; 32], + proposal_pubkey: [i as u8; 32], index: i as u64, }) .collect(); @@ -1707,8 +1707,8 @@ mod tests { let validators: Vec<_> = (0..NUM_VALIDATORS) .map(|i| ethlambda_types::state::Validator { - attestation_pubkey: [i as u8; 52], - proposal_pubkey: [i as u8; 52], + attestation_pubkey: [i as u8; 32], + proposal_pubkey: [i as u8; 32], index: i as u64, }) .collect(); diff --git a/crates/blockchain/src/key_manager.rs b/crates/blockchain/src/key_manager.rs index eb2777a8..26196ec4 100644 --- a/crates/blockchain/src/key_manager.rs +++ b/crates/blockchain/src/key_manager.rs @@ -6,7 +6,7 @@ use ethlambda_types::{ attestation::{AttestationData, XmssSignature}, primitives::{H256, HashTreeRoot as _}, }; -use tracing::{info, warn}; +use tracing::{trace, warn}; use crate::metrics; @@ -23,9 +23,8 @@ pub enum KeyManagerError { /// A validator's dual XMSS key pair for attestation and block proposal signing. /// -/// Each key is independent and advances its OTS preparation separately, -/// allowing the validator to sign both an attestation and a block proposal -/// within the same slot. +/// Each key holds its own one-time leaves, so the validator can sign both an +/// attestation and a block proposal within the same slot. pub struct ValidatorKeyPair { pub attestation_key: ValidatorSecretKey, pub proposal_key: ValidatorSecretKey, @@ -49,14 +48,19 @@ impl KeyManager { self.keys.keys().copied().collect() } - /// Advances every validator's XMSS preparation windows to cover slot - pub fn advance_keys_to(&mut self, slot: u32) { - for (validator_id, key_pair) in self.keys.iter_mut() { - let _ = advance_key(*validator_id, &mut key_pair.attestation_key, slot).inspect_err( - |err| warn!(validator_id, slot, %err, "Failed to advance attestation key preparation window"), + /// Warms every validator's signing cache for `slot`. + /// + /// Pure latency shifting: the key rebuilds the same bottom Merkle subtree + /// inside `sign` on a miss, so this only moves that cost off the duty's + /// critical path. Called one slot ahead, so a miss here is not yet a + /// failure to sign. + pub fn prepare_keys_for(&self, slot: u32) { + for (validator_id, key_pair) in &self.keys { + let _ = prepare_key(&key_pair.attestation_key, slot).inspect_err( + |err| warn!(validator_id, slot, %err, "Failed to warm attestation key signing cache"), ); - let _ = advance_key(*validator_id, &mut key_pair.proposal_key, slot).inspect_err( - |err| warn!(validator_id, slot, %err, "Failed to advance proposal key preparation window"), + let _ = prepare_key(&key_pair.proposal_key, slot).inspect_err( + |err| warn!(validator_id, slot, %err, "Failed to warm proposal key signing cache"), ); } } @@ -93,10 +97,10 @@ impl KeyManager { .get_mut(&validator_id) .ok_or(KeyManagerError::ValidatorKeyNotFound(validator_id))?; - // Advance XMSS key preparation window if the slot is outside the current window. - // Each bottom tree covers 65,536 slots; the window holds 2 at a time. - // Multiple advances may be needed if the node was offline for an extended period. - advance_key(validator_id, &mut key_pair.attestation_key, slot)?; + // A slot outside the key's range can never be signed, however long the + // node waits, so name that rather than letting it surface as a generic + // signing error. + signable_at(validator_id, &key_pair.attestation_key, slot)?; let signature: ValidatorSignature = { let _timing = metrics::time_pq_sig_attestation_signing(); @@ -123,10 +127,7 @@ impl KeyManager { .get_mut(&validator_id) .ok_or(KeyManagerError::ValidatorKeyNotFound(validator_id))?; - // Advance XMSS key preparation window if the slot is outside the current window. - // Each bottom tree covers 65,536 slots; the window holds 2 at a time. - // Multiple advances may be needed if the node was offline for an extended period. - advance_key(validator_id, &mut key_pair.proposal_key, slot)?; + signable_at(validator_id, &key_pair.proposal_key, slot)?; let signature: ValidatorSignature = key_pair .proposal_key @@ -139,32 +140,33 @@ impl KeyManager { } } -fn advance_key( +/// Reject a slot the key cannot sign at. +/// +/// The signable range is fixed at key generation, so this is exhaustion, not a +/// window that will catch up. +fn signable_at( validator_id: u64, - key: &mut ValidatorSecretKey, + key: &ValidatorSecretKey, slot: u32, ) -> Result<(), KeyManagerError> { - if key.is_prepared_for(slot) { + if key.can_sign_at(slot) { return Ok(()); } - info!(validator_id, slot, "Advancing XMSS key preparation window"); + let range = key.signable_slots(); + Err(KeyManagerError::SigningError(format!( + "XMSS key exhausted for validator {validator_id}: slot {slot} is outside \ + the key's signable range [{}, {}]", + range.start(), + range.end() + ))) +} + +/// Warm one key's signing cache, timing the miss that rebuilds a subtree. +fn prepare_key(key: &ValidatorSecretKey, slot: u32) -> Result<(), KeyManagerError> { let start = Instant::now(); - while !key.is_prepared_for(slot) { - let before = key.get_prepared_interval(); - key.advance_preparation(); - if key.get_prepared_interval() == before { - return Err(KeyManagerError::SigningError(format!( - "XMSS key exhausted for validator {validator_id}: \ - slot {slot} is beyond the key's activation interval" - ))); - } - } - info!( - validator_id, - slot, - elapsed = ?start.elapsed(), - "Advanced XMSS key preparation window" - ); + key.prepare(slot) + .map_err(|err| KeyManagerError::SigningError(err.to_string()))?; + trace!(slot, elapsed = ?start.elapsed(), "Warmed XMSS signing cache"); Ok(()) } diff --git a/crates/blockchain/src/lib.rs b/crates/blockchain/src/lib.rs index 8d678e22..41986692 100644 --- a/crates/blockchain/src/lib.rs +++ b/crates/blockchain/src/lib.rs @@ -1,6 +1,7 @@ use std::collections::{HashMap, HashSet, VecDeque}; use std::time::{Duration, Instant, SystemTime}; +use ethlambda_crypto::SignerSet; use ethlambda_crypto::signature::{ValidatorPublicKey, ValidatorSignature}; use ethlambda_network_api::{BlockChainToP2PRef, BlockSource, InitP2P}; use ethlambda_state_transition::is_proposer; @@ -174,15 +175,15 @@ impl BlockChain { metrics::set_node_sync_status(metrics::SyncStatus::Idle); let time_config = *store.config(); let genesis_time = time_config.genesis_time; - let mut key_manager = key_manager::KeyManager::new(validator_keys); + let key_manager = key_manager::KeyManager::new(validator_keys); - // Catch XMSS keys up to the current slot before the first tick + // Warm the XMSS signing caches for the current slot before the first tick. // store.time() doesn't work here: after an offline gap it lags wall-clock by - // exactly the gap we need to catch up through + // exactly the gap the first duty will be at let now_ms = unix_now_ms(); let current_slot = (now_ms.saturating_sub(time_config.genesis_time_ms()) / time_config.milliseconds_per_slot) as u32; - key_manager.advance_keys_to(current_slot); + key_manager.prepare_keys_for(current_slot); let handle = BlockChainServer { store, @@ -475,8 +476,8 @@ impl BlockChainServer { // Update head slot metric (head may change when attestations are promoted at intervals 0/4) metrics::update_head_slot(self.store.head_slot()); - // Advance XMSS keys for next slot so the signing paths don't have to - self.key_manager.advance_keys_to((slot + 1) as u32); + // Warm the XMSS signing caches for the next slot so the signing paths don't have to + self.key_manager.prepare_keys_for((slot + 1) as u32); } /// Kick off a committee-signature aggregation session: @@ -836,10 +837,33 @@ impl BlockChainServer { return; }; - let mut merge_inputs: Vec<(Vec, ByteList512KiB)> = + // Each merge input pairs the proof bytes with the claim its Type-1 + // binds: the message, the slot, and the participants' keys. + // `single_message_aggregates` is ordered to match + // `block.body.attestations`, which is how the claim is recovered for + // each proof. Checked rather than assumed: zipping two + // lists of different lengths would build a proof covering fewer claims + // than the body declares, which only surfaces at import. + if single_message_aggregates.len() != block.body.attestations.len() { + error!( + %slot, %validator_id, + aggregates = single_message_aggregates.len(), + attestations = block.body.attestations.len(), + "Proof list does not line up with the block body" + ); + metrics::inc_block_building_failures(); + return; + } + + let mut merge_inputs: Vec<(SignerSet, ByteList512KiB)> = Vec::with_capacity(single_message_aggregates.len() + 1); let mut resolve_failed = false; - for sma in &single_message_aggregates { + for (attestation, sma) in block + .body + .attestations + .iter() + .zip(&single_message_aggregates) + { let mut pubkeys = Vec::new(); for vid in sma.participant_indices() { let Some(validator) = validators.get(vid as usize) else { @@ -859,18 +883,27 @@ impl BlockChainServer { if resolve_failed { break; } - merge_inputs.push((pubkeys, sma.proof.clone())); + let Ok(attestation_slot) = u32::try_from(attestation.data.slot) else { + error!(%slot, %validator_id, attestation_slot = attestation.data.slot, + "Attestation slot out of range while assembling block"); + resolve_failed = true; + break; + }; + let claim = + SignerSet::new(attestation.data.hash_tree_root(), attestation_slot, pubkeys); + merge_inputs.push((claim, sma.proof.clone())); } if resolve_failed { metrics::inc_block_building_failures(); return; } - merge_inputs.push((vec![proposer_pubkey], proposer_proof_bytes)); + let proposer_claim = SignerSet::new(block_root, slot as u32, vec![proposer_pubkey]); + merge_inputs.push((proposer_claim, proposer_proof_bytes)); - // Merge yields raw lean-multisig type-2 bytes. Per-component - // participants are rederived at verify time from - // `block.body.attestations[i].aggregation_bits` plus - // `block.proposer_index`, so nothing else needs persisting. + // Merge yields raw leanVM aggregate bytes. Per-component participants + // and bindings are rederived at verify time from + // `block.body.attestations[i]` plus `block.proposer_index`, so nothing + // else needs persisting. let merged_bytes = match ethlambda_crypto::merge_type_1s_into_type_2(merge_inputs) { Ok(bytes) => bytes, Err(err) => { diff --git a/crates/blockchain/src/reaggregate.rs b/crates/blockchain/src/reaggregate.rs index 13b433c5..c97d2cb7 100644 --- a/crates/blockchain/src/reaggregate.rs +++ b/crates/blockchain/src/reaggregate.rs @@ -24,6 +24,7 @@ use std::collections::HashSet; +use ethlambda_crypto::SignerSet; use ethlambda_crypto::signature::ValidatorPublicKey; use ethlambda_storage::Store; use ethlambda_types::{ @@ -71,11 +72,11 @@ pub fn reaggregate_from_block( let validators = &parent_state.validators; let num_validators = validators.len() as u64; - // Per-component pubkeys: one entry per body attestation in order, then - // the proposer entry. Layout is invariant per block, so it's resolved - // once and reused for every split call below. - let mut pubkeys_per_component: Vec> = - Vec::with_capacity(attestations.len() + 1); + // The claims the merged proof carries: one per body attestation in order, + // then the proposer's. The layout is invariant per block, so it is resolved + // once and reused for every split call below. Every split needs all of + // them, since none is on the wire and the decode rebuilds the whole set. + let mut components: Vec = Vec::with_capacity(attestations.len() + 1); for att in &attestations { let mut pubkeys = Vec::new(); for vid in validator_indices(&att.aggregation_bits) { @@ -91,7 +92,14 @@ pub fn reaggregate_from_block( }; pubkeys.push(pk); } - pubkeys_per_component.push(pubkeys); + let Ok(att_slot) = u32::try_from(att.data.slot) else { + warn!( + slot = att.data.slot, + "Reaggregation aborted: attestation slot out of range" + ); + return Vec::new(); + }; + components.push(SignerSet::new(att.data.hash_tree_root(), att_slot, pubkeys)); } if block.proposer_index >= num_validators { return Vec::new(); @@ -101,7 +109,18 @@ pub fn reaggregate_from_block( else { return Vec::new(); }; - pubkeys_per_component.push(vec![proposer_pubkey]); + let Ok(block_slot) = u32::try_from(block.slot) else { + warn!( + slot = block.slot, + "Reaggregation aborted: block slot out of range" + ); + return Vec::new(); + }; + components.push(SignerSet::new( + block.hash_tree_root(), + block_slot, + vec![proposer_pubkey], + )); let candidates = select_candidates(store, &attestations); if candidates.is_empty() { @@ -117,17 +136,15 @@ pub fn reaggregate_from_block( for candidate in candidates { let att = &attestations[candidate.idx]; let data_root = candidate.data_root; - let slot_u32: u32 = match att.data.slot.try_into() { - Ok(s) => s, - Err(_) => continue, - }; + // Already range-checked while the claims were resolved above. + let slot_u32 = components[candidate.idx].slot; // Step 1: SNARK-split this attestation's component out of the block's // merged multi-message aggregate proof. let merged_bytes = signed_block.proof.proof_bytes(); let split_bytes = match ethlambda_crypto::split_type_2_by_message( merged_bytes, - pubkeys_per_component.clone(), + &components, &data_root, ) { Ok(bytes) => bytes, @@ -152,7 +169,7 @@ pub fn reaggregate_from_block( // First child: the split-from-block proof, paired with the // pubkeys derived from the block attestation's participant set. - let block_att_pubkeys = pubkeys_per_component[candidate.idx].clone(); + let block_att_pubkeys = components[candidate.idx].public_keys.clone(); children.push((block_att_pubkeys, split_bytes)); // Remaining children: local partial single-message aggregates for the same data. diff --git a/crates/blockchain/src/store.rs b/crates/blockchain/src/store.rs index 010d8ff6..f4eaae03 100644 --- a/crates/blockchain/src/store.rs +++ b/crates/blockchain/src/store.rs @@ -1,5 +1,6 @@ use std::collections::{HashMap, HashSet}; +use ethlambda_crypto::SignerSet; use ethlambda_crypto::signature::{ValidatorPublicKey, ValidatorSignature}; use ethlambda_state_transition::{is_proposer, slot_is_justifiable_after}; use ethlambda_storage::{ForkCheckpoints, Store}; @@ -1114,11 +1115,13 @@ pub enum StoreError { /// Full verification of a signed block's merged multi-message aggregate proof. /// -/// Structural pre-checks (fast fail) ensure the merged proof's `info` list lines -/// up with the block body (one entry per attestation plus a trailing proposer -/// entry; messages, slots, and participants match what the body declares). -/// On success, the lean-multisig devnet5 `verify_type_2` primitive runs the -/// SNARK verifier over the merged proof bytes against the resolved pubkey set. +/// Structural pre-checks (fast fail) bound the body itself: attestation count, +/// no duplicate `AttestationData`, and every participant and the proposer in +/// range of the validator registry. The claims the proof must carry are then +/// rederived from the body, one `(message, slot, pubkeys)` per attestation plus +/// a trailing proposer entry, and handed to leanVM's verifier. None of them is +/// on the wire, so a proof over other messages, slots or keys fails inside the +/// SNARK rather than at a compare ahead of it. /// /// Exposed publicly so RPC handlers (notably the Hive test-driver /// `verify_signatures/run` endpoint) can run the exact same verification path @@ -1160,14 +1163,13 @@ pub fn verify_block_signatures( let block_root = block.hash_tree_root(); let structural_elapsed = total_start.elapsed(); - // Resolve pubkeys per multi-message aggregate component for verify_type_2 and rederive the - // expected (message, slot) bindings from the block body. Attestation - // components use each participant's attestation_pubkey; the trailing - // proposer component uses the proposal_pubkey of `block.proposer_index`. + // Rederive the claims the merged proof must carry from the block body: one + // `(message, slot, pubkeys)` per attestation, then the proposer's. Nothing + // of this is on the wire, so it is the caller's view that binds the proof; + // attestation claims use each participant's attestation_pubkey, the + // trailing proposer claim the proposal_pubkey of `block.proposer_index`. let expected_components = attestations.len() + 1; - let mut pubkeys_per_component: Vec> = - Vec::with_capacity(expected_components); - let mut expected_bindings: Vec<(H256, u32)> = Vec::with_capacity(expected_components); + let mut components: Vec = Vec::with_capacity(expected_components); for attestation in attestations.iter() { let mut pubkeys = Vec::new(); @@ -1181,10 +1183,13 @@ pub fn verify_block_signatures( .map_err(|_| StoreError::PubkeyDecodingFailed(vid))?; pubkeys.push(pk); } - pubkeys_per_component.push(pubkeys); let slot_u32 = u32::try_from(attestation.data.slot) .map_err(|_| StoreError::SlotOutOfRange(attestation.data.slot))?; - expected_bindings.push((attestation.data.hash_tree_root(), slot_u32)); + components.push(SignerSet::new( + attestation.data.hash_tree_root(), + slot_u32, + pubkeys, + )); } let proposer_out_of_range = StoreError::ProposerIndexOutOfRange { @@ -1196,20 +1201,19 @@ pub fn verify_block_signatures( .ok_or(proposer_out_of_range)?; let proposer_pubkey = ValidatorPublicKey::from_bytes(&proposer_validator.proposal_pubkey) .map_err(|_| StoreError::PubkeyDecodingFailed(block.proposer_index))?; - pubkeys_per_component.push(vec![proposer_pubkey]); let block_slot_u32 = u32::try_from(block.slot).map_err(|_| StoreError::SlotOutOfRange(block.slot))?; - expected_bindings.push((block_root, block_slot_u32)); + components.push(SignerSet::new( + block_root, + block_slot_u32, + vec![proposer_pubkey], + )); let merged_bytes = signed_block.proof.proof_bytes(); let crypto_start = std::time::Instant::now(); - ethlambda_crypto::verify_type_2_signature( - merged_bytes, - pubkeys_per_component, - &expected_bindings, - ) - .map_err(StoreError::BlockProofVerificationFailed)?; + ethlambda_crypto::verify_type_2_signature(merged_bytes, &components) + .map_err(StoreError::BlockProofVerificationFailed)?; let crypto_elapsed = crypto_start.elapsed(); let total_elapsed = total_start.elapsed(); @@ -1291,9 +1295,9 @@ mod tests { /// Test helper: placeholder block proof bytes. /// - /// In production the merged proof is the raw `compress_without_pubkeys()` - /// output of `merge_many_type_1`, which can only be built by the - /// lean-multisig prover. Tests that don't go through + /// In production the merged proof is the raw `to_bytes_without_pubkeys()` + /// output of a recursive aggregation over every Type-1, which can only be + /// built by the leanVM prover. Tests that don't go through /// `verify_block_signatures` use an empty blob. fn make_signed_block_proof( _proposer_index: u64, @@ -1869,8 +1873,8 @@ mod tests { fn make_validators(count: u64) -> Vec { (0..count) .map(|index| ethlambda_types::state::Validator { - attestation_pubkey: [0u8; 52], - proposal_pubkey: [0u8; 52], + attestation_pubkey: ethlambda_types::state::ValidatorPubkeyBytes::default(), + proposal_pubkey: ethlambda_types::state::ValidatorPubkeyBytes::default(), index, }) .collect() diff --git a/crates/blockchain/state_transition/src/lib.rs b/crates/blockchain/state_transition/src/lib.rs index d03f74b2..f7e9e50c 100644 --- a/crates/blockchain/state_transition/src/lib.rs +++ b/crates/blockchain/state_transition/src/lib.rs @@ -658,8 +658,8 @@ mod tests { fn make_validators(n: usize) -> Vec { (0..n) .map(|i| Validator { - attestation_pubkey: [i as u8; 52], - proposal_pubkey: [i as u8; 52], + attestation_pubkey: [i as u8; 32], + proposal_pubkey: [i as u8; 32], index: i as u64, }) .collect() diff --git a/crates/blockchain/tests/signature_spectests.rs b/crates/blockchain/tests/signature_spectests.rs index fa727d1e..64b6f7be 100644 --- a/crates/blockchain/tests/signature_spectests.rs +++ b/crates/blockchain/tests/signature_spectests.rs @@ -18,11 +18,15 @@ const SUPPORTED_FIXTURE_FORMAT: &str = "verify_signatures_test"; /// Tests that require cryptographic signature verification at block level. /// -/// Block-level crypto verification is now wired through lean-multisig devnet5's +/// Block-level crypto verification is now wired through leanVM's /// `verify_type_2`, so every fixture is exercised against the real primitive. const SKIP_TESTS: &[&str] = &[]; fn run(path: &Path) -> datatest_stable::Result<()> { + // These fixtures verify real signatures, so they need the backend a binary would set + // up at startup. Idempotent, so calling it per fixture costs nothing after the first. + ethlambda_crypto::init_leanvm(false); + let tests = VerifySignaturesTestVector::from_file(path)?; for (name, test) in tests.tests { diff --git a/crates/common/crypto/Cargo.toml b/crates/common/crypto/Cargo.toml index 5ec86eca..5fcf5a40 100644 --- a/crates/common/crypto/Cargo.toml +++ b/crates/common/crypto/Cargo.toml @@ -12,14 +12,11 @@ version.workspace = true [dependencies] ethlambda-types.workspace = true +leanvm.workspace = true +postcard.workspace = true -lean-multisig = { git = "https://github.com/leanEthereum/leanVM.git", rev = "e2592df" } -# leansig_wrapper provides XmssPublicKey/XmssSignature types used by lean-multisig's public API -leansig_wrapper = { git = "https://github.com/leanEthereum/leanVM.git", rev = "e2592df" } - -leansig.workspace = true thiserror.workspace = true -rand.workspace = true +tracing.workspace = true [features] shadow-integration = [] diff --git a/crates/common/crypto/src/lib.rs b/crates/common/crypto/src/lib.rs index 51d7d282..384fe718 100644 --- a/crates/common/crypto/src/lib.rs +++ b/crates/common/crypto/src/lib.rs @@ -1,37 +1,132 @@ -use std::sync::Once; +//! XMSS signature aggregation and verification, wrapping leanVM. +//! +//! [`init_leanvm`] must be called once at startup, before anything else here: it compiles +//! the aggregation bytecode and fixes the prover's allocator. Proving panics without it, +//! and decoding a stored proof misreports as a corrupt proof. Binaries call it straight +//! after argument parsing; tests that touch a proof call it themselves. +//! +//! Everything else assumes it has run. Aggregation produces Type-1 proofs (one message, +//! one slot) and Type-2 proofs (several messages merged); both travel without their +//! participant pubkeys, which a receiver rebuilds from its own validator registry. +//! +//! # One aggregate type, grouped by slot +//! +//! leanVM has a single `AggregateSignature`, whose XMSS claims are grouped by +//! epoch: one [`XmssGroup`] per slot, carrying the one message signed at it and +//! the group's strictly sorted, deduplicated keys. Type-1 and Type-2 are the +//! same object here, one group versus several, and the wrappers below only +//! differ in how many groups they build. +//! +//! Two consequences run through everything in this module: +//! +//! - **A slot carries one message.** Several distinct `AttestationData` at one +//! slot have no representation inside a single aggregate; that is +//! [`ConflictingMessages`]. +//! - **The binding travels out of band.** The without-pubkeys wire form carries +//! neither the keys nor the `(slot, message)` pairs, so a decode has to +//! rebuild the whole signer set from the caller's own view of it (a +//! [`SignerSet`] per claim). A set or binding other than the one aggregated +//! decodes fine and fails verification, since the proof binds a hash of the +//! structure: the `(message, slot)` check is inside the SNARK now, not a +//! cheap field compare ahead of it. use ethlambda_types::{block::ByteList512KiB, primitives::H256}; use crate::signature::{ValidatorPublicKey, ValidatorSignature}; -use lean_multisig::{ - MultiMessageAggregateSignature as LMType2, ProofError, - SingleMessageAggregateSignature as LMType1, aggregate_single_message_signatures, - merge_single_message_aggregates, setup_prover, setup_verifier, split_multi_message_aggregate, - verify_multi_message_aggregate, verify_single_message_aggregate, -}; -use leansig_wrapper::{XmssPublicKey as LeanSigPubKey, XmssSignature as LeanSigSignature}; +use leanvm::{AggregateSignature, WireKeys, XmssGroup, aggregate, xmss}; +use std::sync::{Mutex, MutexGuard}; use thiserror::Error; +use tracing::error; pub mod signature; #[cfg(feature = "shadow-integration")] pub mod shadow_cost; -/// log(1/rate) for the WHIR commitment scheme used inside lean-multisig. +/// log(1/rate) for the WHIR commitment scheme used inside the aggregation prover. const LOG_INV_RATE: usize = 2; -// Lazy initialization for prover and verifier setup -static PROVER_INIT: Once = Once::new(); -static VERIFIER_INIT: Once = Once::new(); +/// Raw XMSS input as [`aggregate`] takes it: the key, the epoch it signed at, +/// the message, and the signature. +type RawXmss = Vec<( + xmss::XmssPublicKey, + xmss::Epoch, + xmss::Message, + xmss::XmssSignature, +)>; -/// Ensure the prover is initialized. Safe to call multiple times. -pub fn ensure_prover_ready() { - PROVER_INIT.call_once(setup_prover); +/// Initializes the leanVM backend. Call once at startup, before any other function here. +/// +/// Everything downstream assumes this has run: proving panics without the aggregation +/// bytecode, and decoding a stored proof needs it too, since a decode rebuilds the +/// bytecode claim and fails without it, which surfaces as a bogus +/// `DeserializationFailed`. Doing it up front keeps the cost off the first duty. +/// +/// `use_arena` picks the prover's allocator. leanVM's arena recycles the prover's large +/// transient buffers across proofs instead of re-faulting them, so its pages stay +/// resident for the lifetime of the node; the system allocator trades throughput for +/// memory that comes back. +/// +/// Idempotent: every step is `Once`/`OnceLock` guarded. +pub fn init_leanvm(use_arena: bool) { + if use_arena { + leanvm::setup_prover(); + } else { + leanvm::setup_prover_without_arena(); + } +} + +/// Claims the exclusive right to prove. +/// +/// leanVM allows one proof at a time per process; a second concurrent one panics. +/// Proving is legal only while the returned guard is alive, so take it immediately +/// before the prove call: decoding and argument conversion need no permit. +/// +/// The permit guards no data, so a poisoned lock is recovered rather than propagated: +/// one panicking prover must not brick every later proof. It is still an incident. +fn acquire_prover() -> MutexGuard<'static, ()> { + static PROVER_PERMIT: Mutex<()> = Mutex::new(()); + PROVER_PERMIT.lock().unwrap_or_else(|poisoned| { + error!("a previous proving job panicked while holding the permit; continuing"); + poisoned.into_inner() + }) +} + +/// One claim inside an aggregate: the `(message, slot)` a set of validators +/// signed, and the keys of those validators. +/// +/// Both halves are needed to decode a stored proof, and neither is on the wire: +/// see the module docs. +#[derive(Clone, Debug)] +pub struct SignerSet { + /// The signed message: an `AttestationData` root, or a block root. + pub message: H256, + /// The slot the signature was made at, which is the XMSS epoch. + pub slot: u32, + /// The validators the claim binds, in any order and with duplicates allowed; + /// [`wire_keys`] sorts and deduplicates as leanVM's signer set requires. + pub public_keys: Vec, +} + +impl SignerSet { + pub fn new(message: H256, slot: u32, public_keys: Vec) -> Self { + Self { + message, + slot, + public_keys, + } + } } -/// Ensure the verifier is initialized. Safe to call multiple times. -pub fn ensure_verifier_ready() { - VERIFIER_INIT.call_once(setup_verifier); +/// Two claims at one slot carrying different messages. +/// +/// leanVM keys an aggregate's XMSS groups by epoch, so inside one proof the +/// message is a function of the slot. Nothing in this crate can work around it: +/// the group holds a single message, so the second claim has nowhere to go. +#[derive(Debug, Clone, Copy, Error)] +#[error("slot {slot} carries two different messages in one aggregate")] +pub struct ConflictingMessages { + pub slot: u32, } /// Error type for signature aggregation operations. @@ -55,16 +150,13 @@ pub enum AggregationError { #[error("need at least 2 children for recursive aggregation, got {0}")] InsufficientChildren(usize), - #[error("component count ({components}) does not match pubkey-set count ({pubkey_sets})")] - ComponentPubkeyMismatch { - components: usize, - pubkey_sets: usize, - }, + #[error(transparent)] + ConflictingMessages(#[from] ConflictingMessages), - #[error("split-by-message target not found in type-2 components")] + #[error("split-by-message target not found in the aggregate's signer set")] UnknownMessage, - #[error("split-by-message target matched multiple components")] + #[error("split-by-message target matched several slots")] MultipleMessages, #[error("prover failure: {0}")] @@ -74,68 +166,115 @@ pub enum AggregationError { /// Error type for signature verification operations. #[derive(Debug, Error)] pub enum VerificationError { - #[error("public key conversion failed at index {index}: {reason}")] - PublicKeyConversion { index: usize, reason: String }, - #[error("proof deserialization failed")] DeserializationFailed, #[error("verification failed: {0}")] - ProofError(#[from] ProofError), - - #[error( - "(message, slot) mismatch: proof binds {got_slot}/{got_msg:?}, expected {expected_slot}/{expected_msg:?}" - )] - BindingMismatch { - expected_msg: H256, - expected_slot: u32, - got_msg: H256, - got_slot: u32, - }, - - #[error("component count ({components}) does not match pubkey-set count ({pubkey_sets})")] - ComponentPubkeyMismatch { - components: usize, - pubkey_sets: usize, - }, - - #[error("type-2 binds {got} components but {expected} were expected")] - Type2ComponentCountMismatch { expected: usize, got: usize }, + VerificationFailed(String), + + #[error(transparent)] + ConflictingMessages(#[from] ConflictingMessages), } // ===================================================================== // Helpers // ===================================================================== -fn into_lean_pubkeys(pubkeys: Vec) -> Vec { - pubkeys +/// The signer set leanVM binds, built from the caller's view of the claims. +/// +/// One group per slot, holding that slot's message and its keys strictly sorted +/// and deduplicated, with the groups themselves sorted by slot. Claims sharing a +/// slot merge into one group, so their keys are unioned; claims sharing a slot +/// under different messages are [`ConflictingMessages`]. +/// +/// Getting this structure wrong is not caught at decode: it changes the digest +/// the proof is checked against, so it surfaces as a verification failure. +fn wire_keys(components: &[SignerSet]) -> Result { + let mut groups: Vec = Vec::with_capacity(components.len()); + for component in components { + let keys = component.public_keys.iter().map(|pk| pk.as_inner().clone()); + match groups.iter_mut().find(|(slot, ..)| *slot == component.slot) { + Some((_, message, group_keys)) => { + if *message != component.message.0 { + return Err(ConflictingMessages { + slot: component.slot, + }); + } + group_keys.extend(keys); + } + None => groups.push((component.slot, component.message.0, keys.collect())), + } + } + for (_, _, keys) in &mut groups { + sort_dedup(keys); + } + groups.sort_unstable_by_key(|(slot, ..)| *slot); + Ok((groups, Vec::new())) +} + +/// [`wire_keys`] for a single claim, which cannot conflict with itself. +fn one_group(message: &H256, slot: u32, public_keys: &[ValidatorPublicKey]) -> WireKeys { + let mut keys: Vec<_> = public_keys.iter().map(|pk| pk.as_inner().clone()).collect(); + sort_dedup(&mut keys); + (vec![(slot, message.0, keys)], Vec::new()) +} + +/// A group's keys as leanVM's signer set requires them: strictly sorted, so the +/// list's length is a count of distinct claims and no key is covered twice. +/// +/// The aggregator does the same to its raw inputs, so a verifier that skips this +/// hands over a different digest and fails a proof that is in fact valid. +fn sort_dedup(keys: &mut Vec) { + keys.sort_unstable(); + keys.dedup(); +} + +/// Pair raw XMSS keys with their signatures for [`aggregate`], all at one +/// `(message, slot)`. +fn raw_xmss_inputs( + public_keys: Vec, + signatures: Vec, + message: &H256, + slot: u32, +) -> RawXmss { + public_keys .into_iter() - .map(ValidatorPublicKey::into_inner) + .zip(signatures) + .map(|(pk, sig)| (pk.into_inner(), slot, message.0, sig.into_inner())) .collect() } -/// Decompress a stored Type-1 proof (without-pubkeys form) into a native -/// `SingleMessageAggregateSignature` by attaching the resolved validator pubkeys. -fn decompress_type1( - pubkeys: Vec, - proof_bytes: &ByteList512KiB, - index: usize, -) -> Result { - let lean_pks = into_lean_pubkeys(pubkeys); - LMType1::decompress_without_pubkeys(proof_bytes.iter().as_slice(), lean_pks) - .ok_or(AggregationError::ChildDeserializationFailed(index)) +/// Decompress the stored Type-1 children of a recursive aggregation, all of +/// which share one `(message, slot)`. +fn decompress_children( + children: Vec<(Vec, ByteList512KiB)>, + message: &H256, + slot: u32, +) -> Result, AggregationError> { + children + .into_iter() + .enumerate() + .map(|(index, (pubkeys, proof_bytes))| { + let keys = one_group(message, slot, &pubkeys); + AggregateSignature::from_bytes_without_pubkeys(proof_bytes.iter().as_slice(), keys) + .map_err(|_| AggregationError::ChildDeserializationFailed(index)) + }) + .collect() } -fn compress_type1_to_byte_list(sig: &LMType1) -> Result { - let serialized = sig.compress_without_pubkeys(); +fn compress_to_byte_list(sig: &AggregateSignature) -> Result { + let serialized = sig.to_bytes_without_pubkeys(); let len = serialized.len(); ByteList512KiB::try_from(serialized).map_err(|_| AggregationError::ProofTooBig(len)) } -fn compress_type2_to_byte_list(sig: &LMType2) -> Result { - let serialized = sig.compress_without_pubkeys(); - let len = serialized.len(); - ByteList512KiB::try_from(serialized).map_err(|_| AggregationError::ProofTooBig(len)) +/// leanVM's aggregation errors, kept as their own text. +/// +/// They cover both proving failures and malformed requests (a slot carrying two +/// messages, a child that does not verify, too many children); the message says +/// which, and no caller here branches on the distinction. +fn aggregation_failed(err: leanvm::AggregationError) -> AggregationError { + AggregationError::ProverFailure(err.to_string()) } // ===================================================================== @@ -144,12 +283,13 @@ fn compress_type2_to_byte_list(sig: &LMType2) -> Result, signatures: Vec, @@ -179,19 +319,13 @@ pub fn aggregate_signatures( return Ok(dummy); } - ensure_prover_ready(); + let raw_xmss = raw_xmss_inputs(public_keys, signatures, message, slot); - let raw_xmss: Vec<(LeanSigPubKey, LeanSigSignature)> = public_keys - .into_iter() - .zip(signatures) - .map(|(pk, sig)| (pk.into_inner(), sig.into_inner())) - .collect(); + let _permit = acquire_prover(); - let proof = aggregate_single_message_signatures(&[], raw_xmss, message.0, slot, LOG_INV_RATE) - .map_err(|err| AggregationError::ProverFailure(err.to_string()))?; + let proof = aggregate(&[], raw_xmss, vec![], None, LOG_INV_RATE).map_err(aggregation_failed)?; - let result = compress_type1_to_byte_list(&proof)?; - Ok(result) + compress_to_byte_list(&proof) } /// Aggregate both existing Type-1 proofs (children) and raw XMSS signatures. @@ -234,31 +368,15 @@ pub fn aggregate_mixed( return Ok(dummy); } - ensure_prover_ready(); + let children_native = decompress_children(children, message, slot)?; + let raw_xmss = raw_xmss_inputs(raw_public_keys, raw_signatures, message, slot); - let children_native: Vec = children - .into_iter() - .enumerate() - .map(|(i, (pubkeys, proof_bytes))| decompress_type1(pubkeys, &proof_bytes, i)) - .collect::>()?; + let _permit = acquire_prover(); - let raw_xmss: Vec<(LeanSigPubKey, LeanSigSignature)> = raw_public_keys - .into_iter() - .zip(raw_signatures) - .map(|(pk, sig)| (pk.into_inner(), sig.into_inner())) - .collect(); - - let proof = aggregate_single_message_signatures( - &children_native, - raw_xmss, - message.0, - slot, - LOG_INV_RATE, - ) - .map_err(|err| AggregationError::ProverFailure(err.to_string()))?; - - let result = compress_type1_to_byte_list(&proof)?; - Ok(result) + let proof = aggregate(&children_native, raw_xmss, vec![], None, LOG_INV_RATE) + .map_err(aggregation_failed)?; + + compress_to_byte_list(&proof) } /// Recursively aggregate two or more already-aggregated Type-1 proofs into one. @@ -288,33 +406,24 @@ pub fn aggregate_proofs( return Ok(dummy); } - ensure_prover_ready(); + let children_native = decompress_children(children, message, slot)?; - let children_native: Vec = children - .into_iter() - .enumerate() - .map(|(i, (pubkeys, proof_bytes))| decompress_type1(pubkeys, &proof_bytes, i)) - .collect::>()?; + let _permit = acquire_prover(); - let proof = aggregate_single_message_signatures( - &children_native, - vec![], - message.0, - slot, - LOG_INV_RATE, - ) - .map_err(|err| AggregationError::ProverFailure(err.to_string()))?; - - let result = compress_type1_to_byte_list(&proof)?; - Ok(result) + let proof = aggregate(&children_native, vec![], vec![], None, LOG_INV_RATE) + .map_err(aggregation_failed)?; + + compress_to_byte_list(&proof) } /// Verify a Type-1 aggregated signature proof. /// /// Cryptographically verifies that every `public_key` signed `message` at `slot`. /// -/// The verifier checks the bound `(message, slot)` matches what the caller -/// expects, defending against proofs reused from other binding contexts. +/// The binding is checked by being supplied: `(message, slot)` and the key set +/// go into the signer set the proof's digest commits to, so a proof reused from +/// another binding context fails inside the SNARK verifier rather than at a +/// field compare ahead of it. pub fn verify_aggregated_signature( proof_data: &ByteList512KiB, public_keys: Vec, @@ -329,23 +438,13 @@ pub fn verify_aggregated_signature( crate::shadow_cost::sleep(crate::shadow_cost::verify_delay(verify_n)); return Ok(()); } - ensure_verifier_ready(); - - let lean_pubkeys = into_lean_pubkeys(public_keys); - let sig = LMType1::decompress_without_pubkeys(proof_data.iter().as_slice(), lean_pubkeys) - .ok_or(VerificationError::DeserializationFailed)?; - - if sig.info.without_pubkeys.message != message.0 || sig.info.without_pubkeys.slot != slot { - return Err(VerificationError::BindingMismatch { - expected_msg: *message, - expected_slot: slot, - got_msg: H256(sig.info.without_pubkeys.message), - got_slot: sig.info.without_pubkeys.slot, - }); - } - verify_single_message_aggregate(&sig)?; - Ok(()) + let keys = one_group(message, slot, &public_keys); + let sig = AggregateSignature::from_bytes_without_pubkeys(proof_data.iter().as_slice(), keys) + .map_err(|_| VerificationError::DeserializationFailed)?; + + sig.verify() + .map_err(|err| VerificationError::VerificationFailed(format!("{err:?}"))) } // ===================================================================== @@ -354,14 +453,17 @@ pub fn verify_aggregated_signature( /// Merge many independent Type-1 multi-signatures into a single Type-2 proof. /// -/// Each input is `(participant_pubkeys, type_1_proof_bytes)` where the bytes -/// are the `compress_without_pubkeys()` form of a `SingleMessageAggregateSignature`. +/// Each input is `(claim, type_1_proof_bytes)` where the bytes are the +/// `to_bytes_without_pubkeys()` form of an aggregate over exactly that claim. +/// +/// The returned blob is the `to_bytes_without_pubkeys()` form of the merged +/// aggregate, whose signer set is the union of the claims grouped by slot. A +/// verifier decoding it back needs the same claims, in any order. /// -/// The returned blob is the `compress_without_pubkeys()` form of the resulting -/// `MultiMessageAggregateSignature`. A verifier decoding it back needs the per-component -/// pubkey sets in the same order. +/// Two claims at one slot under different messages cannot be merged at all: +/// leanVM rejects the pair rather than producing a proof (see the module docs). pub fn merge_type_1s_into_type_2( - type_1s: Vec<(Vec, ByteList512KiB)>, + type_1s: Vec<(SignerSet, ByteList512KiB)>, ) -> Result { if type_1s.is_empty() { return Err(AggregationError::EmptyInput); @@ -382,91 +484,60 @@ pub fn merge_type_1s_into_type_2( return Ok(dummy); } - ensure_prover_ready(); - - let type_1s_native: Vec = type_1s - .into_iter() + let type_1s_native: Vec = type_1s + .iter() .enumerate() - .map(|(i, (pubkeys, proof_bytes))| decompress_type1(pubkeys, &proof_bytes, i)) + .map(|(index, (claim, proof_bytes))| { + let keys = one_group(&claim.message, claim.slot, &claim.public_keys); + AggregateSignature::from_bytes_without_pubkeys(proof_bytes.iter().as_slice(), keys) + .map_err(|_| AggregationError::ChildDeserializationFailed(index)) + }) .collect::>()?; - let merged = merge_single_message_aggregates(type_1s_native, LOG_INV_RATE) - .map_err(|err| AggregationError::ProverFailure(err.to_string()))?; + let _permit = acquire_prover(); - let result = compress_type2_to_byte_list(&merged)?; - Ok(result) + let merged = aggregate(&type_1s_native, vec![], vec![], None, LOG_INV_RATE) + .map_err(aggregation_failed)?; + + compress_to_byte_list(&merged) } -/// Verify a Type-2 merged proof against the per-component expected bindings. +/// Verify a Type-2 merged proof against the claims the caller expects it to carry. /// -/// The verifier re-derives each component's `(message, slot, pubkeys)` from the -/// caller-supplied lists, checks they match what the proof binds, and then runs -/// the inner SNARK verifier. +/// The claims are rebuilt from the block body, grouped by slot into the signer +/// set the proof's digest commits to, so a proof over other keys, other +/// messages or other slots fails the SNARK verifier. pub fn verify_type_2_signature( proof_data: &[u8], - pubkeys_per_component: Vec>, - expected_bindings: &[(H256, u32)], + components: &[SignerSet], ) -> Result<(), VerificationError> { - if expected_bindings.len() != pubkeys_per_component.len() { - return Err(VerificationError::ComponentPubkeyMismatch { - components: expected_bindings.len(), - pubkey_sets: pubkeys_per_component.len(), - }); - } - #[cfg(feature = "shadow-integration")] if crate::shadow_cost::fake_xmss() { return Ok(()); } - ensure_verifier_ready(); - - let pubkeys_per_info: Vec> = pubkeys_per_component - .into_iter() - .map(into_lean_pubkeys) - .collect(); - - let sig = LMType2::decompress_without_pubkeys(proof_data, pubkeys_per_info) - .ok_or(VerificationError::DeserializationFailed)?; - - if sig.info.len() != expected_bindings.len() { - return Err(VerificationError::Type2ComponentCountMismatch { - expected: expected_bindings.len(), - got: sig.info.len(), - }); - } - - for (idx, ((expected_msg, expected_slot), info)) in - expected_bindings.iter().zip(sig.info.iter()).enumerate() - { - if info.without_pubkeys.message != expected_msg.0 - || info.without_pubkeys.slot != *expected_slot - { - return Err(VerificationError::BindingMismatch { - expected_msg: *expected_msg, - expected_slot: *expected_slot, - got_msg: H256(info.without_pubkeys.message), - got_slot: info.without_pubkeys.slot, - }); - } - let _ = idx; // index reserved for richer diagnostics if needed - } + let keys = wire_keys(components)?; + let sig = AggregateSignature::from_bytes_without_pubkeys(proof_data, keys) + .map_err(|_| VerificationError::DeserializationFailed)?; - verify_multi_message_aggregate(&sig)?; - Ok(()) + sig.verify() + .map_err(|err| VerificationError::VerificationFailed(format!("{err:?}"))) } -/// Split (disaggregate) a Type-2 merged proof into a single Type-1 proof for -/// the component bound to `message`. Generates a fresh SNARK; expensive. +/// Narrow a Type-2 merged proof down to the single claim bound to `message`, +/// yielding a Type-1 for it. Generates a fresh SNARK; expensive. /// -/// Mirrors leanSpec PR #717 `split_multi_message_aggregate_by_message`: the caller -/// supplies the expected message (an attestation data root or the block -/// root) and the wrapper locates the unique matching component inside the -/// decompressed proof. Returns the `compress_without_pubkeys()` form of the -/// resulting Type-1. +/// Mirrors leanSpec PR #717 `split_multi_message_aggregate_by_message`: the +/// caller supplies the expected message (an attestation data root or the block +/// root) and the wrapper narrows the aggregate to the unique slot carrying it. +/// leanVM does this by re-aggregating the parent with a declaration of what to +/// keep, so the result is a proof over that group alone. +/// +/// `components` gives every claim the parent carries, in any order; they are not +/// on the wire, so decoding needs them all even though only one survives. pub fn split_type_2_by_message( proof_data: &[u8], - pubkeys_per_component: Vec>, + components: &[SignerSet], message: &H256, ) -> Result { #[cfg(feature = "shadow-integration")] @@ -477,88 +548,144 @@ pub fn split_type_2_by_message( )); } - ensure_prover_ready(); - - let pubkeys_per_info: Vec> = pubkeys_per_component - .into_iter() - .map(into_lean_pubkeys) - .collect(); - - let type_2 = LMType2::decompress_without_pubkeys(proof_data, pubkeys_per_info) - .ok_or(AggregationError::DeserializationFailed)?; + let keys = wire_keys(components)?; + let type_2 = AggregateSignature::from_bytes_without_pubkeys(proof_data, keys) + .map_err(|_| AggregationError::DeserializationFailed)?; - let matches: Vec = type_2 - .info + // A slot carries one message, so a message that appears at all appears in + // exactly one group unless two slots signed the very same bytes. + let mut matches = type_2 + .xmss_signers() .iter() - .enumerate() - .filter_map(|(i, info)| (info.without_pubkeys.message == message.0).then_some(i)) - .collect(); - let index = match matches.as_slice() { - [i] => *i, - [] => return Err(AggregationError::UnknownMessage), - _ => return Err(AggregationError::MultipleMessages), + .filter(|(_, group_message, _)| *group_message == message.0); + let group = match (matches.next(), matches.next()) { + (Some(group), None) => group.clone(), + (None, _) => return Err(AggregationError::UnknownMessage), + (Some(_), Some(_)) => return Err(AggregationError::MultipleMessages), }; - let component = split_multi_message_aggregate(type_2, index, LOG_INV_RATE) - .map_err(|err| AggregationError::ProverFailure(err.to_string()))?; + let declare: WireKeys = (vec![group], Vec::new()); + + let _permit = acquire_prover(); + + let component = aggregate(&[type_2], vec![], vec![], Some(&declare), LOG_INV_RATE) + .map_err(aggregation_failed)?; - compress_type1_to_byte_list(&component) + compress_to_byte_list(&component) } #[cfg(test)] mod tests { use super::*; - use crate::signature::LeanSignatureScheme; - use leansig::{serialization::Serializable, signature::SignatureScheme}; - use rand::{SeedableRng, rngs::StdRng}; + use leanvm::xmss::{Encode as _, key_gen_from_seed}; /// Generate a test keypair and sign a message. /// /// Note: This is slow because XMSS key generation is computationally expensive. - /// TODO: move to pre-generated keys fn generate_keypair_and_sign( seed: u64, - activation_epoch: u32, - signing_epoch: u32, + first_slot: u32, + signing_slot: u32, message: &H256, ) -> (ValidatorPublicKey, ValidatorSignature) { - let mut rng = StdRng::seed_from_u64(seed); + let mut seed_bytes = [0u8; 32]; + seed_bytes[..8].copy_from_slice(&seed.to_le_bytes()); - // Use a small lifetime for faster test key generation - let log_lifetime = 5; // 2^5 = 32 epochs - let lifetime = 1 << log_lifetime; + // Small slot range (starting at `first_slot` and covering the signing + // slot) for fast key generation. + let (sk, pk) = + key_gen_from_seed(seed_bytes, first_slot, first_slot + 63).expect("valid slot range"); - let (pk, sk) = LeanSignatureScheme::key_gen(&mut rng, activation_epoch as usize, lifetime); + let sig = + xmss::sign(&mut leanvm::rand::rng(), &sk, &message.0, signing_slot).expect("sign"); - let sig = LeanSignatureScheme::sign(&sk, signing_epoch, &message.0).unwrap(); + // Convert to ethlambda types via SSZ wire bytes. + let validator_pk = ValidatorPublicKey::from_bytes(&pk.as_ssz_bytes()).unwrap(); + let validator_sig = ValidatorSignature::from_bytes(&sig.as_ssz_bytes()).unwrap(); - // Convert to ethlambda types via bytes - let pk_bytes = pk.to_bytes(); - let sig_bytes = sig.to_bytes(); + (validator_pk, validator_sig) + } - let validator_pk = ValidatorPublicKey::from_bytes(&pk_bytes).unwrap(); - let validator_sig = ValidatorSignature::from_bytes(&sig_bytes).unwrap(); + /// Stands in for the startup call every binary makes. Without it the prover panics + /// on the missing aggregation bytecode. + fn init() { + init_leanvm(false); + } - (validator_pk, validator_sig) + /// A claim over one validator, the shape every Type-2 component takes here. + fn claim(message: H256, slot: u32, pk: &ValidatorPublicKey) -> SignerSet { + SignerSet::new(message, slot, vec![pk.clone()]) } #[test] + #[ignore = "slow: compiles the leanVM aggregation bytecode (needs a release-sized stack)"] fn test_setup_is_idempotent() { - // Should not panic when called multiple times - ensure_prover_ready(); - ensure_prover_ready(); - ensure_verifier_ready(); - ensure_verifier_ready(); + // Should not panic when called multiple times. The first call compiles + // the self-referential aggregation bytecode; subsequent calls are cheap + // (`OnceLock::get_or_init`). + init_leanvm(false); + init_leanvm(false); + + // The permit is dropped between acquisitions: it is not reentrant, so holding + // both at once would deadlock. That also covers release-on-drop. + drop(acquire_prover()); + drop(acquire_prover()); + } + + /// The claim list a decode rebuilds has to match what was aggregated, and + /// the shapes leanVM's signer set requires are this wrapper's job: one + /// group per slot, keys sorted and deduplicated, groups sorted by slot. + #[test] + fn wire_keys_groups_by_slot_and_sorts() { + let pk = |byte: u8| { + ValidatorPublicKey::from_bytes(&[byte; 32]).expect("any 32 bytes decode as a pubkey") + }; + let msg_a = H256::from([0xaau8; 32]); + let msg_b = H256::from([0xbbu8; 32]); + + // Slot 9 twice (keys unioned) and slot 4 once, handed over out of order. + let components = vec![ + SignerSet::new(msg_b, 9, vec![pk(3), pk(1)]), + SignerSet::new(msg_a, 4, vec![pk(2)]), + SignerSet::new(msg_b, 9, vec![pk(1), pk(2)]), + ]; + let (groups, sphincs) = wire_keys(&components).expect("one message per slot"); + + assert!(sphincs.is_empty(), "ethlambda signs XMSS only"); + let slots: Vec = groups.iter().map(|(slot, ..)| *slot).collect(); + assert_eq!(slots, vec![4, 9], "groups sorted by slot"); + assert_eq!(groups[0].1, msg_a.0); + assert_eq!(groups[1].1, msg_b.0); + assert_eq!(groups[0].2.len(), 1); + // Keys 1, 2, 3 unioned across the two slot-9 claims, deduplicated. + assert_eq!(groups[1].2.len(), 3); + assert!( + groups[1].2.windows(2).all(|w| w[0] < w[1]), + "keys strictly sorted" + ); + } + + /// Distinct `AttestationData` at one slot cannot share an aggregate, so the + /// wrapper says so instead of handing leanVM a set it cannot represent. + #[test] + fn wire_keys_rejects_two_messages_at_one_slot() { + let pk = ValidatorPublicKey::from_bytes(&[7u8; 32]).expect("32 bytes decode"); + let components = vec![ + SignerSet::new(H256::from([1u8; 32]), 6, vec![pk.clone()]), + SignerSet::new(H256::from([2u8; 32]), 6, vec![pk]), + ]; + let err = wire_keys(&components).expect_err("one slot, two messages"); + assert_eq!(err.slot, 6); } #[test] #[ignore = "too slow"] fn test_aggregate_single_signature() { + init(); let message = H256::from([42u8; 32]); let slot = 10u32; - let activation_epoch = 5u32; - let (pk, sig) = generate_keypair_and_sign(1, activation_epoch, slot, &message); + let (pk, sig) = generate_keypair_and_sign(1, 5, slot, &message); let result = aggregate_signatures(vec![pk.clone()], vec![sig], &message, slot); assert!(result.is_ok(), "Aggregation failed: {:?}", result.err()); @@ -578,21 +705,22 @@ mod tests { #[test] #[ignore = "too slow"] fn test_aggregate_multiple_signatures() { + init(); let message = H256::from([42u8; 32]); let slot = 15u32; - // Generate 3 keypairs with different activation epochs + // Generate 3 keypairs whose ranges all cover the signing slot. let configs = vec![ - (1u64, 5u32), // seed, activation_epoch - (2u64, 8u32), // seed, activation_epoch - (3u64, 10u32), // seed, activation_epoch + (1u64, 5u32), // seed, first signable slot + (2u64, 8u32), // seed, first signable slot + (3u64, 10u32), // seed, first signable slot ]; let mut pubkeys = Vec::new(); let mut signatures = Vec::new(); - for (seed, activation_epoch) in configs { - let (pk, sig) = generate_keypair_and_sign(seed, activation_epoch, slot, &message); + for (seed, first_slot) in configs { + let (pk, sig) = generate_keypair_and_sign(seed, first_slot, slot, &message); pubkeys.push(pk); signatures.push(sig); } @@ -614,12 +742,12 @@ mod tests { #[test] #[ignore = "too slow"] fn test_verify_wrong_message_fails() { + init(); let message = H256::from([42u8; 32]); let wrong_message = H256::from([43u8; 32]); let slot = 10u32; - let activation_epoch = 5u32; - let (pk, sig) = generate_keypair_and_sign(1, activation_epoch, slot, &message); + let (pk, sig) = generate_keypair_and_sign(1, 5, slot, &message); let proof_data = aggregate_signatures(vec![pk.clone()], vec![sig], &message, slot).unwrap(); @@ -635,12 +763,12 @@ mod tests { #[test] #[ignore = "too slow"] fn test_verify_wrong_slot_fails() { + init(); let message = H256::from([42u8; 32]); let slot = 10u32; let wrong_slot = 11u32; - let activation_epoch = 5u32; - let (pk, sig) = generate_keypair_and_sign(1, activation_epoch, slot, &message); + let (pk, sig) = generate_keypair_and_sign(1, 5, slot, &message); let proof_data = aggregate_signatures(vec![pk.clone()], vec![sig], &message, slot).unwrap(); @@ -653,12 +781,37 @@ mod tests { ); } + /// The signer set is not carried on the wire, so it is the caller-supplied + /// set that binds a proof to its participants. Supplying a different set of + /// the same size must be rejected: the proof commits to a hash of the set, + /// so it fails inside the SNARK verifier rather than at decode. + #[test] + #[ignore = "too slow"] + fn test_verify_wrong_pubkey_set_fails() { + init(); + let message = H256::from([42u8; 32]); + let slot = 10u32; + + let (pk, sig) = generate_keypair_and_sign(1, 5, slot, &message); + let (other_pk, _) = generate_keypair_and_sign(2, 5, slot, &message); + + let proof_data = aggregate_signatures(vec![pk], vec![sig], &message, slot).unwrap(); + + let verify_result = + verify_aggregated_signature(&proof_data, vec![other_pk], &message, slot); + assert!( + verify_result.is_err(), + "Verification should have failed with a different signer set" + ); + } + /// End-to-end Type-2 round-trip: produce two Type-1s (different (msg, slot)), /// merge them into a Type-2, verify the Type-2, then split out one component /// and verify it as a Type-1. #[test] #[ignore = "too slow"] fn test_type_2_merge_verify_split_round_trip() { + init(); let msg_a = H256::from([0x11u8; 32]); let msg_b = H256::from([0x22u8; 32]); let slot_a: u32 = 7; @@ -670,25 +823,18 @@ mod tests { let pa = aggregate_signatures(vec![pk_a.clone()], vec![sig_a], &msg_a, slot_a).unwrap(); let pb = aggregate_signatures(vec![pk_b.clone()], vec![sig_b], &msg_b, slot_b).unwrap(); - let merged = - merge_type_1s_into_type_2(vec![(vec![pk_a.clone()], pa), (vec![pk_b.clone()], pb)]) - .expect("merge"); - - verify_type_2_signature( - merged.iter().as_slice(), - vec![vec![pk_a.clone()], vec![pk_b.clone()]], - &[(msg_a, slot_a), (msg_b, slot_b)], - ) - .expect("verify type-2"); - - let split = split_type_2_by_message( - merged.iter().as_slice(), - vec![vec![pk_a.clone()], vec![pk_b.clone()]], - &msg_a, - ) - .expect("split"); - - verify_aggregated_signature(&split, vec![pk_a.clone()], &msg_a, slot_a) - .expect("verify split"); + let components = vec![claim(msg_a, slot_a, &pk_a), claim(msg_b, slot_b, &pk_b)]; + let merged = merge_type_1s_into_type_2(vec![ + (components[0].clone(), pa), + (components[1].clone(), pb), + ]) + .expect("merge"); + + verify_type_2_signature(merged.iter().as_slice(), &components).expect("verify type-2"); + + let split = + split_type_2_by_message(merged.iter().as_slice(), &components, &msg_a).expect("split"); + + verify_aggregated_signature(&split, vec![pk_a], &msg_a, slot_a).expect("verify split"); } } diff --git a/crates/common/crypto/src/signature.rs b/crates/common/crypto/src/signature.rs index 39515e53..9dbb65bc 100644 --- a/crates/common/crypto/src/signature.rs +++ b/crates/common/crypto/src/signature.rs @@ -1,183 +1,319 @@ -//! Validator XMSS signatures, public/secret keys, and the leansig-backed +//! Validator XMSS signatures, public/secret keys, and the leanVM-backed //! primitives behind them. -use std::ops::Range; +use std::ops::RangeInclusive; -use ethlambda_types::primitives::H256; -use leansig::{ - serialization::Serializable, - signature::{SignatureScheme, SignatureSchemeSecretKey as _, SigningError}, +use ethlambda_types::{attestation::SIGNATURE_SIZE, primitives::H256, state::PUBLIC_KEY_SIZE}; +use leanvm::xmss::{ + self, Decode, Encode, LOG_LIFETIME, PUB_KEY_SSZ_LEN, SIGNATURE_SSZ_LEN, XmssKeyGenError, + XmssPublicKey, XmssSecretKey, XmssSignError, XmssSignature, }; -/// The XMSS signature scheme used for validator signatures. +// `ethlambda-types` hardcodes the XMSS wire sizes so it can stay free of the +// signing backend. This crate is the only place that sees both sides, so it is +// where they get pinned together: a leanVM bump that changes the scheme +// parameters fails to compile here instead of silently corrupting the wire +// format (every `XmssSignature` / `ValidatorPubkeyBytes` in `ethlambda-types` is +// sized by these constants). +// +// Written as an array-length mismatch rather than `assert!` so that rustc prints +// both evaluated sizes; a const panic message has to be a string literal and so +// cannot name the numbers that actually differ. +// +// When one of these fires, correct the constant in `ethlambda-types` to the size +// rustc reports. Do not take rustc's `help:` suggestion to edit the length on +// these lines, which only silences the check. +const _: [(); SIGNATURE_SIZE] = [(); SIGNATURE_SSZ_LEN]; +const _: [(); PUBLIC_KEY_SIZE] = [(); PUB_KEY_SSZ_LEN]; + +/// Error returned when parsing signature or key bytes fails. +#[derive(Debug, Clone, thiserror::Error)] +#[error("signature parse error: {0}")] +pub struct SignatureParseError(pub String); + +/// What the pinned leanVM's XMSS is, for a genesis key manifest to record. +/// +/// A key set is only usable by a client built against the same scheme, and no +/// file size changes when the scheme does (leanVM's move from Poseidon over +/// KoalaBear to BLAKE2s over binary fields kept both wire sizes), so a manifest +/// has to say which scheme it holds rather than leave it to be inferred. /// -/// This is a post-quantum secure signature scheme based on hash functions. -/// Uses Poseidon1 hashing with an aborting hypercube message hash, -/// 32-bit lifetime (2^32 signatures per key), dimension 46, and base 8. -pub type LeanSignatureScheme = leansig::signature::generalized_xmss::instantiations_aborting::lifetime_2_to_the_32::SIGAbortingTargetSumLifetime32Dim46Base8; +/// Only two of these fields are load-bearing. [`PUBLIC_KEY_BYTES`] and +/// [`LIFETIME`] come from leanVM and so cannot drift; the two labels are +/// hardcoded, because leanVM's facade exports neither a name for the hash nor +/// the `V`/`CHAIN_LENGTH` the scheme label is conventionally built from. That is +/// exactly why a manifest should also carry the leanVM revision, which is the +/// one field that identifies the format on its own. +pub mod scheme { + use super::{LOG_LIFETIME, PUB_KEY_SSZ_LEN}; + + /// The scheme label, in the form the genesis tooling has always read. + /// + /// `Dim`/`Base` are leanVM's `V` and `CHAIN_LENGTH`, neither re-exported; + /// keep this in step with the pinned revision. Note that all three + /// parameters survived the last change of hash function unchanged, so this + /// label does NOT distinguish the two key formats by itself. + pub const NAME: &str = "XmssTargetSumLifetime32Dim42Base8"; -/// The public key type from the leansig library. -pub type LeanSigPublicKey = ::PublicKey; + /// The hash the scheme is built on. Hardcoded, as [`NAME`] is. + pub const HASH_FUNCTION: &str = "BLAKE2s"; -/// The signature type from the leansig library. -pub type LeanSigSignature = ::Signature; + /// The WOTS encoding: the signer grinds randomness until the digest's + /// chunks sum to a fixed target, so there are no checksum chains. + pub const ENCODING: &str = "TargetSum"; -/// The secret key type from the leansig library. -pub type LeanSigSecretKey = ::SecretKey; + /// Epochs a key could cover at most, whatever range it was generated for. + pub const LIFETIME: u64 = 1 << LOG_LIFETIME; -pub type Signature = LeanSigSignature; + /// Bytes in an SSZ-encoded public key. + pub const PUBLIC_KEY_BYTES: usize = PUB_KEY_SSZ_LEN; -/// Error returned when parsing signature or key bytes fails. -#[derive(Debug, Clone, thiserror::Error)] -#[error("signature parse error: {0}")] -pub struct SignatureParseError(pub String); + // [`NAME`] claims a 2^32 lifetime, which leanVM does export, so at least + // that much of the hardcoded label is checked rather than trusted. + const _: [(); 32] = [(); LOG_LIFETIME]; +} #[derive(Clone)] pub struct ValidatorSignature { - inner: LeanSigSignature, + inner: XmssSignature, } impl ValidatorSignature { + /// Parse from the SSZ-encoded wire form (`SIGNATURE_SIZE` bytes). pub fn from_bytes(bytes: &[u8]) -> Result { - let sig = LeanSigSignature::from_bytes(bytes) + let sig = XmssSignature::from_ssz_bytes(bytes) .map_err(|e| SignatureParseError(format!("{e:?}")))?; Ok(Self { inner: sig }) } + /// Encode to the SSZ wire form (`SIGNATURE_SIZE` bytes). pub fn to_bytes(&self) -> Vec { - self.inner.to_bytes() + self.inner.as_ssz_bytes() } pub fn is_valid(&self, pubkey: &ValidatorPublicKey, slot: u32, message: &H256) -> bool { - LeanSignatureScheme::verify(&pubkey.inner, slot, &message.0, &self.inner) + xmss::verify(&pubkey.inner, &message.0, &self.inner, slot).is_ok() } - pub fn into_inner(self) -> LeanSigSignature { + pub fn into_inner(self) -> XmssSignature { self.inner } } -#[derive(Clone)] +// `Debug` only on the public key: the signature and the secret key carry +// material that must not reach a log line by accident. +#[derive(Clone, Debug)] pub struct ValidatorPublicKey { - inner: LeanSigPublicKey, + inner: XmssPublicKey, } impl ValidatorPublicKey { + /// Parse from the SSZ-encoded wire form (`PUBLIC_KEY_SIZE` bytes). pub fn from_bytes(bytes: &[u8]) -> Result { - let pk = LeanSigPublicKey::from_bytes(bytes) + let pk = XmssPublicKey::from_ssz_bytes(bytes) .map_err(|e| SignatureParseError(format!("{e:?}")))?; Ok(Self { inner: pk }) } + /// Encode to the SSZ wire form (`PUBLIC_KEY_SIZE` bytes). pub fn to_bytes(&self) -> Vec { - self.inner.to_bytes() + self.inner.as_ssz_bytes() } - pub fn into_inner(self) -> LeanSigPublicKey { + pub fn into_inner(self) -> XmssPublicKey { self.inner } + + pub fn as_inner(&self) -> &XmssPublicKey { + &self.inner + } } /// Validator private key for signing attestations and blocks. pub struct ValidatorSecretKey { - inner: LeanSigSecretKey, + inner: XmssSecretKey, } impl ValidatorSecretKey { + /// Generate a fresh key able to sign at each slot in `slots`, once. + /// + /// The range is inclusive and fixed for the key's life: a slot outside it + /// can never be signed. Nothing can regenerate the key, the seed coming + /// from the OS. + /// + /// Cost grows with the range, but sublinearly: leanVM stores + /// `O(sqrt(range))` of the tree and fans key generation out over the bottom + /// subtrees. + pub fn generate(slots: RangeInclusive) -> Result { + let (sk, _pk) = xmss::key_gen(&mut leanvm::rand::rng(), *slots.start(), *slots.end())?; + Ok(Self { inner: sk }) + } + + /// Deterministic [`Self::generate`]: one `(seed, slots)` always rebuilds the + /// same key, the seed being its entire secret material. + pub fn generate_from_seed( + seed: [u8; 32], + slots: RangeInclusive, + ) -> Result { + let (sk, _pk) = xmss::key_gen_from_seed(seed, *slots.start(), *slots.end())?; + Ok(Self { inner: sk }) + } + + /// Parse from the postcard-encoded key file produced by the genesis generator. + /// + /// leanVM's `XmssSecretKey` carries serde rather than an SSZ codec: the + /// secret key never appears in consensus data, so only the two wire types + /// get one. pub fn from_bytes(bytes: &[u8]) -> Result { - let sk = LeanSigSecretKey::from_bytes(bytes) + let sk = postcard::from_bytes::(bytes) .map_err(|e| SignatureParseError(format!("{e:?}")))?; Ok(Self { inner: sk }) } - /// Sign a message with this private key. + /// Serialize the secret key to its postcard key-file form. + pub fn to_bytes(&self) -> Result, SignatureParseError> { + postcard::to_allocvec(&self.inner).map_err(|e| SignatureParseError(format!("{e:?}"))) + } + + /// The public key derived from this secret key. + pub fn public_key(&self) -> ValidatorPublicKey { + ValidatorPublicKey { + inner: self.inner.public_key(), + } + } + + /// Sign a message at `slot`. /// - /// The slot is used as part of the XMSS signature scheme to track - /// one-time signature usage. - pub fn sign(&self, slot: u32, message: &H256) -> Result { - let sig = LeanSignatureScheme::sign(&self.inner, slot, &message.0)?; + /// The slot indexes the one-time XMSS leaf; never sign two different + /// messages at the same slot. leanVM draws the signature randomness itself, + /// so even re-signing the same message at one slot leaks key material. + pub fn sign(&self, slot: u32, message: &H256) -> Result { + let sig = xmss::sign(&mut leanvm::rand::rng(), &self.inner, &message.0, slot)?; Ok(ValidatorSignature { inner: sig }) } - /// Returns true if the key is prepared to sign at the given slot. + /// Whether the key covers `slot` at all. /// - /// XMSS keys maintain a sliding window of two bottom trees. Only slots - /// within this window can be signed without advancing the preparation. - pub fn is_prepared_for(&self, slot: u32) -> bool { - self.inner.get_prepared_interval().contains(&(slot as u64)) + /// The range is fixed at key generation; a slot outside it can never be + /// signed, however long the node waits. + pub fn can_sign_at(&self, slot: u32) -> bool { + self.inner.epoch_range().contains(&slot) } - /// Returns the slot range currently covered by the prepared window. - pub fn get_prepared_interval(&self) -> Range { - self.inner.get_prepared_interval() + /// The inclusive slot range this key can sign for. + pub fn signable_slots(&self) -> RangeInclusive { + self.inner.epoch_range() } - /// Advance the prepared window forward by one bottom tree. + /// Warm the signing cache for `slot`, so [`Self::sign`] there does not pay + /// for rebuilding the bottom Merkle subtree. /// - /// Each call slides the window by sqrt(LIFETIME) = 65,536 slots. - /// If the window is already at the end of the key's activation interval, - /// this is a no-op. - pub fn advance_preparation(&mut self) { - self.inner.advance_preparation(); + /// The key holds one cached subtree, so this is worth calling only for the + /// slot about to be signed, and it is pure latency shifting: a miss inside + /// `sign` rebuilds the same subtree. Errors only when `slot` is outside + /// [`Self::signable_slots`]. + pub fn prepare(&self, slot: u32) -> Result<(), XmssSignError> { + self.inner.prepare(slot) } } #[cfg(test)] mod tests { use super::*; - use leansig::serialization::Serializable; - use rand::{SeedableRng, rngs::StdRng}; - const LEAVES_PER_BOTTOM_TREE: u32 = 1 << 16; // 65,536 + /// Generate a validator key pair over a small slot range (fast key generation). + fn generate_key(seed: [u8; 32], first_slot: u32, last_slot: u32) -> ValidatorSecretKey { + ValidatorSecretKey::generate_from_seed(seed, first_slot..=last_slot) + .expect("valid slot range") + } - /// Generate a ValidatorSecretKey with 3 bottom trees so advance_preparation can be tested. - /// - /// This is slow (~minutes) because it computes 3 bottom trees of 65,536 leaves each. - fn generate_key_with_three_bottom_trees() -> ValidatorSecretKey { - let mut rng = StdRng::seed_from_u64(42); - // Request enough active epochs for 3 bottom trees (> 2 * 65,536) - let num_active_epochs = (LEAVES_PER_BOTTOM_TREE as usize) * 2 + 1; - let (_pk, sk) = LeanSignatureScheme::key_gen(&mut rng, 0, num_active_epochs); - let sk_bytes = sk.to_bytes(); - ValidatorSecretKey::from_bytes(&sk_bytes).expect("valid secret key") + /// The scheme label is what a key manifest is read back against, so a + /// leanVM bump that moves a parameter has to move the label with it. + #[test] + fn scheme_reports_the_pinned_parameters() { + assert_eq!(scheme::NAME, "XmssTargetSumLifetime32Dim42Base8"); + assert_eq!(scheme::PUBLIC_KEY_BYTES, PUBLIC_KEY_SIZE); + assert_eq!(scheme::LIFETIME, 1 << 32); + } + + /// A generated key has to survive the postcard round trip the node's key + /// loader performs, and keep signing for the same public key. + #[test] + #[ignore = "slow: XMSS key generation and signing"] + fn generated_key_round_trips_through_its_file_form() { + let sk = ValidatorSecretKey::generate(0..=63).expect("valid slot range"); + let pk = sk.public_key(); + + let reloaded = ValidatorSecretKey::from_bytes(&sk.to_bytes().expect("serializes")) + .expect("its own file form parses"); + assert_eq!(reloaded.public_key().to_bytes(), pk.to_bytes()); + assert_eq!(reloaded.signable_slots(), sk.signable_slots()); + + let message = H256::from([5u8; 32]); + let signature = reloaded.sign(7, &message).expect("sign"); + assert!(signature.is_valid(&pk, 7, &message)); + } + + /// The seeded path is what makes a key set reproducible. + #[test] + #[ignore = "slow: XMSS key generation"] + fn seeded_generation_is_deterministic() { + let first = ValidatorSecretKey::generate_from_seed([11u8; 32], 0..=15).expect("range"); + let again = ValidatorSecretKey::generate_from_seed([11u8; 32], 0..=15).expect("range"); + let other = ValidatorSecretKey::generate_from_seed([12u8; 32], 0..=15).expect("range"); + + assert_eq!(first.public_key().to_bytes(), again.public_key().to_bytes()); + assert_ne!(first.public_key().to_bytes(), other.public_key().to_bytes()); + } + + /// An inverted range has no epochs to sign at, so it is rejected rather + /// than yielding a key that can do nothing. + #[test] + fn an_inverted_slot_range_is_rejected() { + // Built from variables: as a literal, `10..=9` is a clippy error rather + // than the input under test. + let (first, last) = (10u32, 9u32); + assert!(ValidatorSecretKey::generate_from_seed([0u8; 32], first..=last).is_err()); } #[test] - #[ignore = "slow: generates production-size XMSS key (~minutes)"] - fn test_advance_preparation_duration() { - println!("Generating XMSS key with 3 bottom trees (this takes a while)..."); - let keygen_start = std::time::Instant::now(); - let mut sk = generate_key_with_three_bottom_trees(); - println!("Key generation took: {:?}", keygen_start.elapsed()); - - // Initial window covers [0, 131072) - assert!(sk.is_prepared_for(0)); - assert!(sk.is_prepared_for(LEAVES_PER_BOTTOM_TREE - 1)); - assert!(sk.is_prepared_for(2 * LEAVES_PER_BOTTOM_TREE - 1)); - assert!(!sk.is_prepared_for(2 * LEAVES_PER_BOTTOM_TREE)); - - // Time the advance_preparation call - let advance_start = std::time::Instant::now(); - sk.advance_preparation(); - let advance_duration = advance_start.elapsed(); - - println!("advance_preparation() took: {advance_duration:?}"); - - // Window should now cover [65536, 196608) - assert!(!sk.is_prepared_for(0)); - assert!(sk.is_prepared_for(LEAVES_PER_BOTTOM_TREE)); - assert!(sk.is_prepared_for(3 * LEAVES_PER_BOTTOM_TREE - 1)); - - // Verify signing works in the new window + #[ignore = "slow: XMSS key generation and signing"] + fn sign_verify_round_trip() { + let sk = generate_key([7u8; 32], 0, 63); + let pk = sk.public_key(); + + assert!(sk.can_sign_at(0)); + assert!(sk.can_sign_at(63)); + assert!(!sk.can_sign_at(64)); + assert_eq!(sk.signable_slots(), 0..=63); + let message = H256::from([42u8; 32]); - let slot = 2 * LEAVES_PER_BOTTOM_TREE; // slot 131,072 — the one that crashed the devnet - let sign_start = std::time::Instant::now(); - let result = sk.sign(slot, &message); - println!("Signing at slot {slot} took: {:?}", sign_start.elapsed()); - assert!( - result.is_ok(), - "signing should succeed after advance: {}", - result.err().map_or(String::new(), |e| e.to_string()) - ); + let slot = 10u32; + sk.prepare(slot).expect("slot in range"); + let sig = sk.sign(slot, &message).expect("sign"); + assert!(sig.is_valid(&pk, slot, &message)); + assert!(!sig.is_valid(&pk, slot, &H256::from([43u8; 32]))); + assert!(!sig.is_valid(&pk, slot + 1, &message)); + } + + #[test] + #[ignore = "slow: XMSS key generation and signing"] + fn sign_out_of_range_fails() { + let sk = generate_key([9u8; 32], 100, 131); + let message = H256::from([1u8; 32]); + // Slot 0 is outside the key's range [100, 131]. + assert!(sk.sign(0, &message).is_err()); + assert!(sk.prepare(0).is_err()); + } + + #[test] + #[ignore = "slow: XMSS key generation"] + fn public_key_ssz_round_trip() { + let sk = generate_key([3u8; 32], 0, 15); + let pk = sk.public_key(); + let bytes = pk.to_bytes(); + assert_eq!(bytes.len(), PUBLIC_KEY_SIZE); + let parsed = ValidatorPublicKey::from_bytes(&bytes).expect("round trip"); + assert_eq!(parsed.to_bytes(), bytes); } } diff --git a/crates/common/crypto/tests/arena.rs b/crates/common/crypto/tests/arena.rs new file mode 100644 index 00000000..789465d8 --- /dev/null +++ b/crates/common/crypto/tests/arena.rs @@ -0,0 +1,48 @@ +//! Coverage for the arena path of [`init_leanvm`]. +//! +//! leanVM's arena engages process-wide and cannot be disengaged, so this cannot live in +//! the lib test binary: it would change the allocator under every other test. An +//! integration test gets its own process. + +use ethlambda_crypto::{ + aggregate_signatures, init_leanvm, + signature::{ValidatorPublicKey, ValidatorSignature}, + verify_aggregated_signature, +}; +use ethlambda_types::primitives::H256; +use leanvm::xmss::{self, Encode as _, key_gen_from_seed}; + +/// Mirrors the lib tests' helper: a small slot range keeps key generation fast. +fn keypair_and_signature( + seed: u64, + first_slot: u32, + signing_slot: u32, + message: &H256, +) -> (ValidatorPublicKey, ValidatorSignature) { + let mut seed_bytes = [0u8; 32]; + seed_bytes[..8].copy_from_slice(&seed.to_le_bytes()); + + let (sk, pk) = + key_gen_from_seed(seed_bytes, first_slot, first_slot + 63).expect("valid slot range"); + let sig = xmss::sign(&mut leanvm::rand::rng(), &sk, &message.0, signing_slot).expect("sign"); + + ( + ValidatorPublicKey::from_bytes(&pk.as_ssz_bytes()).unwrap(), + ValidatorSignature::from_bytes(&sig.as_ssz_bytes()).unwrap(), + ) +} + +#[test] +#[ignore = "too slow"] +fn aggregates_on_the_arena_when_enabled() { + init_leanvm(true); + + let message = H256::from([7u8; 32]); + let slot = 10u32; + let (pk, sig) = keypair_and_signature(1, 5, slot, &message); + + // Proves on the arena: the same round trip the lib tests run on the system allocator. + let proof = aggregate_signatures(vec![pk.clone()], vec![sig], &message, slot) + .expect("aggregation on the arena"); + verify_aggregated_signature(&proof, vec![pk], &message, slot).expect("verification"); +} diff --git a/crates/common/test-fixtures/src/common.rs b/crates/common/test-fixtures/src/common.rs index d4b2c2a9..c2485d44 100644 --- a/crates/common/test-fixtures/src/common.rs +++ b/crates/common/test-fixtures/src/common.rs @@ -329,7 +329,12 @@ where let pubkey: ValidatorPubkeyBytes = hex::decode(value.strip_prefix("0x").unwrap_or(&value)) .map_err(|_| D::Error::custom("ValidatorPubkey value is not valid hex"))? .try_into() - .map_err(|_| D::Error::custom("ValidatorPubkey length != 52"))?; + .map_err(|_| { + D::Error::custom(format!( + "ValidatorPubkey length != {}", + ethlambda_types::state::PUBLIC_KEY_SIZE + )) + })?; Ok(pubkey) } diff --git a/crates/common/test-fixtures/src/fork_choice.rs b/crates/common/test-fixtures/src/fork_choice.rs index c6c9e58c..93904d95 100644 --- a/crates/common/test-fixtures/src/fork_choice.rs +++ b/crates/common/test-fixtures/src/fork_choice.rs @@ -138,8 +138,8 @@ pub struct AttestationStepData { /// steps (leanSpec PR #717 schema). /// /// `participants` arrives as `{ data: [bool, ...] }` and `proof` as -/// `{ data: "0x" }`; the latter is the lean-multisig single-message -/// aggregate `compress_without_pubkeys()` bytes for that AttestationData. +/// `{ data: "0x" }`; the latter is the leanVM single-message +/// aggregate `to_bytes_without_pubkeys()` bytes for that AttestationData. #[derive(Debug, Clone, Deserialize)] #[serde(deny_unknown_fields)] pub struct ProofStepData { diff --git a/crates/common/test-fixtures/src/verify_signatures.rs b/crates/common/test-fixtures/src/verify_signatures.rs index a425ac81..d3693dfb 100644 --- a/crates/common/test-fixtures/src/verify_signatures.rs +++ b/crates/common/test-fixtures/src/verify_signatures.rs @@ -71,7 +71,7 @@ pub struct TestSignedBlock { /// Merged multi-message aggregate proof container for `SignedBlock.proof` /// (leanSpec PR #799). /// -/// The multi-signature container nests the raw lean-multisig wire one level +/// The multi-signature container nests the raw leanVM wire one level /// deep: `{ "proof": { "data": "0x..." } }`. #[derive(Debug, Clone, Deserialize)] #[serde(deny_unknown_fields)] @@ -135,7 +135,7 @@ impl TestSignedBlock { /// Materialize a `SignedBlock` preserving the fixture-supplied merged /// multi-message aggregate proof bytes verbatim. /// - /// The container carries the raw lean-multisig wire in the + /// The container carries the raw leanVM wire in the /// `MultiMessageAggregate` stored by `SignedBlock.proof`. pub fn try_into_signed_block_with_proofs(self) -> Result { let bytes = self diff --git a/crates/common/types/src/attestation.rs b/crates/common/types/src/attestation.rs index 91d00105..e4555235 100644 --- a/crates/common/types/src/attestation.rs +++ b/crates/common/types/src/attestation.rs @@ -54,44 +54,27 @@ pub struct SignedAttestation { pub signature: XmssSignature, } -/// Size of an XMSS signature in bytes. +/// Size of an SSZ-encoded XMSS signature in bytes. /// -/// Computed from: path(32*8*4) + rho(7*4) + hashes(46*8*4) + ssz_offsets(3*4) = 2536. -/// This is the SSZ wire size, independent of the `leansig` scheme itself, so it -/// lives here (leansig-free) rather than in `ethlambda-crypto`. -pub const SIGNATURE_SIZE: usize = 2536; +/// Mirrors leanVM's `xmss::SIGNATURE_SSZ_LEN`, hardcoded so this crate stays +/// free of the signing backend: parent containers only need the length, not the +/// scheme. `ethlambda-crypto` static-asserts the two agree, so a leanVM bump +/// that changes the scheme parameters breaks the build rather than silently +/// producing blobs of the wrong length. +pub const SIGNATURE_SIZE: usize = 1208; /// XMSS signature as a fixed-length byte vector (`SIGNATURE_SIZE` bytes). pub type XmssSignature = SszVector; -/// SSZ offset (in bytes) of the `path` body inside an XMSS `Signature` container. -/// -/// Layout: 4-byte path offset + 28-byte rho + 4-byte hashes offset = 36. -const SIGNATURE_PATH_OFFSET: u32 = 36; - -/// SSZ offset (in bytes) of the `hashes` body inside an XMSS `Signature`. -/// -/// `path` body is 4-byte siblings offset + LOG_LIFETIME (32) siblings × 32-byte -/// digest = 1028, starting at byte 36, so hashes start at 36 + 1028 = 1064. -const SIGNATURE_HASHES_OFFSET: u32 = 1064; - -/// SSZ offset (in bytes) of the `siblings` list inside the `path` container. -const SIGNATURE_PATH_SIBLINGS_OFFSET: u32 = 4; - -/// Build a placeholder XMSS signature that decodes as a structurally valid -/// leanSpec `Signature` container of all-zero hashes. +/// Build a placeholder XMSS signature of all-zero field elements. /// /// Used for genesis-style anchor blocks that were never proposed and therefore -/// have no real signature. Parent containers inline this as an opaque -/// `SIGNATURE_SIZE`-byte blob; consumers that decode the inner `Signature` -/// container see `path = HashTreeOpening { siblings = [0; 32] }`, `rho = 0`, -/// `hashes = [0; 46]`. Matches ream's `Signature::blank()` so the wire format -/// is byte-identical across clients. +/// have no real signature. leanVM's `XmssSignature` SSZ-encodes as a +/// fixed-length sequence of field elements (no variable-length offsets), and a +/// zero word is a valid canonical field element, so an all-zero blob decodes as +/// a structurally valid (but unverifiable) signature. pub fn blank_xmss_signature() -> XmssSignature { - let mut bytes = vec![0u8; SIGNATURE_SIZE]; - bytes[..4].copy_from_slice(&SIGNATURE_PATH_OFFSET.to_le_bytes()); - bytes[32..36].copy_from_slice(&SIGNATURE_HASHES_OFFSET.to_le_bytes()); - bytes[36..40].copy_from_slice(&SIGNATURE_PATH_SIBLINGS_OFFSET.to_le_bytes()); + let bytes = vec![0u8; SIGNATURE_SIZE]; XmssSignature::try_from(bytes).expect("size matches SIGNATURE_SIZE") } @@ -317,35 +300,14 @@ mod tests { assert!(!bits_is_subset(&a, &b)); } - /// Guard the three SSZ offsets at fixed byte positions so a one-off in any - /// constant doesn't silently produce a blob that still has the right outer - /// length but decodes incorrectly at the inner `Signature` container level. + /// The blank placeholder is exactly `SIGNATURE_SIZE` all-zero bytes and + /// decodes back as a structurally valid signature. #[test] - fn blank_xmss_signature_has_expected_ssz_offsets() { + fn blank_xmss_signature_is_zero_and_decodes() { let sig = blank_xmss_signature(); let bytes: Vec = sig.into_iter().collect(); assert_eq!(bytes.len(), SIGNATURE_SIZE); - assert_eq!( - u32::from_le_bytes(bytes[0..4].try_into().unwrap()), - SIGNATURE_PATH_OFFSET, - ); - assert_eq!( - u32::from_le_bytes(bytes[32..36].try_into().unwrap()), - SIGNATURE_HASHES_OFFSET, - ); - assert_eq!( - u32::from_le_bytes(bytes[36..40].try_into().unwrap()), - SIGNATURE_PATH_SIBLINGS_OFFSET, - ); - - // Everything outside the three offset slots must be zero — the - // placeholder is "all-zero hashes" once the offsets locate them. - for (i, b) in bytes.iter().enumerate() { - let in_offset_slot = matches!(i, 0..4 | 32..36 | 36..40); - if !in_offset_slot { - assert_eq!(*b, 0, "non-offset byte at index {i} should be zero"); - } - } + assert!(bytes.iter().all(|b| *b == 0)); } } diff --git a/crates/common/types/src/block.rs b/crates/common/types/src/block.rs index 5c5508a2..5bc21e89 100644 --- a/crates/common/types/src/block.rs +++ b/crates/common/types/src/block.rs @@ -48,11 +48,13 @@ pub type ByteList512KiB = ByteList<524_288>; /// A merged proof covering multiple messages with a single proof blob. /// -/// Also known as a *type-2* proof: the lean-multisig term for an aggregate -/// binding several distinct messages, each with its own participant set. +/// Also known as a *type-2* proof: leanVM's term for an aggregate binding +/// several distinct messages, each with its own participant set. /// -/// The proof bytes use lean-multisig's compact public-key-free -/// representation. SSZ encoding this container adds the offset required for +/// The proof bytes are the `to_bytes_without_pubkeys()` form of leanVM's +/// `MultiMessageAggregateSignature`: participant pubkeys stay off the wire, and +/// a verifier rebuilds each component's set from the surrounding block body +/// before decoding. SSZ encoding this container adds the offset required for /// its variable-length field. #[derive(Debug, Default, Clone, PartialEq, Eq, SszEncode, SszDecode, HashTreeRoot)] pub struct MultiMessageAggregate { @@ -66,7 +68,7 @@ impl MultiMessageAggregate { Self { proof } } - /// Copy raw lean-multisig proof bytes into the bounded SSZ container. + /// Copy raw leanVM aggregation proof bytes into the bounded SSZ container. pub fn from_bytes(bytes: &[u8]) -> Result { let len = bytes.len(); ByteList512KiB::try_from(bytes.to_vec()) @@ -74,7 +76,7 @@ impl MultiMessageAggregate { .map_err(|_| MultiMessageAggregateError::ProofTooLarge(len)) } - /// Return the raw lean-multisig proof bytes. + /// Return the raw leanVM aggregation proof bytes. pub fn proof_bytes(&self) -> &[u8] { self.proof.iter().as_slice() } @@ -98,7 +100,7 @@ pub enum MultiMessageAggregateError { // from the surrounding block body (attestation `data` + slot for body // components, block root + slot for the proposer component). // -// `MultiMessageAggregate` carries the raw lean-multisig type-2 bytes. +// `MultiMessageAggregate` carries the raw leanVM type-2 bytes. // Component participant bitfields come from // `block.body.attestations[i].aggregation_bits` (and `block.proposer_index` for // the trailing proposer entry). @@ -110,8 +112,8 @@ pub const MAX_ATTESTATIONS_DATA: usize = 8; /// A single-message proof aggregating signatures from many validators. /// -/// Also known as a *type-1* proof: the lean-multisig term for an aggregate -/// where every participant signs the same message for the same slot. +/// Also known as a *type-1* proof: leanVM's term for an aggregate where every +/// participant signs the same message for the same slot. /// /// Used: /// - as a gossip-level `SignedAggregatedAttestation.proof`, @@ -119,15 +121,19 @@ pub const MAX_ATTESTATIONS_DATA: usize = 8; /// - as one of the components fed into `merge_type_1s_into_type_2` when /// building a block proof. /// -/// `participants` and `proof` are independent fields: the proof bytes are -/// the lean-multisig `compress_without_pubkeys()` form; `participants` is -/// the bitfield identifying which validators are bound by the proof. The -/// verifier resolves pubkeys from `participants` at verify time. +/// `participants` and `proof` are independent fields: the proof bytes are the +/// `to_bytes_without_pubkeys()` form of leanVM's +/// `SingleMessageAggregateSignature`, and `participants` is the bitfield +/// identifying which validators are bound. Pubkeys are not on the wire: the +/// verifier resolves them from `participants` against its own validator +/// registry and attaches them when decoding. Attaching a set other than the one +/// aggregated succeeds at decode and fails at verification, since the proof +/// binds a hash of the set. #[derive(Debug, Clone, SszEncode, SszDecode, HashTreeRoot)] pub struct SingleMessageAggregate { /// Bitfield identifying validators bound by this proof. pub participants: AggregationBits, - /// Aggregated proof bytes in lean-multisig compact (no-pubkeys) form. + /// Aggregated proof bytes in leanVM's `to_bytes_without_pubkeys()` form. pub proof: ByteList512KiB, } diff --git a/crates/common/types/src/genesis.rs b/crates/common/types/src/genesis.rs index 84091adc..4e91bf02 100644 --- a/crates/common/types/src/genesis.rs +++ b/crates/common/types/src/genesis.rs @@ -192,7 +192,11 @@ where let bytes = hex::decode(s).map_err(|_| D::Error::custom(format!("pubkey is not valid hex: {s}")))?; bytes.try_into().map_err(|v: Vec| { - D::Error::custom(format!("pubkey has length {} (expected 52)", v.len())) + D::Error::custom(format!( + "pubkey has length {} (expected {})", + v.len(), + crate::state::PUBLIC_KEY_SIZE + )) }) } @@ -204,12 +208,12 @@ mod tests { state::{State, Validator}, }; - const ATT_PUBKEY_A: &str = "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d2d25ca7b19604f5a180037046cf2a02e1da4a800"; - const PROP_PUBKEY_A: &str = "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940fa03e4c012e8a0e656b731934aeac124a5337e333"; - const ATT_PUBKEY_B: &str = "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05e143e305017847445859650d6dd06e6efb3f8410"; - const PROP_PUBKEY_B: &str = "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d2d25ca7b19604f5a180037046cf2a02e1da4a800"; - const ATT_PUBKEY_C: &str = "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940fa03e4c012e8a0e656b731934aeac124a5337e333"; - const PROP_PUBKEY_C: &str = "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05e143e305017847445859650d6dd06e6efb3f8410"; + const ATT_PUBKEY_A: &str = "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d"; + const PROP_PUBKEY_A: &str = "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940f"; + const ATT_PUBKEY_B: &str = "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05"; + const PROP_PUBKEY_B: &str = "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d"; + const ATT_PUBKEY_C: &str = "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940f"; + const PROP_PUBKEY_C: &str = "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05"; const TEST_CONFIG_YAML: &str = r#"# Genesis Settings GENESIS_TIME: 1770407233 @@ -222,12 +226,12 @@ VALIDATOR_COUNT: 3 # Genesis Validator Pubkeys GENESIS_VALIDATORS: - - attestation_pubkey: "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d2d25ca7b19604f5a180037046cf2a02e1da4a800" - proposal_pubkey: "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940fa03e4c012e8a0e656b731934aeac124a5337e333" - - attestation_pubkey: "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05e143e305017847445859650d6dd06e6efb3f8410" - proposal_pubkey: "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d2d25ca7b19604f5a180037046cf2a02e1da4a800" - - attestation_pubkey: "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940fa03e4c012e8a0e656b731934aeac124a5337e333" - proposal_pubkey: "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05e143e305017847445859650d6dd06e6efb3f8410" + - attestation_pubkey: "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d" + proposal_pubkey: "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940f" + - attestation_pubkey: "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05" + proposal_pubkey: "cd323f232b34ab26d6db7402c886e74ca81cfd3a0c659d2fe022356f25592f7d" + - attestation_pubkey: "b7b0f72e24801b02bda64073cb4de6699a416b37dfead227d7ca3922647c940f" + proposal_pubkey: "8d9cbc508b20ef43e165f8559c1bdd18aaeda805ef565a4f9ffd6e4fbed01c05" "#; #[test] @@ -294,7 +298,7 @@ GENESIS_VALIDATORS: // Pin the state root so SSZ layout changes are caught immediately. let expected_state_root = crate::primitives::H256::from_slice( - &hex::decode("babcdc9235a29dfc0d605961df51cfc85732f85291c2beea8b7510a92ec458fe") + &hex::decode("3e8c8507e94e045327c2fc66a58db374805cb490a087b3101bb13a9b8b611b54") .unwrap(), ); assert_eq!(root, expected_state_root, "state root mismatch"); @@ -303,7 +307,7 @@ GENESIS_VALIDATORS: block.state_root = root; let block_root = block.hash_tree_root(); let expected_block_root = crate::primitives::H256::from_slice( - &hex::decode("66a8beaa81d2aaeac7212d4bf8f5fea2bd22d479566a33a83c891661c21235ef") + &hex::decode("ba3502921697db025b3a6d7c05fbaf58e52155575438cca9794e22e6e9872090") .unwrap(), ); assert_eq!(block_root, expected_block_root, "block root mismatch"); diff --git a/crates/common/types/src/state.rs b/crates/common/types/src/state.rs index d117151e..5ef6100c 100644 --- a/crates/common/types/src/state.rs +++ b/crates/common/types/src/state.rs @@ -85,7 +85,16 @@ where serializer.serialize_str(&hex::encode(pubkey)) } -pub type ValidatorPubkeyBytes = [u8; 52]; +/// Size of an SSZ-encoded XMSS public key in bytes. +/// +/// Mirrors leanVM's `xmss::PUB_KEY_SSZ_LEN`, hardcoded so this crate stays free +/// of the signing backend: the validator registry is a plain wire type. +/// `ethlambda-crypto` static-asserts the two agree, so a leanVM bump that +/// changes the scheme parameters breaks the build rather than silently +/// producing keys of the wrong length. +pub const PUBLIC_KEY_SIZE: usize = 32; + +pub type ValidatorPubkeyBytes = [u8; PUBLIC_KEY_SIZE]; impl State { pub fn from_genesis(genesis_time: u64, validators: Vec) -> Self { diff --git a/crates/net/rpc/src/genesis.rs b/crates/net/rpc/src/genesis.rs index 76633aca..e8defc5f 100644 --- a/crates/net/rpc/src/genesis.rs +++ b/crates/net/rpc/src/genesis.rs @@ -43,8 +43,8 @@ mod tests { async fn genesis_returns_time_and_validator_count() { // Build a state with 3 validators so the assertion is non-vacuous. let dummy_validator = |index: u64| Validator { - attestation_pubkey: [0u8; 52], - proposal_pubkey: [0u8; 52], + attestation_pubkey: [0u8; 32], + proposal_pubkey: [0u8; 32], index, }; let validators = vec![dummy_validator(0), dummy_validator(1), dummy_validator(2)]; diff --git a/crates/storage/Cargo.toml b/crates/storage/Cargo.toml index 035d802a..f230862d 100644 --- a/crates/storage/Cargo.toml +++ b/crates/storage/Cargo.toml @@ -25,5 +25,3 @@ lru.workspace = true [dev-dependencies] tempfile = "3" -leansig.workspace = true -rand.workspace = true diff --git a/crates/storage/src/state_diff.rs b/crates/storage/src/state_diff.rs index 1e9e5028..261619f8 100644 --- a/crates/storage/src/state_diff.rs +++ b/crates/storage/src/state_diff.rs @@ -248,13 +248,13 @@ mod tests { fn base_state() -> State { let validators = vec![ Validator { - attestation_pubkey: [1u8; 52], - proposal_pubkey: [2u8; 52], + attestation_pubkey: [1u8; 32], + proposal_pubkey: [2u8; 32], index: 0, }, Validator { - attestation_pubkey: [3u8; 52], - proposal_pubkey: [4u8; 52], + attestation_pubkey: [3u8; 32], + proposal_pubkey: [4u8; 32], index: 1, }, ]; diff --git a/crates/storage/src/store.rs b/crates/storage/src/store.rs index 2c775c1c..998cfc67 100644 --- a/crates/storage/src/store.rs +++ b/crates/storage/src/store.rs @@ -1825,13 +1825,14 @@ mod tests { use crate::backend::InMemoryBackend; use ethlambda_types::constants::DEFAULT_MILLISECONDS_PER_SLOT; use ethlambda_types::genesis::{GenesisMismatch, GenesisValidatorEntry}; + use ethlambda_types::state::PUBLIC_KEY_SIZE; /// Validator at `index` whose two pubkeys are filled with `seed`, so /// changing the seed changes the registry without changing its size. fn validator(index: u64, seed: u8) -> Validator { Validator { - attestation_pubkey: [seed; 52], - proposal_pubkey: [seed.wrapping_add(1); 52], + attestation_pubkey: [seed; PUBLIC_KEY_SIZE], + proposal_pubkey: [seed.wrapping_add(1); PUBLIC_KEY_SIZE], index, } } @@ -2203,8 +2204,8 @@ mod tests { /// post-state's `latest_block_header`). fn sample_state(slot: u64, parent_root: H256, hbh: Vec) -> State { let validators = vec![Validator { - attestation_pubkey: [7u8; 52], - proposal_pubkey: [9u8; 52], + attestation_pubkey: [7u8; 32], + proposal_pubkey: [9u8; 32], index: 0, }]; let mut state = State::from_genesis(1_000, validators); @@ -2892,19 +2893,12 @@ mod tests { // ============ GossipSignatureBuffer Tests ============ fn make_dummy_sig() -> ValidatorSignature { - use ethlambda_crypto::signature::LeanSignatureScheme; - use leansig::{serialization::Serializable, signature::SignatureScheme}; - use rand::{SeedableRng, rngs::StdRng}; - - static CACHED_SIG: std::sync::LazyLock> = std::sync::LazyLock::new(|| { - let mut rng = StdRng::seed_from_u64(42); - let lifetime = 1 << 5; // small for speed - let (_pk, sk) = LeanSignatureScheme::key_gen(&mut rng, 0, lifetime); - let sig = LeanSignatureScheme::sign(&sk, 0, &[0u8; 32]).unwrap(); - sig.to_bytes() - }); - - ValidatorSignature::from_bytes(&CACHED_SIG).expect("cached test signature") + // These tests never check signature validity, only that a signature + // decodes and carries through the buffers. An all-zero blob is a + // structurally valid (unverifiable) XMSS signature. + use ethlambda_types::attestation::SIGNATURE_SIZE; + ValidatorSignature::from_bytes(&vec![0u8; SIGNATURE_SIZE]) + .expect("all-zero test signature decodes") } #[test] diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 6b6f9c50..43d4ccde 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -20,6 +20,7 @@ - [Fork Choice Visualization](./fork_choice_visualization.md) - [Data Storage](./data_storage.md) - [Peer discovery](./discovery.md) +- [Validator Key Generation](./keygen.md) # Development diff --git a/docs/benchmarking.md b/docs/benchmarking.md index 924437b7..e21126b6 100644 --- a/docs/benchmarking.md +++ b/docs/benchmarking.md @@ -79,7 +79,7 @@ otherwise, because a mis-attributed report is worse than no report. Block-building benchmark — synthetic workload (mock crypto) validators=8 warmup_slots=8 iterations=10 proofs_per_data=1 seed=42 enable_proposer_aggregation=false max_attestations_per_block=3 - ethlambda/v0.1.0/aarch64-apple-darwin/rustc-v1.97.1 leansig=15cbdd43 leanvm=e2592df4 os=macos arch=aarch64 threads=14 + ethlambda/v0.1.0/aarch64-apple-darwin/rustc-v1.97.1 leanvm=5a4f55c1 os=macos arch=aarch64 threads=14 iter compact select_payloads stf_simulate overhead wall root 1 0.000ms 0.002ms 0.015ms 0.068ms 0.085ms 0x7282cc99 @@ -91,7 +91,7 @@ Block-building benchmark — synthetic workload (mock crypto) ``` Every measured iteration gets its own row, and the summary follows below it. -Outliers are never discarded: XMSS signing and OTS window advancement produce +Outliers are never discarded: XMSS signing and its Merkle-subtree cache misses produce legitimate heavy tails, and hiding them would misrepresent the thing being measured. A coefficient of variation above 10% is flagged so a noisy run is not mistaken for a result. @@ -112,10 +112,9 @@ Same seed and same parameters produce identical root sequences, so a baseline and a candidate can be diffed directly. The header line exists to tell you when they *cannot* be compared: -- `leansig` and `leanvm` are the resolved revisions the binary was built - against, read from `Cargo.lock` at build time. leanSig tracks a moving branch - and leanVM performs the signature aggregation, so either one moving changes - the measured crypto. +- `leanvm` is the resolved revision the binary was built against, read from + `Cargo.lock` at build time. leanVM owns the whole signature stack (XMSS and + aggregation), so a rev bump changes the measured crypto. - `os`, `arch` and `threads` change results across machines. Two reports that disagree on any of those are not measuring the same thing. diff --git a/docs/keygen.md b/docs/keygen.md new file mode 100644 index 00000000..9e74263c --- /dev/null +++ b/docs/keygen.md @@ -0,0 +1,117 @@ +# Validator Key Generation + +`ethlambda keygen` writes the validator XMSS key set and manifest that +`--hash-sig-keys-dir` reads, in the layout `hash-sig-cli generate` produces. + +```bash +# One validator's attester and proposer pair +ethlambda keygen --output-dir keys + +# A genesis for a three-node devnet +ethlambda keygen --num-validators 3 --output-dir local-devnet/genesis/hash-sig-keys +``` + +`--output-dir` is the only required flag. + +## Why the client generates its own keys + +An XMSS key file is only usable by a client built against the same signature +scheme, and the scheme lives in leanVM, which ethlambda pins to one revision. + +The trap is that nothing in the file layout changes when the scheme does. +leanVM's move from Poseidon over KoalaBear to BLAKE2s over binary fields kept +the public key at 32 SSZ bytes and the secret key in postcard, so a key set from +the wrong revision satisfies every check a genesis generator makes: the file +names match, the sizes match, and the manifest parses. It fails only later, when +a signature is verified, and then it looks like a consensus bug rather than a +provisioning one. + +Generating here removes the second pin. These keys come from the same +`ethlambda-crypto` types the node loads them with, so the generator and the +loader cannot disagree about the format. + +## Flags + +| Flag | Default | Meaning | +| --- | --- | --- | +| `--num-validators ` | `1` | Validators to generate a key pair for | +| `--log-num-active-epochs ` | `18` | Log2 of the slots each key can sign at, from slot 0 | +| `--output-dir ` | required | Where to write; created if absent | +| `--create-manifest ` | `true` | Write `validator-keys-manifest.yaml` | +| `--distributed` | off | Name validators by public key rather than by index | +| `--force` | off | Replace key files already in `--output-dir` | + +Each validator gets **two** independent keys, an attester and a proposer, so it +can sign an attestation and a block in the same slot without spending one slot's +one-time leaf twice. + +`--log-num-active-epochs` is the network's lifetime, not a tuning knob: a key +cannot sign past its range, and the range is fixed at generation. At the default +cadence 2^18 slots is about 12 days, after which every validator holding such a +key stops signing. + +`--force` is off by default because a key is one-time-use material. Replacing a +set that validators are still signing with makes each of them sign twice at the +same slot, under a key someone else now holds. + +## Output + +``` +hash-sig-keys/ +├── validator_0_attester_key_pk.ssz 32 bytes, SSZ +├── validator_0_attester_key_sk.ssz postcard +├── validator_0_proposer_key_pk.ssz +├── validator_0_proposer_key_sk.ssz +├── validator_1_... +└── validator-keys-manifest.yaml +``` + +The `.ssz` extension on a secret key is a misnomer kept for the tooling's sake: +SSZ has no encoding for one, so it is postcard. + +`hash-sig-cli` had an `--export-format both` that additionally dumped each key as +serde JSON. Its own help called that legacy and nothing reads it, so this writes +only the form above and takes no format flag. A caller carrying +`--export-format ssz` from the old invocation has to drop it. + +## The manifest + +```yaml +key_scheme: XmssTargetSumLifetime32Dim42Base8 +hash_function: BLAKE2s +encoding: TargetSum +pubkey_bytes: 32 +lifetime: 4294967296 +leanvm_rev: 5a4f55c1138759f43f78483a7b70fde973e4a1ee +log_num_active_epochs: 18 +num_active_epochs: 262144 +num_validators: 3 + +validators: + - index: 0 + proposer_key_pubkey_hex: 0x... + proposer_key_privkey_file: validator_0_proposer_key_sk.ssz + attester_key_pubkey_hex: 0x... + attester_key_privkey_file: validator_0_attester_key_sk.ssz +``` + +`leanvm_rev` is ethlambda's addition, and it is the field to trust. Read against +the scheme parameters alone, two key sets from either side of the BLAKE2s rewrite +are indistinguishable: `key_scheme` is built from the lifetime, `V` and +`CHAIN_LENGTH`, and all three came through that change unaltered. `hash_function` +does separate them but is a hardcoded label, since leanVM's facade exports no +name for its hash. The revision is resolved from `Cargo.lock` at build time and +cannot drift from what the binary actually links. + +`generate-genesis.sh` reads `key_scheme` and cross-checks `pubkey_bytes` against +the pubkeys the manifest holds; the rest is informational. + +## Cost + +Key generation is `O(sqrt(range))` in memory and fans out over the bottom Merkle +subtrees, so a large range is cheaper than it looks: about 3 seconds per +validator at 2^18 epochs on an M4 Max, generated serially. A 1024-validator +genesis is therefore tens of minutes, and worth doing once and keeping. + +Building on aarch64 without the crypto extensions makes this, and every other +leanVM operation, far slower. See the aarch64 entry in `.cargo/config.toml`.