Add Power-of-Two-Choices Peak-EWMA load balancer (p2c)#3367
Conversation
Implements the p2c load balancing policy proposed in apache#3340: each selection samples two random servers (configurable via choices=N) and routes to the lower peak-EWMA latency * (inflight+1) / weight score. Upward latency spikes take effect immediately while recovery decays over tau_ms (default 10s), so a degraded server is shed within one observation at O(1) selection cost. Uses only existing LoadBalancer hooks (SelectServer/Feedback, need_feedback) with DoublyBufferedData membership like rr/la; per-node stats are shared_ptr-owned by both buffers. Registered as "p2c" in global.cpp. Includes unit tests (functional, weighted, exclusion, error punishment, concurrency churn) and docs in cn/en client.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a new client-side load balancing policy, p2c (Power-of-Two-Choices with Peak-EWMA latency scoring), to improve tail latency by quickly shifting traffic away from slow or failing backends using per-node peak-sensitive EWMA latency and in-flight request tracking.
Changes:
- Introduces
P2CEwmaLoadBalancerimplementing P2C sampling with Peak-EWMA latency scoring and in-flight bias. - Registers the new balancer under the name
p2cin global load balancer extensions. - Adds unit tests and updates user-facing documentation and tooling help text to describe/enable
p2c.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/rpc_press/rpc_press.cpp | Adds p2c to the -lb_policy help text. |
| test/brpc_p2c_ewma_load_balancer_unittest.cpp | Adds unit tests covering selection behavior, weights, errors, exclusions, and concurrent churn. |
| src/brpc/policy/p2c_ewma_load_balancer.h | Declares the new P2CEwmaLoadBalancer and its per-node stats/membership structures. |
| src/brpc/policy/p2c_ewma_load_balancer.cpp | Implements P2C selection, Peak-EWMA scoring/decay, and feedback updates. |
| src/brpc/global.cpp | Registers p2c in the global load balancer registry. |
| docs/en/client.md | Documents the p2c policy and its parameters. |
| docs/cn/rpc_press.md | Updates rpc_press documentation to list p2c as a supported -lb_policy. |
| docs/cn/client.md | Adds Chinese documentation for the p2c policy and its parameters. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1894ef1 to
aeb33d5
Compare
…ap init - p2c_default_choices/p2c_default_tau_ms gflags as channel-overridable defaults - p2c_max_punish_ms(default 30s) caps the doubled-EWMA error punishment so a persistently failing server recovers in bounded time after turning healthy - Servers::server_map relies on FlatMap small-map auto-init - tests: error_punish_is_capped + feedback_lock_overhead benchmark Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@chenBright thanks for the review — all four comments are addressed in the latest commit: choices/tau defaults are now gflags, error punishment is capped by |
What problem does this PR solve?
Issue Number: resolve #3340
Problem Summary: brpc has no Power-of-Two-Choices load balancer — the most widely deployed tail-latency-aware policy (Envoy LEAST_REQUEST, Finagle/linkerd Peak-EWMA). A single degraded backend keeps receiving 25% of
rrtraffic until humans intervene;la's averaging window reacts slower than one observation.What is changed and the side effects?
Changed: New
p2cpolicy (src/brpc/policy/p2c_ewma_load_balancer.{h,cpp}): each selection samples two random servers (p2c:choices=Nwidens; evaluates all when N ≥ cluster size, with random tie-breaking to avoid herding) and routes to the lowerpeak_ewma_latency_us * (inflight + 1) / weight. Latency spikes replace the average immediately; recovery decays overtau_ms(default 10s); failures are punished with at least the RPC timeout. Uses only existing hooks (SelectServer/Feedback,DoublyBufferedDatamembership likerr); per-node stats are stable pointers shared by both buffers as inla. Registered inglobal.cpp; 10 unit tests;p2cdocs indocs/{cn,en}/client.md.Benchmark (rpc_press, 4 echo backends, slow = +5ms
bthread_usleep, qps=4000, 15s, 50 threads, 3 reps averaged, zero errors; p99/p999 µs):lap2c(best-of-2)p2c:choices=4rrSlow-node traffic share (1 slow of 4):
rr25%,p2c1.3%,p2c:choices=40.6%,la0.0% — consistent with the numbers on #3340.Side effects:
p2cselection is O(1) (two score evaluations).Check List:
brpc_p2c_ewma_load_balancer_unittestpasses 10/10 andbrpc_load_balancer_unittestpasses 16/16.cc @chenBright (thanks for the green light on #3340!) @zyearn