HBASE-30092 Avoid stringifying protobuf RPC parameters in task snapshots - #8489
HBASE-30092 Avoid stringifying protobuf RPC parameters in task snapshots#8489Sigma-Ma wants to merge 1 commit into
Conversation
|
Hi maintainers, the three GitHub Actions workflows for this fork PR are awaiting approval. This PR does not modify any workflow files. Could someone with write access approve them when convenient? Thanks. |
|
Hi @liuxiaocs7, sorry for the direct ping. This small HBASE-30092 fix is currently blocked because the three fork workflows still require maintainer approval. It avoids stringifying large protobuf RPC parameters in MonitoredRPCHandlerImpl and includes a focused regression test. Could you please approve the workflows and take a look when convenient, or suggest a more appropriate reviewer? Thanks! |
|
Hi @Apache9, sorry for the direct ping. This small HBASE-30092 fix has been open for about a month and currently has no reviewer. All three GitHub Actions checks are green, and the PR is mergeable. The change avoids stringifying large in-flight protobuf requests in MonitoredRPCHandlerImpl, preventing the RPC monitoring and RegionServer heartbeat path from causing excessive heap allocation. A focused regression test is included. Could you please take a look when convenient, or suggest a more appropriate reviewer? Thanks! |
JIRA: https://issues.apache.org/jira/browse/HBASE-30092
What changes were proposed in this pull request?
This change avoids calling
Message.toString()whenMonitoredRPCHandlerImplgenerates RPC monitoring information.For protobuf parameters, HBase now records only the protobuf message type, such as
ScanRequest. The existingpacketlengthfield continues to report the serialized request size.Why are the changes needed?
HBASE-30092 reports that
MonitoredRPCHandlerImpl.generateCallInfoMap()can cause an out-of-memory error while generating RegionServer heartbeat information for a large in-flight RPC request.Calling
toString()on a protobuf recursively materializes the complete TextFormat representation. For large RPC requests, this creates a significant additional allocation in the monitoring path and can exhaust the RegionServer heap.Recording only the protobuf message type keeps monitoring allocation independent of the request payload size while preserving the RPC method and serialized packet length.
How was this patch tested?
Added a regression test to
TestTaskMonitor, covering:ScanRequest.The focused unit test can be run with: