Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors PartialResultStream to optimize row buffering and chunk processing by replacing dynamic array pushes with a pre-allocated array and a write cursor. It also structures chunk processing into five distinct stages and adds comprehensive unit tests covering backpressure, multi-chunk streaming, and buffer reuse. The review feedback highlights potential runtime errors and out-of-bounds access if an incoming chunk is empty during pending value merging or chunked value handling, and suggests simplifying a redundant ternary check when initializing the row values array.
ef1b67f to
177101d
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors PartialResultStream to use a pre-allocated row buffer (_rowValues) and a cursor index (_valueIndex) instead of a dynamic array, improving performance and memory usage during chunked streaming. It also updates chunk processing, adds checkpoint tracking for stream retries, and introduces new tests for backpressure and multi-chunk streaming. However, a critical bug was identified where _rowValues and _valueIndex are overwritten when a stream resumes and receives metadata, discarding the restored state and leading to data corruption. Removing the state reset from the metadata block is recommended to preserve the resume checkpoint.
177101d to
d0ba78d
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors PartialResultStream to use a pre-allocated row buffer (_rowValues) and index tracking (_valueIndex) instead of dynamically pushing to an array, improving memory efficiency and streaming robustness. It also adds comprehensive unit tests covering backpressure, multi-chunk streaming, and stream resumption. The review feedback recommends clearing stale elements in _rowValues from the reset _valueIndex to the end of the array during _resetPendingValues() to prevent potential memory retention of large decoded objects.
…esultStream
Eliminate intermediate array allocations and array mutations during query
stream processing in PartialResultStream:
- Process chunk values in-place using `startIndex` and `endIndex` cursors
in `_addChunk`, eliminating the intermediate `values` array allocation and
O(N) `shift`/`unshift` array operations during chunked value merging.
- Pre-allocate a single row buffer (`_rowValues`) per result set and track
cell progress with an index cursor (`_valueIndex`), eliminating per-row
array allocations in `_addValue`.
- Preserve the row write cursor (`_valueIndexForResume`) alongside pending
chunked values on resume token checkpoints, ensuring retries across
incomplete rows correctly resume at the exact column position.
- Guard `this.emit('paused')` to fire at most once per chunk upon entering
the backpressured state.
d0ba78d to
7a91087
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors PartialResultStream to optimize row assembly and stream chunk processing. It replaces the dynamic _values array with a pre-allocated _rowValues array and a _valueIndex cursor, improving memory usage and avoiding cross-row contamination. Additionally, it enhances resume token checkpointing by preserving the exact column position (_valueIndexForResume) and pending chunked values during retries. Comprehensive unit tests have been added to verify backpressure handling, multi-chunk streaming, row buffer reuse, and state restoration upon stream resumption. No review comments were provided, so there is no feedback to address.
Eliminate intermediate array allocations and array mutations during query stream processing in PartialResultStream:
startIndexandendIndexcursors in_addChunk, eliminating the intermediatevaluesarray allocation and O(N)shift/unshiftarray operations during chunked value merging._rowValues) per result set and track cell progress with an index cursor (_valueIndex), eliminating per-row array allocations in_addValue._valueIndexForResume) alongside pending chunked values on resume token checkpoints, ensuring retries across incomplete rows correctly resume at the exact column position.this.emit('paused')to fire at most once per chunk upon entering the backpressured state.