Skip to content

feat: support bitslice for rangebitmap#174

Merged
lxy-9602 merged 3 commits intoalibaba:mainfrom
fafacao86:rangebitmap-bitslice
Mar 14, 2026
Merged

feat: support bitslice for rangebitmap#174
lxy-9602 merged 3 commits intoalibaba:mainfrom
fafacao86:rangebitmap-bitslice

Conversation

@fafacao86
Copy link
Contributor

Purpose

Linked issue: close #146

Tests

UT tests in src/paimon/common/file_index/rangebitmap/bit_slice_index_bitmap_test.cpp

  1. Functional tests: Serialize and Deserialize bit slices and test EQ GT GTE ISNOTNULL queries
  2. Random data test: generate random data and test EQ GT GET ISNOTNULL queries
  3. Edge cases: emtpy bitslice, bitslice with only one 0, bitslice with Interger MAX 2147483647 as code.
Filename Sort by name	Line Coverage Sort by line coverage	Functions Sort by function coverage
[bit_slice_index_bitmap.cpp](http://172.16.25.167:8000/coverage/common/file_index/rangebitmap/bit_slice_index_bitmap.cpp.gcov.html)	
98.9%98.9%
98.9 %	175 / 177	100.0 %	14 / 14
[bit_slice_index_bitmap_test.cpp](http://172.16.25.167:8000/coverage/common/file_index/rangebitmap/bit_slice_index_bitmap_test.cpp.gcov.html)	
100.0%
100.0 %	124 / 124	100.0 %	31 / 31

@fafacao86
Copy link
Contributor Author

/home/runner/work/paimon-cpp/paimon-cpp/src/paimon/common/metrics/histogram_windowing_test.cpp:110: Failure
Expected equality of these values:
s1.count
Which is: 1
100
/home/runner/work/paimon-cpp/paimon-cpp/src/paimon/common/metrics/histogram_windowing_test.cpp:111: Failure
Expected equality of these values:
s1.min
Which is: 2
1
[ FAILED ] HistogramWindowingImplTest.TestMinNumPerWindow100Case (12 ms)
[ RUN ] HistogramWindowingImplTest.TestLargeDatasetInSingleWindow .
Hi @lxy-9602 @SGZW Is this a flaky test? 2ms a window might be too small? I guess.

@SGZW
Copy link
Contributor

SGZW commented Mar 11, 2026

/home/runner/work/paimon-cpp/paimon-cpp/src/paimon/common/metrics/histogram_windowing_test.cpp:110: Failure Expected equality of these values: s1.count Which is: 1 100 /home/runner/work/paimon-cpp/paimon-cpp/src/paimon/common/metrics/histogram_windowing_test.cpp:111: Failure Expected equality of these values: s1.min Which is: 2 1 [ FAILED ] HistogramWindowingImplTest.TestMinNumPerWindow100Case (12 ms) [ RUN ] HistogramWindowingImplTest.TestLargeDatasetInSingleWindow . Hi @lxy-9602 @SGZW Is this a flaky test? 2ms a window might be too small? I guess.

It may indeed cause instability. If convenient, you can directly increase the time period, which will make it more stable, or wait for me to make the modifications.

@fafacao86
Copy link
Contributor Author

It may indeed cause instability. If convenient, you can directly increase the time period, which will make it more stable, or wait for me to make the modifications.

Thanks for the reply. I can wait for your modification—you're more familiar with the underlying logic.

@lucasfang lucasfang requested a review from Copilot March 12, 2026 01:49
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a Bit-Sliced Index (BSI) bitmap implementation under rangebitmap, enabling EQ/GT/GTE/ISNOTNULL queries with serialization/deserialization support, plus unit tests.

Changes:

  • Introduced BitSliceIndexBitmap with on-demand slice loading and query operations.
  • Added unit tests covering functional behavior, random data validation, and edge cases.
  • Wired the new implementation and tests into the build via CMake.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/paimon/common/file_index/rangebitmap/bit_slice_index_bitmap.h Declares the BitSliceIndexBitmap API and Appender serialization interface.
src/paimon/common/file_index/rangebitmap/bit_slice_index_bitmap.cpp Implements serialization format, slice loading, and query evaluation logic.
src/paimon/common/file_index/rangebitmap/bit_slice_index_bitmap_test.cpp Adds UT coverage for equality/range/nullability queries and edge cases.
src/paimon/common/file_index/CMakeLists.txt Adds the new .cpp to the file index library sources.
src/paimon/CMakeLists.txt Registers the new test file in the test build.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@fafacao86 fafacao86 force-pushed the rangebitmap-bitslice branch from 18dbee5 to 5bf5cb3 Compare March 13, 2026 13:39
Copy link
Collaborator

@lxy-9602 lxy-9602 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602 lxy-9602 merged commit 2b2c05f into alibaba:main Mar 14, 2026
8 checks passed
@lxy-9602
Copy link
Collaborator

Nice work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support RangeBitmap File Index

4 participants