Skip to content

optimize(interleave): implement interleave for FixedSizeList/Map type#10046

Open
mapleFU wants to merge 2 commits into
apache:mainfrom
mapleFU:interleave-fsl-map
Open

optimize(interleave): implement interleave for FixedSizeList/Map type#10046
mapleFU wants to merge 2 commits into
apache:mainfrom
mapleFU:interleave-fsl-map

Conversation

@mapleFU
Copy link
Copy Markdown
Member

@mapleFU mapleFU commented Jun 2, 2026

Which issue does this PR close?

Rationale for this change

Implement interleave for Map/FSL type.

Note: This might a slow implementation for FSL. For map it's a struct type, and at would have two childs. So even if it's Map<i32, i32>, it wouldn't be slower than before.

For FSL, if it's fixed sized list with short i32 list, it may hit the problem in #10025 , making them use same code to handle list is better.

What changes are included in this PR?

Implement interleave for Map/FSL type.

Are these changes tested?

  1. Test covered by existing
  2. benchmark is provided

Are there any user-facing changes?

No

@github-actions github-actions Bot added the arrow Changes to the arrow crate label Jun 2, 2026
Ok(Arc::new(list_array))
}

fn interleave_fixed_size_list(
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might reusing an optimization with #10025 . Otherwise it might get slower in some case

("list_view<i64>(0.0,0.0,20)", &list_view_i64_no_nulls),
("list_view_overlapping<i64>(80x,20)", &list_view_overlapping),
("fixed_size_list<i64,5>(0.0,0.0)", &fsl_i64),
("fixed_size_list<i64,5>(0.1,0.1)", &fsl_i64_nulls),
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I can also support large fsl, like vector<1024, f32>

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know should I extract benchmark to a separate pr ( containing this and concat benchmark?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: native interleave for fsl/map type

1 participant