@@ -26,11 +26,11 @@ helpful to have a roadmap for some of the major items that require coordination
2626
2727### Iceberg Integration
2828
29- Iceberg integration is still a work-in-progress ( [ # 2060 ] ), with major improvements expected in the next few
30- releases. The default ` auto ` scan mode now uses ` native_iceberg_compat ` instead of ` native_comet ` , enabling
31- support for complex types .
29+ Iceberg tables reads are now fully native, powered by a scan operator backed by Iceberg-rust ( [ # 2528 ] ). We anticipate
30+ major improvements expected in the next few releases, including bringing Iceberg table format V3 features ( _ e.g. _ ,
31+ encryption) to the reader .
3232
33- [ #2060 ] : https://github.com/apache/datafusion-comet/issues/2060
33+ [ #2528 ] : https://github.com/apache/datafusion-comet/pull/2528
3434
3535### Spark 4.0 Support
3636
@@ -40,6 +40,17 @@ more Spark SQL tests and fully implementing ANSI support ([#313]) for all suppor
4040[ #313 ] : https://github.com/apache/datafusion-comet/issues/313
4141[ #1637 ] : https://github.com/apache/datafusion-comet/issues/1637
4242
43+ ### Dynamic Partition Pruning
44+
45+ Iceberg table scans support Dynamic Partition Pruning (DPP) filters generated by Spark's ` PlanDynamicPruningFilters `
46+ optimizer rule ([ #3349 ] ). However, we still need to bring this functionality to our Parquet reader. Furthermore,
47+ Spark's ` PlanAdaptiveDynamicPruningFilters ` optimizer rule runs after Comet's rules, so DPP with Adaptive Query
48+ Execution requires a redesign of Comet's plan translation. We are focused on implementing DPP to keep Comet competitive
49+ with benchmarks that benefit from this feature like TPC-DS. This effort can be tracked at [ #3510 ] .
50+
51+ [ #3349 ] : https://github.com/apache/datafusion-comet/pull/3349
52+ [ #3510 ] : https://github.com/apache/datafusion-comet/issues/3510
53+
4354### Removing the native_comet scan implementation
4455
4556The ` native_comet ` scan implementation is now deprecated and will be removed in a future release ([ #2186 ] , [ #2177 ] ).
0 commit comments