Skip to content

feat: add flexible dependency source resolution#259

Open
suxiaogang223 wants to merge 14 commits intoalibaba:mainfrom
suxiaogang223:codex/issue-103-dependency-source
Open

feat: add flexible dependency source resolution#259
suxiaogang223 wants to merge 14 commits intoalibaba:mainfrom
suxiaogang223:codex/issue-103-dependency-source

Conversation

@suxiaogang223
Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #103

This PR introduces a flexible dependency source resolution interface for selected third-party dependencies.

It adds:

  • PAIMON_DEPENDENCY_SOURCE=AUTO|BUNDLED|SYSTEM
  • per-dependency *_SOURCE overrides
  • PAIMON_PACKAGE_PREFIX
  • compatibility FindXxxAlt.cmake modules
  • a centralized dependency resolution summary
  • target-based include propagation cleanup for ORC/Avro format targets

Package-manager-specific modes are intentionally left out of this first PR and should be discussed separately.

Tests

Fedora:

  • cmake -S . -B build-scope-bundled -DPAIMON_DEPENDENCY_SOURCE=BUNDLED
  • cmake --build build-scope-bundled -j$(nproc)
  • cmake -S . -B build-scope-auto -DPAIMON_DEPENDENCY_SOURCE=AUTO
  • cmake --build build-scope-auto -j$(nproc)
  • cmake -S . -B build-scope-tests -DPAIMON_DEPENDENCY_SOURCE=BUNDLED -DPAIMON_BUILD_TESTS=ON
  • cmake --build build-scope-tests --target paimon-tests -j$(nproc)

Also verified invalid dependency source values are rejected by configure-time validation.

API and Format

No storage format or public C++ API changes.

Documentation

Updated README and building docs for dependency source configuration.

Generative AI tooling

Generated-by: OpenAI Codex

@suxiaogang223 suxiaogang223 force-pushed the codex/issue-103-dependency-source branch from 2b78054 to bfd2720 Compare April 30, 2026 07:23
@suxiaogang223
Copy link
Copy Markdown
Contributor Author

@zjw1111 Hi, could you help review this PR when you have time?

This PR now focuses on flexible dependency source resolution and the sanitizer CI issue has been fixed in the latest commit. The remaining follow-up work and broader design discussion will be tracked in #260, so we can keep this PR scoped and easier to review.

Thanks.🤓

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a flexible CMake dependency-resolution layer so paimon-cpp can use either bundled third-party builds or system-installed packages, with a global default (PAIMON_DEPENDENCY_SOURCE=AUTO|BUNDLED|SYSTEM) and per-dependency overrides, plus a configure-time resolution summary.

Changes:

  • Introduces resolve_dependency() and related helper functions to select between bundled builds and system find_package() lookups, and prints a dependency resolution summary during configure.
  • Adds FindXxxAlt.cmake compatibility modules to normalize targets/variables across system package variations.
  • Cleans up ORC/Avro format targets to rely on target-based include propagation (removing manual include dirs) and adjusts test link inputs.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/paimon/format/orc/CMakeLists.txt Removes manual ORC include injection; updates ORC test linking to explicitly link orc::orc.
src/paimon/format/avro/CMakeLists.txt Removes manual Avro include injection; updates Avro test linking to explicitly link avro.
README.md Documents the new dependency source configuration knobs and recommended usage.
docs/source/building.rst Adds build-docs section for dependency source configuration and summary behavior.
CMakeLists.txt Switches GTest provisioning to resolve_dependency(GTest) and prints dependency resolution summary.
cmake_modules/ThirdpartyToolchain.cmake Implements the dependency source resolution framework, defaults propagation, and integrates it into third-party setup.
cmake_modules/BuildUtils.cmake Improves include propagation by linking external deps to object libraries; marks internal libs to avoid unintended propagation.
cmake_modules/DefineOptions.cmake Adds CMake cache options for global/per-dependency source selection and shared-library preference.
cmake_modules/FindArrowAlt.cmake Provides a compatibility finder for Arrow/Parquet/Dataset/Acero targets.
cmake_modules/FindAvroAlt.cmake Provides a compatibility finder for Avro (avro-cpp) with a normalized avro target.
cmake_modules/FindfmtAlt.cmake Provides a compatibility finder for fmt with a normalized fmt target.
cmake_modules/FindglogAlt.cmake Provides a compatibility finder for glog with a normalized glog target.
cmake_modules/FindGTestAlt.cmake Provides a compatibility finder for GTest/GMock and sets GTEST_LINK_TOOLCHAIN.
cmake_modules/FindLZ4Alt.cmake Provides a compatibility finder for LZ4 with a normalized lz4 target.
cmake_modules/FindORCAlt.cmake Provides a compatibility finder for ORC with a normalized orc::orc target.
cmake_modules/FindProtobufAlt.cmake Provides a compatibility finder for Protobuf and normalized libprotobuf/libprotoc/protoc.
cmake_modules/FindRapidJSONAlt.cmake Provides a compatibility finder for RapidJSON with a normalized RapidJSON target.
cmake_modules/FindRE2Alt.cmake Provides a compatibility finder for RE2 with re2::re2 support.
cmake_modules/FindSnappyAlt.cmake Provides a compatibility finder for Snappy with a normalized snappy target.
cmake_modules/FindTBBAlt.cmake Provides a compatibility finder for TBB with a normalized tbb target.
cmake_modules/FindZLIBAlt.cmake Provides a compatibility finder for zlib with a normalized zlib target.
cmake_modules/FindzstdAlt.cmake Provides a compatibility finder for zstd with a normalized zstd target.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmake_modules/ThirdpartyToolchain.cmake Outdated
Comment thread cmake_modules/ThirdpartyToolchain.cmake
Comment thread cmake_modules/FindAvroAlt.cmake
Comment thread cmake_modules/ThirdpartyToolchain.cmake Outdated
Comment thread cmake_modules/FindArrowAlt.cmake Outdated
Comment thread cmake_modules/FindArrowAlt.cmake
@suxiaogang223 suxiaogang223 force-pushed the codex/issue-103-dependency-source branch from 54be18d to e23df80 Compare May 9, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Flexible Third-Party Dependency Management System

5 participants