Skip to content

fix(duckdb): do not cascade table/view drops in DuckLake catalogs - #6034

Open
kingjaiteh wants to merge 1 commit into
SQLMesh:mainfrom
kingjaiteh:fix/6032-ducklake-drop-cascade
Open

fix(duckdb): do not cascade table/view drops in DuckLake catalogs#6034
kingjaiteh wants to merge 1 commit into
SQLMesh:mainfrom
kingjaiteh:fix/6032-ducklake-drop-cascade

Conversation

@kingjaiteh

Copy link
Copy Markdown

DuckLake does not implement DROP TABLE ... CASCADE or DROP VIEW ... CASCADE and raises "Cascade Drop not supported in DuckLake". Since #5133 the janitor always deletes snapshot tables with cascade=True, so on a DuckLake catalog it can never reclaim space and silently reports "Cleanup complete." until a snapshot expires.

Cascade support is declared per engine adapter (SUPPORTED_DROP_CASCADE_OBJECT_KINDS), but on DuckDB it varies per attached catalog: a native catalog accepts DROP TABLE ... CASCADE, a DuckLake catalog attached in the same connection does not.

Changes:

  • DuckDBEngineAdapter._drop_object resolves the target catalog (explicit catalog on the name, else the current one) and looks up its type in duckdb_databases(). For ducklake it drops TABLE/VIEW without CASCADE. SCHEMA drops are untouched, DuckLake supports cascade there.
  • The lookup is factored into _get_catalog_type, which _create_table already did inline for the DuckLake partitioning check, so both share it.
  • Native DuckDB catalogs behave exactly as before, so users running with enable_view_dependencies keep cascade.

The list-only alternative (removing TABLE/VIEW from the DuckDB cascade list) would also fix DuckLake but would regress enable_view_dependencies on native catalogs, which is what #4767 added it for.

Test Plan

  • test_drop_table_ducklake_no_cascade attaches a real DuckLake catalog (same setup as test_ducklake_partitioning), creates a table with a dependent view in it and a native table/view in memory, and drops all four with cascade=True. Fails on main with the NotImplementedException above.
  • test_drop_object_cascade_by_catalog_type asserts the emitted SQL with a mocked adapter: no CASCADE for DuckLake table/view, CASCADE kept for DuckLake schema and for a native catalog table.
  • tests/core/engine_adapter/test_duckdb.py and tests/core/test_snapshot_evaluator.py pass on DuckDB 1.5.5, Python 3.12, Windows.
  • The issue reporter has offered to run the branch against their DuckLake project.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

DuckLake does not implement `DROP TABLE ... CASCADE` or `DROP VIEW ... CASCADE`
and raises "Cascade Drop not supported in DuckLake". Since SQLMesh#5133 the janitor
always deletes snapshot tables with cascade=True, so on a DuckLake catalog it
can never reclaim space and silently reports "Cleanup complete." until a
snapshot expires.

Cascade support is declared per engine adapter, but on DuckDB it varies per
attached catalog. Resolve the target catalog's type from duckdb_databases()
and omit CASCADE for TABLE/VIEW drops in DuckLake catalogs. SCHEMA cascade is
still supported by DuckLake and is left untouched, as is behaviour for native
DuckDB catalogs.

Fixes SQLMesh#6032

Signed-off-by: kingjaiteh <omarjaiteh453@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant