Skip to content

fix(clickhouse): strip virtual catalogs from executed queries and inserts - #6036

Open
mday-io wants to merge 1 commit into
SQLMesh:mainfrom
mday-io:mday/fix-clickhouse-insert-virtual-catalog
Open

fix(clickhouse): strip virtual catalogs from executed queries and inserts#6036
mday-io wants to merge 1 commit into
SQLMesh:mainfrom
mday-io:mday/fix-clickhouse-insert-virtual-catalog

Conversation

@mday-io

@mday-io mday-io commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Why this is necessary: SQLMesh uses a synthetic catalog to identify a gateway, but ClickHouse expects database.table names. Direct queries, including audit queries and INSERT post-statements used for initial history loads, bypass create_view normalization. Without this fix, otherwise valid SQL reaches ClickHouse with unsupported catalog-qualified table and column names and can fail during execution. Fixing the adapter avoids requiring each project to add its own rendering workaround.
  • Strip the configured ClickHouse virtual catalog from table and qualified-column references when executing SELECT or INSERT expressions directly.
  • For example, INSERT INTO __gw__.db.target SELECT ... FROM __gw__.db.source now reaches ClickHouse using db.target and db.source. Direct execution bypasses the view-creation handling added in fix(clickhouse): strip virtual catalog from view sources #5940; fix(clickhouse): initialize virtual catalog for cleanup #5941 addresses cleanup initialization separately.
  • Preserve the caller’s expression, unrelated catalogs, and literal values. Existing create-view catalog validation remains intact.

Test Plan

  • Reproduced failing INSERT and SELECT regressions before the fix; all 38 ClickHouse adapter tests pass after it.
  • Covered target/source names, qualified columns, configured catalog fallback, expression immutability, and preservation of unrelated catalogs/literals.
  • make style passes: Ruff, formatting, mypy, and migration checks.
  • make fast-test passes: 2,625 tests passed and four skipped; all isolated, registry-isolation, and dialect-isolation suites passed.
  • git diff --check passes.

…erts

Signed-off-by: mday-io <mdaytn@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant