Skip to content

Comments

spike: port adapter from pyodbc/ODBC to ADBC#633

Draft
dataders wants to merge 2 commits intomasterfrom
spike/adbc-dbc
Draft

spike: port adapter from pyodbc/ODBC to ADBC#633
dataders wants to merge 2 commits intomasterfrom
spike/adbc-dbc

Conversation

@dataders
Copy link
Collaborator

Summary

Spike to replace the entire pyodbc/msodbcsql18 stack with ADBC (Arrow Database Connectivity) using the mssql driver from dbc, and remove the dbt-fabric dependency entirely to make dbt-sqlserver a standalone adapter.

Why: ODBC driver installation is the #1 pain point for dbt-sqlserver users across platforms. ADBC's mssql driver is a single binary installed via dbc install mssql — no ODBC driver manager, no msodbcsql18, no platform-specific setup.

Scope: SQL auth only, local Docker SQL Server, proving the adapter works end-to-end.

Changes

Python adapter (remove dbt-fabric dependency)

  • setup.py — Remove dbt-fabric==1.9.6, add adbc-driver-manager>=1.9.0 + pyarrow>=20.0.0
  • __init__.py — Remove dependencies=["fabric"] from AdapterPlugin
  • sqlserver_credentials.py — Inherit from Credentials (was FabricCredentials); add build_adbc_uri() for go-mssqldb connection URIs
  • sqlserver_connections.py — Inherit from SQLConnectionManager (was FabricConnectionManager); connect via adbc_dbapi.connect(driver='mssql', ...); add data_type_code_to_name() mapping Arrow type codes → SQL Server types
  • sqlserver_adapter.py — Inherit from SQLAdapter (was FabricAdapter); inline Fabric's type conversion and SQL helper methods
  • sqlserver_column.py — Inherit from Column (was FabricColumn); inline type labels and methods
  • sqlserver_configs.py — Inherit from AdapterConfig (was FabricConfigs)

Macros (inline fabric macros with sqlserver__ prefix)

Since we removed dependencies=["fabric"], fabric macros are no longer in the dispatch chain. All ~25 fabric macros that sqlserver depended on were copied in with sqlserver__ prefix:

  • Metadatalist_schemas, check_schema_exists, list_relations_without_caching, get_relation_without_caching, get_relation_last_modified
  • Relationmake_temp_relation, get_drop_sql, rename_relation
  • Schemacreate_schema, drop_schema
  • Columnsget_columns_in_relation, alter_relation_add_remove_columns
  • Grantsapply_grants, get_show_grant_sql, get_grant_sql, get_revoke_sql
  • Showget_limit_sql (OFFSET/FETCH)
  • Incrementalget_merge_sql, get_delete_insert_merge_sql, get_incremental_default_sql
  • Snapshotpost_snapshot, get_true_sql, snapshot_hash_arguments, build_snapshot_table, snapshot_staging_table
  • DDLbuild_columns_constraints, build_model_constraints
  • Utils (14 new files)any_value, array_construct, cast_bool_to_text, concat, date_trunc, dateadd, get_tables_by_pattern, hash, last_day, length, listagg, position, safe_cast, timestamps

Seed loading fix

ADBC's mssql driver does not support parameterized queries with ? placeholders. Rewrote sqlserver__load_csv_rows to inline literal values directly into INSERT statements, with proper handling for NULL, booleans (is sameas true/false), numbers, and strings (with quote escaping).

Unit test macro fix

Updated unit_test_table.sql to pass column_name_to_quoted to get_expected_sql() (required by newer dbt-core API).

Test Results

Suite Result
Unit tests 10/10 passed
dbt debug 10/10 passed
Functional tests 123 passed, 8 failed, 45 skipped, 1 xfailed

8 remaining failures (all known ADBC behavioral differences)

Test Root Cause Fixable?
4 grant tests Need a test user/login in the database — not an ADBC issue Yes, test setup
3 query comment tests ADBC log format differs from pyodbc; tests match log output Yes, needs investigation
1 timestamp naive test ADBC returns Arrow timestamps with UTC tz; pyodbc returned naive datetimes Yes, override test expectation

Known limitations (acceptable for spike)

  1. SQL auth only — Azure AD/MSI/SP auth not implemented (go-mssqldb supports ?fedauth= for future work)
  2. No parameterized queries — Seeds inline values; this may have SQL injection implications for seed data with malicious content (acceptable for trusted seed CSVs)
  3. Query timeout — No direct ADBC equivalent for pyodbc's handle.timeout
  4. Timestamp timezone — ADBC returns all timestamps as UTC-aware; may affect snapshot comparisons in edge cases

How to test locally

# Install dbc and ADBC mssql driver
uv pip install dbc
dbc install mssql
uv pip install -e .

# Start SQL Server (Azure SQL Edge for ARM Macs)
docker run -d --name sqlserver-test \
  -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=YourPassword1 \
  -p 1433:1433 mcr.microsoft.com/azure-sql-edge:latest

# Run tests
export SQLSERVER_TEST_HOST=127.0.0.1 SQLSERVER_TEST_USER=sa \
  SQLSERVER_TEST_PASS=YourPassword1 SQLSERVER_TEST_DBNAME=TestDB \
  SQLSERVER_TEST_ENCRYPT=True SQLSERVER_TEST_TRUST_CERT=True
pytest tests/unit -v
pytest tests/functional -v --profile user

🤖 Generated with Claude Code

dataders and others added 2 commits February 16, 2026 13:12
Replace the entire pyodbc/msodbcsql18 stack with ADBC (Arrow Database
Connectivity) using the mssql driver from dbc, and remove the dbt-fabric
dependency to make dbt-sqlserver a standalone adapter.

Key changes:
- Remove dbt-fabric dependency; inherit from dbt-adapters base classes
- Rewrite connection layer to use adbc-driver-manager dbapi
- Build go-mssqldb connection URIs from profile credentials
- Inline ~25 fabric macros with sqlserver__ prefix
- Rewrite seed loading to inline literal values (ADBC doesn't support
  parameterized queries with ? placeholders)
- Add Arrow type code -> SQL Server type name mapping
- Update unit tests for ADBC/URI-based connection model

Test results: 123 passed, 8 failed, 45 skipped (10/10 unit tests pass)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant