Bug Report
Describe the bug
Running databricks bundle deployment migrate (with or without --noplancheck) fails when a DLT pipeline resource has Spark configuration keys containing dots (e.g. europris.swipe.egress_streaming_schema).
The Direct engine's path resolver treats dotted key names as nested map traversal paths instead of flat string keys. This causes the planning step inside CalculatePlan to fail with:
Error: cannot plan resources.pipelines.receipt_stream: cannot update configuration.europris.swipe.egress_streaming_schema with value of "${resources.schemas.egress_streaming_schema.catalog_name}...": cannot update configuration.europris.swipe.egress_streaming_schema to string: failed to navigate to parent configuration.europris.swipe: configuration.europris: key "europris" not found in map
The same config deploys correctly with the Terraform engine.
Steps to reproduce
- Define a DLT pipeline resource with dotted keys in the
configuration block:
resources:
pipelines:
my_pipeline:
configuration:
europris.swipe.egress_streaming_schema: "${resources.schemas.my_schema.catalog_name}.${resources.schemas.my_schema.name}"
europris.azure.subscription: ${var.AZURE.subscription}
- Deploy with the Terraform engine (
databricks bundle deploy)
- Run
databricks bundle deployment migrate
Expected behavior
Migration succeeds. Dotted keys in the DLT configuration map are treated as opaque flat strings, not as nested path segments.
Actual behavior
CalculatePlan attempts to traverse the dotted key as a nested map path, fails to find the intermediate node (e.g. europris), and aborts with key not found in map.
Using --noplancheck does not help — the pre-check subprocess is skipped, but the same CalculatePlan call inside migrate.go still hits the bug.
CLI version
Additional context
The failure path is in migrate.go → deploymentBundle.CalculatePlan → Direct engine path resolution via structaccess.Set / structpath. The Terraform engine is not affected because it serializes pipeline configuration differently.
Bug Report
Describe the bug
Running
databricks bundle deployment migrate(with or without--noplancheck) fails when a DLT pipeline resource has Spark configuration keys containing dots (e.g.europris.swipe.egress_streaming_schema).The Direct engine's path resolver treats dotted key names as nested map traversal paths instead of flat string keys. This causes the planning step inside
CalculatePlanto fail with:The same config deploys correctly with the Terraform engine.
Steps to reproduce
configurationblock:databricks bundle deploy)databricks bundle deployment migrateExpected behavior
Migration succeeds. Dotted keys in the DLT
configurationmap are treated as opaque flat strings, not as nested path segments.Actual behavior
CalculatePlanattempts to traverse the dotted key as a nested map path, fails to find the intermediate node (e.g.europris), and aborts withkey not found in map.Using
--noplancheckdoes not help — the pre-check subprocess is skipped, but the sameCalculatePlancall insidemigrate.gostill hits the bug.CLI version
Additional context
The failure path is in
migrate.go→deploymentBundle.CalculatePlan→ Direct engine path resolution viastructaccess.Set/structpath. The Terraform engine is not affected because it serializes pipeline configuration differently.