Describe the issue
Model Serving Endpoints can currently be defined in conventional Databricks Asset Bundles (YAML-based), but cannot be defined or managed using Python bundles (pyDABs). It is desirable to manage Mosaic AI Gateway & External Model endpoint configurations programmatically, as YAML-only configurations tend to get quite verbose.
The OpenAPI schema for model serving isn't immediately compatible for databricks-bundles packages' codegen because:
- Enum values containing dashes (e.g.
amazon-bedrock) don't presently resolve to valid Python variables.
- Related model serving types being split across adjacent namespaces (model_serving_endpoints and serving), making the Python API inconsistent with existing usage patterns.
Configuration
This, or a derivative thereof (e.g. resources.add_model_serving_endpoint) are desired:
from databricks.bundles.model_serving_endpoints import ModelServingEndpoint
endpoint = ModelServingEndpoint(
name="my-endpoint",
config={
"served_models": [...],
"endpoint_tags": [...],
"provider": "openai",
},
)Desired Behavior
- Model Serving Endpoints can be defined and managed in Python bundles
- Enums with dashes (e.g. amazon-bedrock) resolve correctly in Python
- Model serving–related types (e.g. EndpointTag) are accessible from a consistent namespace aligned with the existing API
- Users can configure Model Serving Endpoints programmatically (e.g. using OmegaConf) across environments
- Ability to manage Databricks Hosted LLM endpoints in DAB, e.g. AI Gateway config of Databricks GPT 5.2
Actual Behavior
- Model Serving Endpoints are only supported via YAML in conventional DABs
- Python bundles lack a ModelServingEndpoint resource
- Enum values with dashes cause issues in existing Python codegen if model serving namespaces are naively added
- Model serving types are fragmented across namespaces, making the API awkward to use without changes
Proposed solution (high level)
- Add a ModelServingEndpoint resource to Python bundles
- Update Python codegen to handle enum values containing dashes
- Provide cross-namespace re-exports for model serving types so related APIs are accessible under a consistent namespace
- Align Python bundle support with existing YAML-based DAB functionality
- Enable limited attribute management of Databricks Hosted LLM endpoints via DABs/PyDABs
Context
I initially submitted the below solution as a PR, and on request am redirecting to Issue. I currently manage Microsoft AI Foundry serving endpoints in our Azure Databricks workspaces using this implementation:
Describe the issue
Model Serving Endpoints can currently be defined in conventional Databricks Asset Bundles (YAML-based), but cannot be defined or managed using Python bundles (pyDABs). It is desirable to manage Mosaic AI Gateway & External Model endpoint configurations programmatically, as YAML-only configurations tend to get quite verbose.
The OpenAPI schema for model serving isn't immediately compatible for
databricks-bundlespackages' codegen because:amazon-bedrock) don't presently resolve to valid Python variables.Configuration
This, or a derivative thereof (e.g.
resources.add_model_serving_endpoint) are desired:Desired Behavior
Actual Behavior
Proposed solution (high level)
Context
I initially submitted the below solution as a PR, and on request am redirecting to Issue. I currently manage Microsoft AI Foundry serving endpoints in our Azure Databricks workspaces using this implementation:
databricks-bundlespackage #4340