Skip to content

feat: SDK-native column constraints (create, update, typed read) — Layer 1 of cutting raw-urllib usage - #206

Open
arorashivam96 wants to merge 6 commits into
mainfrom
users/shivamarora/rich-column-spec
Open

arorashivam96 wants to merge 6 commits into
mainfrom
users/shivamarora/rich-column-spec

Conversation

@arorashivam96

@arorashivam96 arorashivam96 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

SDK-native column constraints — Layer 1 of reducing raw-urllib usage

The problem this fixes

In the Dataverse plugin's telemetry, a large share of traffic is raw urllib (hand-rolled HTTP) instead of the managed SDK. The single biggest driver is schema metadata on the dv-metadata skill: agents were forced to hand-build calls to EntityDefinitions/Attributes because the SDK could not express column constraints. Concretely, three operations had no SDK path:

  1. Create a column with constraints (MaxLength, MinValue/MaxValue, Format, RequiredLevel, a custom DisplayName).
  2. Update an existing column's constraints (requires the counter-intuitive PUT-not-PATCH plus a derived @odata.type discriminator).
  3. Read type-specific fields back (needs a second request with an @odata.type URL cast).

So for a routine ask like "add a Rating 1–5 and a 2000-char Comment", the agent dropped to raw urllib, hit the metadata quirks, and looped. This PR gives all three an SDK path, so agents stay on the managed client (which carries auth, retries, and telemetry attribution) and never hand-roll the metadata Web API.

Scope: this is Layer 1 (SDK / product fixes). Layer 2 — fixing the skills (updating dv-metadata to teach the new SDK spec and stop describing the raw Web-API workaround) — is a separate follow-up in the plugin repo and depends on this PR shipping.

What's in this PR

Four changes, all backward-compatible, sync + async parity.

1. Create with constraints (closes #194)

tables.create / add_columns now accept a dict column spec alongside the existing str / Enum:

client.tables.create("cfb_CustomerFeedback", {
    "cfb_Name":    "string",                                   # unchanged
    "cfb_Rating":  {"type": "int",  "min_value": 1, "max_value": 5},
    "cfb_Comment": {"type": "memo", "max_length": 2000, "display_name": "Comment"},
})

Override keys: max_length, min_value, max_value, precision, format, required, display_name.

2. Update an existing column (closes #202)

New tables.update_column / update_columns. The SDK reads the typed attribute, applies the overrides, and issues the PUT with the right @odata.type internally — the caller never sees a verb or a discriminator:

client.tables.update_column("cfb_CustomerFeedback", "cfb_Comment",
                            {"max_length": 4000, "display_name": "Customer Comment"})

3. Typed reads in one request (closes #203)

typed=True on list_columns / get_column, backed by RetrieveMetadataChanges with property projection — type-specific fields come back in a single request, no @odata.type URL cast:

col  = client.tables.get_column("account", "emailaddress1", typed=True)   # col["MaxLength"], etc.
cols = client.tables.list_columns("account", typed=True)

4. Attribution allowlist

Add dv-overview + erp-xpp to OperationContext._ALLOWED_SKILLS (parity with the plugin's scripts/auth.py, which allows all 10). Previously get_client('dv-overview'|'erp-xpp') raised ValueError on the SDK side, and the plugin silently fell back to raw urllib for those two skills.

Compatibility

str / Enum column specs and every existing signature are unchanged; the only new parameter is a keyword-only typed=False on list_columns. No CHANGELOG / version edit (per CONTRIBUTING — maintainers handle those at release).

Testing

Unit — full suite green: 2437 passed (sync + async). New coverage: dict-spec overrides, update_column (PUT verb + @odata.type + override application + error paths), RetrieveMetadataChanges typed reads (request shape, projection, criteria), get_column(typed=True), and allowlist parity.

Live end-to-end against a Dataverse org (*.crm10.dynamics.com) — 12/12 checks pass:

Check Result
#194 create table w/ bounded int (1–5) + 2000-char renamed memo PASS
#203 typed read → MinValue=1, MaxValue=5, MaxLength=2000 (one request) PASS
#203 list_columns(typed=True) returns typed fields PASS
#202 update_column memo 2000 → 4000, re-read confirms PASS
core CRUD (create / retrieve / update / list / delete) — no regression PASS

For contrast, the unmodified SDK raises ValueError: Unsupported column spec type ... <class 'dict'> on the very same create — the exact gap this PR closes.

…reate/add_columns

tables.create/add_columns previously accepted only a type string (or Enum) per column, so MaxLength, MinValue/MaxValue, Precision, Format, RequiredLevel, and a custom DisplayName could not be set - callers had to drop to the raw EntityDefinitions/Attributes Web API. A column-spec value may now also be a dict pairing a base type with overrides, e.g. {'type': 'int', 'min_value': 1, 'max_value': 5} or {'type': 'memo', 'max_length': 2000, 'display_name': 'Comment'}. str/Enum specs are unchanged (backward-compatible).

_attribute_payload now extracts overrides from a dict spec, builds the base payload via _base_attribute_payload, then applies the overrides (string/memo format -> FormatName, int/date format -> Format). Unknown keys and a missing 'type' raise ValueError. Adds 10 unit tests; sync+async suites green. Implements #194.
…mn, typed reads + allowlist

Folds the full metadata-constraint story into one change so agents never hand-roll the EntityDefinitions/Attributes Web API (the dominant driver of raw-urllib usage in the plugin telemetry):

#194 create with constraints: tables.create/add_columns accept a dict column spec {'type','max_length','min_value','max_value','precision','format','required','display_name'}; str/Enum unchanged. #202 tables.update_column/update_columns: update existing column constraints (GET typed attr -> PUT + derived @odata.type, hidden from the caller). #203 typed reads: tables.list_columns(typed=True) and tables.get_column(..., typed=True) via RetrieveMetadataChanges with property projection - one request, no @odata.type URL cast. Allowlist: add dv-overview + erp-xpp to OperationContext._ALLOWED_SKILLS (parity with the plugin auth.py). Sync + async parity; full unit suite green.
@arorashivam96
arorashivam96 requested a review from a team as a code owner September 30, 2026 22:43
Copilot AI balanced review requested due to automatic review settings September 30, 2026 22:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Typed reads silently ignore filters, missing-column errors use the wrong subcode, and async coverage and public documentation are incomplete.

Review effort: Balanced
Findings: 2 Medium severity · 2 Low severity

Open (4)
What changed in this PR

Adds SDK-native column constraints, typed metadata reads, column updates, and operation-context allowlist parity.

Changes:

  • Supports constraint dictionaries when creating columns.
  • Adds typed column reads and constraint updates for sync/async clients.
  • Allows dv-overview and erp-xpp operation contexts.
File Description
tests/​unit/​test_tables_operations.py Updates sync delegation expectations.
tests/​unit/​test_operation_context.py Tests added skill identifiers.
tests/​unit/​data/​test_odata_internal.py Tests constraints, updates, and typed reads.
tests/​unit/​aio/​test_async_tables.py Updates async delegation expectations.
src/​PowerPlatform/​Dataverse/​operations/​tables.py Exposes new sync table APIs.
src/​PowerPlatform/​Dataverse/​data/​_odata.py Implements sync typed reads and updates.
src/​PowerPlatform/​Dataverse/​data/​_odata_base.py Adds shared constraint handling.
src/​PowerPlatform/​Dataverse/​core/​config.py Expands the skill allowlist.
src/​PowerPlatform/​Dataverse/​aio/​operations/​async_tables.py Exposes async table APIs.
src/​PowerPlatform/​Dataverse/​aio/​data/​_async_odata.py Implements async typed reads and updates.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/PowerPlatform/Dataverse/aio/data/_async_odata.py
Comment thread src/PowerPlatform/Dataverse/data/_odata.py
Comment thread src/PowerPlatform/Dataverse/data/_odata_base.py
Comment thread src/PowerPlatform/Dataverse/operations/tables.py
@arorashivam96 arorashivam96 changed the title feat: SDK-native column constraints (create/update/typed reads) + operation-context allowlist feat: SDK-native column constraints (create, update, typed read) — Layer 1 of cutting raw-urllib usage Sep 30, 2026
Comment thread src/PowerPlatform/Dataverse/data/_odata.py
Comment thread src/PowerPlatform/Dataverse/data/_odata.py Outdated
Comment thread src/PowerPlatform/Dataverse/data/_odata.py
Comment thread src/PowerPlatform/Dataverse/operations/tables.py
… full-definition metadata PUT + MergeLabels, pre-validate update_columns
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants