Introduce GA API surface: namespace operations, FetchXML paging, streaming queries, and v0→v1 migration tool

[sagebree]

Summary

This PR aligns the SDK with its GA public API contract. It introduces a clean operations namespace,
replaces the flat client.* v0 surface with structured sub-namespaces, adds full FetchXML support
with correct paging cookie handling, and ships a migration tool for existing v0 callers.

---

Changes

1. API surface: v0 removal and v1 namespace introduction

All deprecated v0 methods (create, update, delete, get, list, etc.) have
been removed from DataverseClient (~570 lines). The client now exposes three clean namespaces:

client.records   # CRUD operations
client.query     # Query operations (builder, SQL, FetchXML)
client.batch     # Bulk/batch operations
` ` `

---

### 2. Record operations (`client.records`)

New methods in `operations/records.py`:
- `create()`, `update()`, `delete()`, `upsert()` — single and bulk variants
- `retrieve()` — returns a `QueryResult`
- `list()` — returns a `QueryResult` (filter, select, top)
- `list_pages()` — lazy iterator yielding one `QueryResult` per HTTP page

---

### 3. Query operations (`client.query`)

- `builder(table)` — fluent `QueryBuilder` with `.where()`, `.select()`, `.order_by()`, `.expand()`, `.execute()`, `.execute_pages()`
- `sql(query)` — pass-through SQL SELECT
- `fetchxml(xml)` — returns an inert `FetchXmlQuery`; no HTTP until `.execute()` or `.execute_pages()` is called

`execute(by_page=True/False)` on `QueryBuilder` is deprecated; use `execute_pages()` instead.

---

### 4. FetchXML support (`models/fetchxml_query.py`)

New `FetchXmlQuery` implementing the correct Dataverse paging cookie algorithm per
[public documentation](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/fetchxml/page-results?tabs=webapi):

- Annotation parsed as outer XML `<cookie pagenumber="N" pagingcookie="DOUBLE_ENCODED" />`
- `pagingcookie` attribute extracted and double URL-decoded to get the inner cookie XML
- `pagenumber` from annotation used for next page (trusts server over local counter)
- `morerecords` handled as both `bool` and string `"true"`
- Simple paging fallback (no cookie returned) continues with page increment and emits `UserWarning`
- Safety guards: 32,768-character URL limit (documented Dataverse GET cap per
  [compose-http-requests-handle-errors](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/webapi/compose-http-requests-handle-errors#maximum-url-length)),
  10,000-page circuit breaker
- Input validation: type, non-empty, XML well-formedness, entity element and name presence, URL length pre-check

---

### 5. `QueryResult` (`models/record.py`)

New typed result wrapper returned by all read operations:

```python
result.first()         # first Record or None
result.to_dataframe()  # pandas DataFrame
len(result)            # record count
for record in result:  # iterable
` ` `

---

### 6. `DataverseModel` protocol (`models/protocol.py`)

New `DataverseModel` typed protocol for table-bound model classes, for future use

---

### 7. Filter expressions (`models/filters.py`)

Enhanced `FilterExpression`, `col()`, and `raw()` — now exported from the top-level package:

```python
from PowerPlatform.Dataverse import col, raw, QueryResult, DataverseModel
` ` `

---

### 8. v0→v1 migration tool (`tools/migrate_v0_to_v1.py`)

New CLI tool that rewrites existing v0 call sites to the v1 namespace API. Supports `--dry-run`.
Covers create, update, delete, list, get, fetchxml, and query builder patterns.

---

### 9. Tests

Four new phase test files covering the full GA surface:

| File | Scope |
|---|---|
| `test_phase1_ga.py` | v0 removals, deprecation guards |
| `test_phase2_ga.py` | QueryResult, execute(), exports |
| `test_phase3_ga.py` | retrieve(), list(), DataverseModel |
| `test_phase4_ga.py` | fetchxml(), input validation, paging cookie, deprecated odata helpers |

---

### 10. Examples and documentation

- New `examples/advanced/fetchxml.py` — end-to-end FetchXML scenarios (basic, paging, aggregates, link-entity, system tables)
- Updated `examples/advanced/sql_examples.py`, `batch.py`, `walkthrough.py`, and others to v1 API
- `README.md` updated with v1 usage patterns
- SKILL.md updated to reflect GA surface

[Copilot]

Pull request overview

Summary

This PR aligns the SDK with its GA public API contract. It introduces a clean operations namespace,
replaces the flat client.* v0 surface with structured sub-namespaces, adds full FetchXML support
with correct paging cookie handling, and ships a migration tool for existing v0 callers.

---

Changes

1. API surface: v0 removal and v1 namespace introduction
2. Record operations (client.records)
3. Query operations (client.query)
4. FetchXML support (models/fetchxml_query.py)
5. QueryResult (models/record.py)
6. DataverseModel protocol (models/protocol.py)
7. Filter expressions (models/filters.py)
8. v0→v1 migration tool (tools/migrate_v0_to_v1.py)
9. Tests
10. Examples and documentation

Reviewed changes

Copilot reviewed 38 out of 39 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tools/init.py	Marks tools/ as a Python package for the migration codemod module.
tools/migrate_v0_to_v1.py	Adds a LibCST-based codemod to rewrite common v0 API usage to v1 GA patterns.
tests/unit/test_records_operations.py	Adds unit tests for records.list_pages(); existing tests cover CRUD/get behaviors.
tests/unit/test_query_operations.py	Updates query builder tests to use col() expressions and GA patterns; revises dataframe test expectations.
tests/unit/test_phase2_ga.py	Adds regression tests validating QueryResult, builder flat execution, exports, and warning behavior.
tests/unit/test_phase3_ga.py	Adds regression tests for GA record APIs (retrieve, list), DataverseModel protocol, and warning suppression.
tests/unit/test_phase4_ga.py	Adds regression tests for FetchXML query support, GA-clean warning guarantees, and codemod transforms.
tests/unit/test_client.py	Refactors/updates client construction tests and verifies removed flat methods raise AttributeError.
tests/unit/test_client_deprecations.py	Updates tests to assert formerly-deprecated flat methods are removed (raise AttributeError).
src/PowerPlatform/Dataverse/operations/records.py	Adds DataverseModel overloads, GA retrieve/list/list_pages, and deprecates records.get().
src/PowerPlatform/Dataverse/operations/query.py	Adds query.fetch_xml() plus GA deprecation warnings for odata_* helpers; removes SQL join/select helpers.
src/PowerPlatform/Dataverse/operations/dataframe.py	Deprecates dataframe.get() and suppresses internal records.get() deprecation leakage.
src/PowerPlatform/Dataverse/operations/batch.py	Deprecates batch.records.get(), adds batch.records.retrieve() and batch.records.list() (single-page).
src/PowerPlatform/Dataverse/models/record.py	Introduces QueryResult wrapper and exports it alongside Record.
src/PowerPlatform/Dataverse/models/query_builder.py	Removes fluent filter_* methods, adds execute_pages(), changes execute() to return QueryResult, and deprecates by_page flag + to_dataframe().
src/PowerPlatform/Dataverse/models/protocol.py	Adds runtime-checkable DataverseModel protocol for typed entity CRUD integration.
src/PowerPlatform/Dataverse/models/fetch_xml_query.py	Adds inert FetchXmlQuery with .execute() and .execute_pages() supporting paging-cookie propagation.
src/PowerPlatform/Dataverse/models/init.py	Re-exports col, raw, DataverseModel, and QueryResult.
src/PowerPlatform/Dataverse/data/_odata.py	Adds _build_list() to construct single-page multi-record GET requests (used for batch list).
src/PowerPlatform/Dataverse/data/_batch.py	Adds _RecordList and resolves it via _build_list() for batch multi-record reads.
src/PowerPlatform/Dataverse/client.py	Removes deprecated flat client methods, leaving namespace-based APIs as the GA surface.
src/PowerPlatform/Dataverse/claude_skill/dataverse-sdk-use/SKILL.md	Updates guidance/docs for GA patterns and deprecations.
.claude/skills/dataverse-sdk-use/SKILL.md	Mirrors GA guidance updates for the Claude skill copy of the SDK usage doc.
README.md	Updates public documentation for GA APIs (retrieve/list/list_pages, execute_pages, fetch_xml, deprecations).
pyproject.toml	Adds an optional migration extra (libcst) to support the codemod tool.
examples/basic/installation_example.py	Updates sample usage to GA retrieve/list calls (currently has a loop-shape bug).
examples/basic/functional_testing.py	Updates functional test harness to GA list/list_pages patterns and improves idempotency for table/relationship setup.
examples/advanced/walkthrough.py	Updates the walkthrough to use col() expressions, GA paging (execute_pages), and GA read APIs.
examples/advanced/sql_examples.py	Removes GA-removed SQL helper usage and updates narrative to recommend sql_columns/manual joins or FetchXML.
examples/advanced/prodev_quick_start.py	Updates dataframe querying patterns and improves entity set name resolution for @odata.bind.
examples/advanced/fetch_xml.py	Adds an end-to-end FetchXML example script demonstrating .execute() and .execute_pages() usage.
examples/advanced/datascience_risk_assessment.py	Updates dataframe extraction to GA builder patterns using col/raw.
examples/advanced/dataframe_operations.py	Updates dataframe “get” examples to GA builder patterns using raw/col.
examples/advanced/batch.py	Updates batch examples, switches to interactive base_url input, and uses GA batch record methods.
examples/advanced/alternate_keys_upsert.py	Makes table/key creation idempotent and updates verification loop to GA records.list().

Comments suppressed due to low confidence (1)

pyproject.toml:63

• The migration extra adds a libcst dependency, and the README/tests reference running python -m tools.migrate_v0_to_v1 after installing the package. But the tools/ package is outside src/ and won’t be included in the wheel with the current setuptools where = ["src"] config, so the module (and any CLI) won’t exist for installed users. Consider moving the codemod under src/PowerPlatform/Dataverse/... (or adjusting packaging) and/or adding a console_script entry so the migration tool is actually available when installed.

[project.scripts]
dataverse-install-claude-skill = "PowerPlatform.Dataverse._skill_installer:main"

[project.optional-dependencies]
dev = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "black>=23.0.0",
    "isort>=5.12.0",
    "mypy>=1.0.0",
    "ruff>=0.1.0",
]
migration = ["libcst>=1.0.0"]

[tool.setuptools]
package-dir = {"" = "src"}
zip-safe = false

[tool.setuptools.packages.find]
where = ["src"]
include = ["PowerPlatform*"]
namespaces = false

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.