test(integration): dynamic-stack CMA SDK sanity suite + AM 2.0 coverage + asset Content-Type fix (v1.10.1)#168
Open
aniket-shikhare-cstk wants to merge 5 commits into
Open
Conversation
…api_version header
Add a self-contained pytest integration suite under tests/integration that creates a fresh stack per run, exercises every SDK resource method (positive, negative, and edge cases) against the live CMA API, and tears the stack down. - framework/: dynamic stack setup/teardown, request+cURL capture, response/error validators, tracked assertions, and a custom dashboard HTML report - api/: 30 resource files with full method coverage - data/: complex content-type schemas (modular blocks, groups, references, JSON RTE) and entry payloads - strict, bug-catching assertions; genuine SDK/environment issues tracked via xfail - timestamped HTML report + cURL log written to repo root (gitignored) Also: update AGENTS.md to document the sanity suite + env vars, add pytest/pytest-order to requirements, gitignore secrets/reports/docs.
Cover the asset scanning feature (DAM/AM 2.0) against both the normal org and the AM 2.0 (DAM-enabled) org: - normal-org scan tests (test_06_asset): upload returns _asset_scan_status 'pending'; field absent unless include_asset_scan_status=true; clean file scans 'clean'; EICAR test file scans 'quarantined'; listing includes status - test_31_am_assets: AM-org assets get 'am'-prefixed UIDs, full CRUD round-trip, the same scan lifecycle, publish() with the api_version: 3.2 header (publish- only; 404 on fetch) - framework: am_stack fixture (stack in AM_ORG_UID; whole suite skips when unset), runtime-generated EICAR fixture (base64-encoded so the signature is not committed raw), wait_for_scan() polling helper, and api_version header reset for per-test isolation Note: the correct query param is include_asset_scan_status (the response field is _asset_scan_status); verified live.
Asset.update() forced Content-Type: multipart/form-data while sending a JSON body, and Asset.replace() set a bare multipart/form-data header (no boundary) while passing files=. Both made the CMA API reject the request with 422 'Please send a valid multipart/form-data payload', and both leaked the wrong Content-Type onto subsequent requests on the shared client. - update(): send the JSON body as application/json (matches the JS SDK and the live API, which returns 200) - replace(): let the HTTP layer build the multipart body with a proper boundary Bump version to 1.10.1 and update the asset update unit test accordingly.
- taxonomy/terms/global-field delete: drop Content-Type on the body-less DELETE (the SDK merges application/json, which the API rejects 400/500; the JS SDK omits it) — these now pass instead of xfail - bulk update_workflow: provide a real target stage uid + notify field - branch delete: poll-retry the transient 'branch not valid' (905) while the branch finishes async provisioning - asset update/replace: flip from xfail to passing (fixed in the SDK, v1.10.1) - capture/setup: retry transient network errors (ReadTimeout/ConnectionError) 2x with backoff and bump the client timeout to 120s so dev11 blips don't fail the run
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
✅ BUILD PASSED - All security checks passed |
netrajpatel
approved these changes
Jun 26, 2026
There was a problem hiding this comment.
Pull request overview
Adds a new live “sanity” integration test suite under tests/integration/ that provisions a fresh stack per run, captures all CMA traffic into an HTML/cURL report, and exercises broad SDK resource coverage. It also includes an SDK fix for asset Content-Type handling (replace() boundary + update() JSON), bumps the package version to 1.10.1, and updates the changelog/agent docs.
Changes:
- Introduces a full dynamic-stack integration framework (setup/teardown, request capture, HTML reporting, shared store/fixtures).
- Adds ordered, per-resource live API coverage tests (including AM 2.0 asset scanning/publish behavior).
- Fixes asset request header behavior for
replace()andupdate(), and updates versioning/docs.
Reviewed changes
Copilot reviewed 46 out of 51 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/assets/test_assets_unit.py | Updates asset unit assertions (incl. new scan-status/api_version coverage). |
| tests/integration/pytest.ini | Integration-only pytest config (test discovery, markers, opts). |
| tests/integration/conftest.py | Session fixtures, dynamic setup/teardown, per-test header isolation, report hooks. |
| tests/integration/framework/init.py | Framework package marker. |
| tests/integration/framework/context.py | Shared run context + cross-file UID store. |
| tests/integration/framework/helpers.py | Generators, retry/wait helpers, response validators, tracked assertions. |
| tests/integration/framework/capture.py | Monkeypatches requests.request to capture traffic + generate cURL + retry transients. |
| tests/integration/framework/report.py | Renders a self-contained HTML dashboard + plain cURL log. |
| tests/integration/framework/setup.py | Login + dynamic stack lifecycle + management token + optional Personalize project. |
| tests/integration/data/init.py | Data package marker. |
| tests/integration/data/content_types.py | Content-type schema payload factories for integration coverage. |
| tests/integration/data/entries.py | Entry payload factories for integration coverage. |
| tests/integration/data/assets/extension.html | HTML asset used for extension upload tests. |
| tests/integration/api/test_01_user.py | Live tests for user endpoints and safe auth ops. |
| tests/integration/api/test_02_organization.py | Live tests for organization endpoints and safe ownership negatives. |
| tests/integration/api/test_03_stack.py | Live tests for stack endpoints (settings/users/sharing). |
| tests/integration/api/test_04_locale.py | Live tests for locale CRUD + fallback operations. |
| tests/integration/api/test_05_environment.py | Live tests for environment CRUD. |
| tests/integration/api/test_06_asset.py | Live tests for asset CRUD, folders, versions, publish, and scan status. |
| tests/integration/api/test_07_taxonomy.py | Live tests for taxonomy CRUD. |
| tests/integration/api/test_08_terms.py | Live tests for taxonomy terms CRUD + hierarchy/search. |
| tests/integration/api/test_09_extension.py | Live tests for extensions CRUD + upload. |
| tests/integration/api/test_10_webhook.py | Live tests for webhook CRUD + executions/logs/retry. |
| tests/integration/api/test_11_global_field.py | Live tests for global field CRUD + export. |
| tests/integration/api/test_12_content_type.py | Live tests for content type CRUD + complex schema round-trip. |
| tests/integration/api/test_13_label.py | Live tests for label CRUD. |
| tests/integration/api/test_14_entry.py | Live tests for entry CRUD, complex content, atomic ops, localization, publish. |
| tests/integration/api/test_15_variant_group.py | Live tests for variant group CRUD (Personalize-gated). |
| tests/integration/api/test_16_variants.py | Live tests for variants CRUD (Personalize-gated). |
| tests/integration/api/test_17_entry_variants.py | Live tests for entry variants endpoints (Personalize-gated). |
| tests/integration/api/test_18_branch.py | Live tests for branch CRUD with force delete + retries. |
| tests/integration/api/test_19_alias.py | Live tests for branch alias assign/find/fetch/delete. |
| tests/integration/api/test_20_role.py | Live tests for role CRUD. |
| tests/integration/api/test_21_workflow.py | Live tests for workflow CRUD, stages, publish rules, tasks. |
| tests/integration/api/test_22_delivery_token.py | Live tests for delivery token CRUD. |
| tests/integration/api/test_23_management_token.py | Live tests for management token CRUD. |
| tests/integration/api/test_24_release.py | Live tests for release CRUD + clone. |
| tests/integration/api/test_25_release_item.py | Live tests for release items operations. |
| tests/integration/api/test_26_bulk_operation.py | Live tests for bulk publish/unpublish/delete/workflow update. |
| tests/integration/api/test_27_publish_queue.py | Live tests for publish queue find/fetch/cancel. |
| tests/integration/api/test_28_metadata.py | Live tests for metadata endpoints (extension-backed). |
| tests/integration/api/test_29_auditlog.py | Live tests for audit log find/fetch. |
| tests/integration/api/test_30_oauth.py | Live tests for OAuth handler surface (authorize URL, token accessors). |
| tests/integration/api/test_31_am_assets.py | Live tests for AM 2.0 assets (UID prefix, scan, publish-only api_version header). |
| requirements.txt | Adds integration-suite deps (pytest, pytest-order). |
| contentstack_management/assets/assets.py | Fixes asset replace() multipart boundary handling and update() Content-Type logic. |
| contentstack_management/init.py | Bumps SDK version to 1.10.1. |
| CHANGELOG.md | Adds v1.10.1 entry describing asset header fixes. |
| AGENTS.md | Documents new integration suite usage and env vars. |
| .gitignore | Ignores integration outputs/env files and timestamped reports. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+14
to
+16
| Env vars (see .env.example): | ||
| Required: EMAIL, PASSWORD, HOST, ORGANIZATION | ||
| Optional: MFA_SECRET, DELETE_DYNAMIC_RESOURCES (default 'true') |
Comment on lines
+48
to
+51
| raise RuntimeError( | ||
| f"Missing required environment variables: {', '.join(missing)}. " | ||
| f"See tests/integration/.env.example." | ||
| ) |
Comment on lines
+180
to
182
| self.client.headers.pop("Content-Type", None) | ||
| files = {"asset": open(f"{file_path}",'rb')} | ||
| return self.client.put(url, headers = self.client.headers, params = self.params, files = files) |
Comment on lines
412
to
416
| url = f"assets/{self.asset_uid}" | ||
| Parameter.add_header(self, "Content-Type", "multipart/form-data") | ||
| # Updating an asset's title/description sends a JSON body, so it must use | ||
| # application/json. Forcing multipart/form-data here makes the API reject | ||
| # the request with 422 "Please send a valid multipart/form-data payload". | ||
| return self.client.put(url, headers = self.client.headers, params = self.params, data = data) |
Comment on lines
+138
to
+142
| for key, val in list(files.items()): | ||
| name = getattr(val[1] if isinstance(val, (tuple, list)) else val, "name", None) | ||
| if name and os.path.exists(name): | ||
| kwargs["files"][key] = (val[0], open(name, "rb"), val[2]) if isinstance(val, (tuple, list)) else open(name, "rb") | ||
| raise last_exc |
Comment on lines
+262
to
+283
| def test_fetch_includes_scan_status_param(self): | ||
| asset = self.client.stack(api_key).assets(asset_uid) | ||
| asset.add_param("_asset_scan_status", True) | ||
| response = asset.fetch() | ||
| self.assertIn("_asset_scan_status=True", response.request.url) | ||
| self.assertEqual(response.request.method, "GET") | ||
|
|
||
| def test_find_includes_scan_status_param(self): | ||
| asset = self.client.stack(api_key).assets() | ||
| asset.add_param("_asset_scan_status", True) | ||
| response = asset.find() | ||
| self.assertIn("_asset_scan_status=True", response.request.url) | ||
| self.assertEqual(response.request.method, "GET") | ||
|
|
||
| def test_upload_includes_scan_status_param(self): | ||
| file_path = "tests/resources/mock_assets/chaat.jpeg" | ||
| asset = self.client.stack(api_key).assets() | ||
| asset.add_param("_asset_scan_status", True) | ||
| response = asset.upload(file_path) | ||
| self.assertIn("_asset_scan_status=True", response.request.url) | ||
| self.assertEqual(response.request.method, "POST") | ||
|
|
Comment on lines
+284
to
+307
| def test_fetch_without_scan_status_param_field_absent(self): | ||
| response = self.client.stack(api_key).assets(asset_uid).fetch() | ||
| self.assertNotIn("_asset_scan_status", response.request.url) | ||
| self.assertEqual(response.request.method, "GET") | ||
|
|
||
| def test_find_without_scan_status_param_field_absent(self): | ||
| response = self.client.stack(api_key).assets().find() | ||
| self.assertNotIn("_asset_scan_status", response.request.url) | ||
| self.assertEqual(response.request.method, "GET") | ||
|
|
||
| def test_upload_without_scan_status_param_field_absent(self): | ||
| file_path = "tests/resources/mock_assets/chaat.jpeg" | ||
| response = self.client.stack(api_key).assets().upload(file_path) | ||
| self.assertNotIn("_asset_scan_status", response.request.url) | ||
| self.assertEqual(response.request.method, "POST") | ||
|
|
||
| def test_scan_status_param_coexists_with_other_params(self): | ||
| asset = self.client.stack(api_key).assets(asset_uid) | ||
| asset.add_param("locale", "en-us") | ||
| asset.add_param("_asset_scan_status", True) | ||
| response = asset.fetch() | ||
| self.assertIn("locale=en-us", response.request.url) | ||
| self.assertIn("_asset_scan_status=True", response.request.url) | ||
| self.assertEqual(response.request.method, "GET") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a comprehensive, dynamic-stack live integration ("sanity") test suite for the CMA Python SDK under
tests/integration/, replacing the ~2-year-old API tests. Each run creates a fresh stack, exercises every SDK resource method (positive / negative / edge cases) against the live CMA API, then tears the stack down. Modeled on the JS CMA SDK sanity suite, adapted to pytest.What's included
Framework (
tests/integration/framework/)setup.py— login → create stack → management token → (optional) Personalize project → flag-gated teardowncapture.py— monkeypatchesrequeststo record every call (method/url/headers/body/cURL) + transient-error retriesreport.py— self-contained dashboard HTML report (summary, coverage-by-resource, per-test request/response/cURL); honest status (passed-with-warnings, xfail)helpers.py— generators, response/error validators,tracked_assert, scan-status pollingconftest.py— session fixtures (ctx,stack,store, AM stack, EICAR file),pytest-ordersequencing, per-test header isolation, report hooksCoverage (
tests/integration/api/, 31 files) — user, org, stack, locale, environment, asset, taxonomy, terms, extension, webhook, global field, content type, label, entry, variant group, variants, entry variants, branch, alias, role, workflow, delivery/management tokens, release, release items, bulk ops, publish queue, metadata, audit log, oauth, AM 2.0 assets. Complex content-type schemas (modular blocks, groups, references, JSON RTE) round-tripped through entries.AM 2.0 / asset scanning — covers
include_asset_scan_status=truesurfacing_asset_scan_status(pending → clean | quarantined),am-prefixed UIDs in DAM-enabled orgs, and the publish-onlyapi_version: 3.2header. The asset-scan suite skips gracefully when the DAM-enabled org isn't configured.SDK bug fix (v1.10.1) — included here, see commit
fix(assets): …Asset.update()sent a JSON body withContent-Type: multipart/form-data→ API 422. Fixed toapplication/json.Asset.replace()set a baremultipart/form-data(no boundary) → API 422. Fixed to let the HTTP layer set the boundary.1.10.0 → 1.10.1+ CHANGELOG + unit-test assertion updated. Backward-compatible (the methods were previously always failing).Notes for reviewers
pytest-orderenforces dependency order; cross-file UIDs pass via a sharedstore.tests/unit+tests/mockare untouched (offline coverage retained); legacytests/apiis superseded by this suite (kept for now).tests/unit/assets/), so the asset-scan unit coverage lands with this PR.