Skip to content

🤖 feat: activate the CoderTemplateTest controller - #210

Merged
ThomasK33 merged 2 commits into
mainfrom
feat/template-test-15-activate
Oct 3, 2026
Merged

ThomasK33 merged 2 commits into
mainfrom
feat/template-test-15-activate

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

Summary

Plan PR 7 of the CoderTemplateTest stack (Refs #152): activation. The operator now runs the CoderTemplateTest controller and installs its CRD. The Kind E2E runs real tests against the agent fixture from #208.

Required items

  1. Kind namespace-deletion test (plan A13.2, A1). This is the last driver phase. A test with startup_delay=120 reaches Running/WaitingForAgents. The driver then sets spec.replicas=0 on the CoderControlPlane, so Coder is unreachable while the control plane still exists. The operator keeps that desired state and does not undo it. The test reports CoderUnavailable, and then the driver deletes namespace coder. Within 300 s, the test and the control plane must be gone, and the namespace must report NamespaceContentRemaining=False and NamespaceFinalizersRemaining=False. In the local Kind run, this took about 11 s.

  2. How-to notes (docs/how-to/test-templates.md):

    • how to release a stuck ControlPlaneUnavailable or CoderUnavailable test,
    • that the TTL does not apply to Retained or ControlPlaneGone tests,
    • that workspaces with more than 100 builds stay unproven,
    • TLS (🤖 Use the internal Coder URL for provisioner keys and the aggregated API when TLS is on #198, documented as a known limit, not fixed). With TLS on, CoderTemplateTest calls Coder at the internal http:// service URL, so its operator token crosses the cluster network as plain HTTP. Separately, the CoderProvisioner controller and the aggregated API server use the https:// service URL and can fail TLS verification. I did not verify that failure in a live cluster.
  3. Dormant-CRD CI guard. Removed, together with the whole dormant mechanism: config/crd/dormant/, DORMANT_CRDS in hack/update-manifests.sh, the dormant skip in hack/update-reference-docs.sh, the second envtest CRD directory, and the CI drift paths.

  4. Tester key count in Kind (plan A4). The driver counts the tester's api_keys rows with a read-only SELECT count(*) in the CNPG primary, before and after three tests. As a check that the query works, the operator user's key count must be above 0. Result: 0 before and 0 after. Coder v2.37.2 explains this: every start build creates the owner's workspace session token, and every stop or delete build removes it (coderd/provisionerdserver/provisionerdserver.go:744-756, :3296-3332). Keys do not accumulate, so I filed no follow-up issue. The how-to states this, including that a retained workspace keeps its key.

  5. Dormant owners (approved). Coder creates API users as dormant and refuses a dormant owner's agents with 401. The owner check now allows only active users, and the message says to activate the user. Test: the dormant user row in TestTemplateTestOwnerEligibility.

  6. retain needs the control plane's opt-in (review round 1, plan A17). Anyone who can create a test can set coder.com/deletion-policy: retain at create time, and RBAC cannot see annotations. The new field CoderControlPlane.spec.templateTests.allowRetain (default false) gates it.

    • When it is false, the controller ignores retain, names the reason in the WorkspaceDeleted message, and runs normal cleanup. It deletes the workspace, or keeps the finalizer while Coder is unreachable.
    • When it is true, retain works as before.
    • The how-to says that the opt-in delegates retention to every test creator on that control plane, and that without it an admin releases a stuck test by removing the finalizer.
    • Tests: TestTemplateTestRetain (ignored without the opt-in, then a release with it), TestTemplateTestControlPlaneUnavailable (without the opt-in the finalizer stays while Coder is down, and with it the release needs no Coder call), and the OwnershipUnknown path. Mutation check: allowing retain without the opt-in fails those tests.

#208 follow-ups

  1. coder_api sends the token as -H @file and the body as --data @file, from 0600 files that are removed at exit. No secret appears in argv, and the offline tests check this.
  2. CI retries the alpine docker pull 3 times, 10 s apart.
  3. A rerun with an existing e2e-tester user (409) reuses it after the same checks (password login, default organization) and still activates it. Offline scenarios: tester-exists and tester-exists-oidc.

What

  • internal/app/controllerapp: registers CoderTemplateTestReconciler with the real clock.
  • internal/controller/codertemplatetest_controller.go: RBAC markers (codertemplatetests get/list/watch/update/patch/delete, the status and finalizers subresources), and an updated type comment.
  • Generated: the CRD in config/crd/bases, config/rbac/role.yaml, dist/install.yaml, config/default/kustomization.yaml, and docs/reference/api/codertemplatetest.md.
  • CI: the e2e paths filter adds api/v1alpha1/**, internal/controller/codertemplatetest*, and internal/app/controllerapp/** (plan 5.5).
  • Driver phases, after the tester and agent steps:
    1. Configure ownerUserID.
    2. Count the tester's keys.
    3. Run three passing tests (each Succeeded, Ready=True, WorkspaceDeleted=True Deleted), with a check that the first run created exactly one workspace row, deleted rows included.
    4. Check that deploy/coder did not roll after the owner patch (plan risk R5).
    5. Count the keys again.
    6. Delete the namespace.
  • Offline stubs: a success path and a failure scenario for each new step.
  • Plan amendment A16 records these results. Plan PR 7b adds the remaining Kind phases: agent failure, bad parameter, restart, immutability, dry run, and TTL.

Review round 1

  • r4173598528 (security: retain set at create time bypasses cleanup): fixed with the allowRetain opt-in.
  • r4173601592 (stale "controller is not enabled" sentence on ownerUserID): removed. The CRD, the install bundle, and the API reference are regenerated.
  • CI docs-quality: markdownlint flagged raw HTML from <organization>.<template> in the template field comment. The comment now puts it in backticks, and .cspell.json lists codertemplatetest(s). Both linters pass locally with CI's versions.

Validation

  • Local Kind run, replicating the whole e2e-kind job including the APIService wrong-CA step after the driver: the driver exited 0 with 22 passed cases in 152 s, and the job took 297 s. The three tests took 23 s, 22 s, and 16 s.
  • Offline driver tests (79 checks), make test-scripts, shellcheck, and actionlint pass.
  • All on the pushed tree, with exit 0 each: make verify-vendor, make test, make test-integration, make build, make lint, make codegen, make manifests, make docs-reference (no diff afterwards), go test -race ./internal/controller/..., make docs-check, and CI's markdownlint-cli2 and cspell versions.
  • E2E job duration: about 3 min 55 s before (🤖 test: add a Kind E2E workspace fixture with a real agent #208). This PR's full E2E run took 5 min 8 s on the review-clean head (job), about 1 min 13 s more, within the plan's 10-minute budget.

Line count

Hand-written: 17 files changed, 412 insertions(+), 81 deletions(-). Generated: 6 files changed, 450 insertions(+), 6 deletions(-) (CRD, RBAC role, install bundle, API reference).


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Register the CoderTemplateTest reconciler with the manager, add its RBAC
markers, and ship its CRD in config/crd/bases, the install bundle, and the
API reference. The dormant-CRD mechanism and its CI guard are gone, and
the Kind E2E now runs on changes to the controller and its API.

The controller refuses dormant owners: Coder creates API users as dormant
and refuses a dormant owner's agents. A short how-to covers the tester,
RBAC, cleanup, and the known limits, including #198.

The Kind E2E runs three passing tests (exactly one workspace each), counts
the tester's API keys before and after, and deletes the namespace while a
test runs and Coder is unreachable. The driver keeps request bodies and the
token out of argv, reuses an existing tester on a rerun, and CI retries the
agent image pull.

Refs #152

Change-Id: Idf51eead4cedefdbd04180c3bfbfe4a5079bef04
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-03T15:10:20.414051Z 3327ad0 Manual request
🔒 Security Review ✅ Completed 2026-10-03T15:11:50.895198Z 3327ad0 Manual request

Security findings

Advisory findings (1)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review

Here are some automated security review suggestions for this pull request.

Reviewed commit: af897bd3ce

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread internal/app/controllerapp/controllerapp.go

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af897bd3ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/how-to/test-templates.md
Anyone who creates a CoderTemplateTest can set coder.com/deletion-policy: retain, and RBAC cannot see annotations. Retain now works only when the CoderControlPlane sets spec.templateTests.allowRetain. Otherwise the controller ignores it, says so in the WorkspaceDeleted message, and deletes the workspace.

Also drop the stale "controller not enabled" sentence from ownerUserID, keep the template name format out of raw HTML in the API reference, and add the kind to the spell list.

Refs #152

Change-Id: Ie668cc34e04dfdbe94bda6623d63b787ea6b2617
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 3327ad083b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 3327ad083b

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Oct 3, 2026
Merged via the queue into main with commit dabb113 Oct 3, 2026
13 checks passed
@ThomasK33
ThomasK33 deleted the feat/template-test-15-activate branch October 3, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant