Skip to content

🤖 test: run the remaining CoderTemplateTest Kind phases - #211

Merged
ThomasK33 merged 2 commits into
mainfrom
feat/template-test-16-kind-phases
Oct 3, 2026
Merged

ThomasK33 merged 2 commits into
mainfrom
feat/template-test-16-kind-phases

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

Summary

Plan PR 7b of the CoderTemplateTest stack (Refs #152): the remaining Kind E2E phases from plan 5.4, split out of the activation PR (plan A16.5), plus two nits from the #210 assessment.

Based on main after #210 (activation) landed.

New driver phases

They run after the three passing tests from #210 and before the key count. The namespace-deletion phase stays last.

  1. Agent failure. fail=true must end Failed AgentStartError with WorkspaceDeleted=True (Deleted), and one workspace row.
  2. Bad parameter. fail=notabool must end Failed CreateRejected with WorkspaceDeleted=True (NotCreated), and zero workspace rows.
  3. Restart. With startup_delay=60, at Running/WaitingForAgents, the driver deletes the operator pod and waits for a new Ready pod. The test must still end Succeeded, with one workspace row.
  4. Delete mid-run. With startup_delay=60, at Running/WaitingForAgents, the driver deletes the test. The object must disappear within 300 s, Coder must show a succeeded delete build for its workspaceID, and there must be one workspace row.
  5. Immutability. A merge patch of spec.timeoutSeconds must fail with spec is immutable.
  6. Dry run. kubectl create --dry-run=server must store no object. The driver derives the workspace name the test would get from the UID in the dry-run answer and requires zero rows with that name. It checks again after the TTL phase, so a late workspace is still caught.
  7. TTL. A test with ttlSecondsAfterFinished: 0 must disappear. A watch started before the create must show a MODIFIED event with Succeeded and WorkspaceDeleted=True, then a DELETED event. This proves the test passed before it was removed.

The receipt adds phase_seconds. Workspace rows are counted in Coder's workspaces table, deleted rows included.

#210 assessment nits

  1. The controlPlaneGone doc comment is back above its function. retainAllowed had ended up between them.
  2. The OwnershipUnknown message no longer names retain when the control plane does not set spec.templateTests.allowRetain. It then says that only removing the finalizer releases the test. With the opt-in, it names both. Test: TestTemplateTestCleanupRechecks checks both messages, without Coder reads.

The upgrade note (apply the new CRD before or together with the new image) goes into the docs PR 8b.

Offline tests

The stubs cover the success path and one failure scenario per phase: fail-ignored, badparam-created, restart-failed, midrun-delete-failed, immutable-accepted, dryrun-persists, and ttl-failed. Each one exits nonzero with a clear message.

Validation

  • Local Kind replica of the whole e2e-kind job, including the step after the driver: the driver exited 0 with 28 passed cases in 306 s, and the job took 482 s.

    Phase Duration
    Agent failure 23 s
    Bad parameter 0 s (final on the first read)
    Restart (a new operator pod took over) 78 s
    Delete mid-run 26 s
    Immutability and dry run 1 s
    TTL 25 s
  • Offline driver tests, make test-scripts, shellcheck, and actionlint pass.

  • All on the pushed tree, with exit 0 each: make verify-vendor, make test, make test-integration, make build, make lint, make codegen, make manifests, make docs-reference (no diff afterwards), and go test -race ./internal/controller/....

Known limits

  • The restart phase needs startup_delay=60 to outlast the operator restart, which took about 15 s locally. On a much slower runner, the test could finish before the restart. That would weaken the phase without failing it.
  • These phases add about 2.5 minutes to the driver. This PR's full E2E run on CI took 7 min 15 s (job), against 5 min 8 s for 🤖 feat: activate the CoderTemplateTest controller #210, within the plan's 10-minute budget for all new phases.

Line count

Hand-written: 4 files changed, 155 insertions(+), 24 deletions(-) (no generated or vendored files).


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-03T15:44:06.299557Z 83b1eac Manual request
🔒 Security Review ✅ Completed 2026-10-03T15:35:31.114048Z 83b1eac Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: c02435346f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Base automatically changed from feat/template-test-15-activate to main October 3, 2026 15:24
The lifecycle driver now runs the plan 5.4 phases before the namespace
deletion: fail=true ends Failed/AgentStartError with the workspace deleted;
fail=notabool ends Failed/CreateRejected with no workspace; deleting the
operator pod at WaitingForAgents still ends Succeeded; deleting a test at
WaitingForAgents removes it and its workspace; a spec patch is refused as
immutable; a server dry run stores no test and no workspace; and
ttlSecondsAfterFinished=0 removes a test after a watched Succeeded.
Workspaces rows prove one workspace per run (zero for the bad parameter
and the dry run), and the receipt records each phase's duration. Offline
stubs cover the success path and one failure per phase.

Refs #152

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

Change-Id: I2625040736fc55f889c0c639e559ca81f7ab6630
Without spec.templateTests.allowRetain, the OwnershipUnknown message now says that only removing the finalizer releases the test. The controlPlaneGone doc comment is back above its function.

Refs #152

Change-Id: Ic91a6816d0c08c397876f384defb10a52aaeb222
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33
ThomasK33 force-pushed the feat/template-test-16-kind-phases branch from c024353 to 83b1eac Compare October 3, 2026 15:33
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 83b1eacf9e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 83b1eacf9e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
ThomasK33 added this pull request to the merge queue Oct 3, 2026
Merged via the queue into main with commit 43a6855 Oct 3, 2026
10 checks passed
@ThomasK33
ThomasK33 deleted the feat/template-test-16-kind-phases branch October 3, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant