feat(audit): scan existing resources at startup and after upgrades with durable operator findings

[purp]

User Story

As a Platform Admin, I want OpenShell to check existing resources against the running version's operational and security requirements at startup, including after an upgrade, and promptly surface actionable findings so that I can review affected workloads and choose a remedy.

Problem Statement

OpenShell already performs database migrations, selected startup validation, and sandbox lifecycle reconciliation. It lacks a unified operator-facing result for checks of existing resources: which checks completed, which resources need attention, what could not be checked, and which findings remain unresolved after a gateway restart.

Impact / Why This Matters

Operators must combine release notes, manual inspection, and process logs to assess existing workloads after an upgrade. Transient events can be missed, repeated restarts can repeat the same warning, and a successfully started gateway does not tell the operator whether all existing resources passed the relevant checks.

Proposed Design

• Run applicable checks on gateway startup, including the first startup after an upgrade. Cover persisted sandboxes and workload templates, and compare live runtime state where a check requires it and the driver supports that inspection.
• Provide a durable findings view for Platform Admins. Each finding identifies its check and revision, affected workspace and resource, severity, evidence summary, first/last observation, resolution state, and recommended action. Keep sensitive values out of notifications.
• Distinguish a detected problem, missing approval or evidence, and an incomplete check. An incomplete scan must never appear as a clean result; a configuration finding must not be presented as proof of exploitation.
• Emit a structured event promptly when a finding first appears or materially changes, plus a startup scan summary. Repeated scans and multiple gateway replicas must not produce duplicate unresolved findings or notification storms.
• Preserve running workloads for operator review. When a check makes further activation conditional on remediation or operator approval, hold affected start/resume operations, including automatic recovery, while keeping administrative access available. Let the Platform Admin inspect, approve where supported, stop, or delete affected workloads explicitly. Acknowledging a notification alone must not satisfy a required approval.
• Allow Platform Admins to inspect findings and rerun checks through a documented API and CLI workflow. Reuse the existing event and observability work for delivery; client UI integration can follow.

This feature owns scan results, durable finding state, and their operator workflow. Coordinate event production with #2911 and use the delivery surfaces tracked in #1933 and #2762; a new email or webhook service is not required by this proposal. Document the available notification destination and any operator setup required for prompt delivery.

Acceptance Criteria

• Startup and post-upgrade checks report completion, check revisions, resource coverage, findings, and any incomplete checks.
• An initial documented check exercises the full scan-to-finding-to-resolution workflow; the feature is not only an empty framework.
• Findings survive gateway restarts and are discoverable by Platform Admins without subscribing before startup.
• Unchanged results are deduplicated across repeated scans and gateway replicas; new, materially changed, and resolved findings emit documented events.
• A failed or unavailable check is visible and cannot be interpreted as a pass.
• Applicable activation checks run before automatic recovery or explicit start/resume can activate an affected workload.
• Detection alone does not terminate running workloads; administrative inspection and remediation remain available.
• Findings distinguish missing approval/evidence from demonstrated unauthorized behavior; notification acknowledgement does not grant approval.
• Cross-workspace findings are restricted to Platform Admins; any workspace-facing detail obeys workspace authorization and redaction rules.
• Operator documentation explains inspection, rerun, resolution, notification setup, activation behavior, and coverage limitations.
• Tests cover restart persistence, repeated scans, multiple replicas, incomplete scans, lifecycle enforcement, notification transitions, and authorization boundaries.

Alternatives Considered

• Manual post-upgrade commands alone: useful as a rerun option, but easy to omit during deployment.
• Startup log warnings alone: inexpensive, but provide no durable unresolved-findings view or reliable resolution workflow.
• Refuse all gateway startup on any finding: prevents administrators from using the API to investigate and remediate unaffected or already-running resources.
• Automatically terminate every flagged workload: conflates a finding with a confirmed incident and removes the operator's decision about disruption.

Agent Investigation

At source revision 292559c41c1b010de499f1d082583c1aec7b3a4a:

• crates/openshell-server/src/lib.rs already sequences startup validation and persisted sandbox recovery.
• crates/openshell-server/src/compute/mod.rs enumerates persisted sandboxes and reconciles driver state. Audit ordering must account for interrupted lifecycle recovery as well as the ordinary startup sweep.
• Existing sandbox watch and event buses provide live delivery, but their event buffers are in memory. Sandbox conditions are also rebuilt from driver observations, so durable findings need explicit preservation and ownership.
• The generic persistence store and structured event builders offer reusable foundations. Existing planned event export and subscription work should supply delivery rather than be duplicated here.