diff --git a/public/docs/img/kubernetes/live-object-status/live-status-orphaned-objects.png b/public/docs/img/kubernetes/live-object-status/live-status-orphaned-objects.png new file mode 100644 index 0000000000..a914deaee7 Binary files /dev/null and b/public/docs/img/kubernetes/live-object-status/live-status-orphaned-objects.png differ diff --git a/src/pages/docs/argo-cd/live-object-status/index.md b/src/pages/docs/argo-cd/live-object-status/index.md index 416ab69ecd..75a1c91702 100644 --- a/src/pages/docs/argo-cd/live-object-status/index.md +++ b/src/pages/docs/argo-cd/live-object-status/index.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-03-28 -modDate: 2026-03-19 +modDate: 2026-08-21 navSection: Live Object Status title: Overview navTitle: Overview @@ -90,6 +90,10 @@ Sync Status tracks whether the changes Octopus pushed to git still match what Ar | Git Drift | | Octopus has detected that the changes it applied to git have been modified since the last deployment (e.g. by a manual edit) | | Unknown | | We don't have information about the live status of this object | +:::div{.info} +Objects observed through Argo CD are never marked as **Orphaned**. Argo CD has its own [pruning](https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#no-prune-resources) behavior for objects removed from git. +::: + ### Detailed object information Selecting an object or application name in the table will open a drawer containing detailed information. diff --git a/src/pages/docs/kubernetes/live-object-status/deleting-orphaned-objects.md b/src/pages/docs/kubernetes/live-object-status/deleting-orphaned-objects.md new file mode 100644 index 0000000000..ce228b2519 --- /dev/null +++ b/src/pages/docs/kubernetes/live-object-status/deleting-orphaned-objects.md @@ -0,0 +1,86 @@ +--- +layout: src/layouts/Default.astro +pubDate: 2026-08-19 +modDate: 2026-08-21 +navSection: Deleting orphaned objects +title: Deleting orphaned objects +navTitle: Deleting orphaned objects +description: How to review and delete Kubernetes objects that Octopus no longer deploys but that are still running in your cluster. +navOrder: 60 +--- + +When a deployment stops including an object that a previous release deployed, Octopus marks that object as **Orphaned** in Kubernetes Live Object Status, and it keeps running in your cluster until someone removes it. Octopus can delete orphaned objects for you, one at a time or in bulk, from the project's Live Status page. + +## Requirements + +To delete orphaned objects you need: + +- Octopus Server 2026.3 or later +- Every Kubernetes monitor in the application on agent version 2.38.3 or later (v2) / 3.0.1 or later (v3), the same requirement as orphan tracking itself +- The `DeployedResourceAdminister` permission for the project, environment, and tenant you are deleting in +- A Kubernetes agent whose service account is allowed to delete the objects in question +- Objects reported by the [Kubernetes monitor](/docs/kubernetes/targets/kubernetes-agent/kubernetes-monitor). Objects observed through Argo CD are never orphaned and cannot be deleted this way + +## Delete orphaned objects + +Open your project and select **Live Status** for the environment (and tenant) you want. The table offers a delete action on every orphaned row, and when an application has orphaned objects the page also shows a card counting them. To list only orphaned objects, use the **Sync status** filter and select **Orphaned**. + +:::figure +![Live Status table showing orphaned objects](/docs/img/kubernetes/live-object-status/live-status-orphaned-objects.png) +::: + +### A single object + +1. Open the row's **Resource actions** menu and choose **Review & delete**. +2. Confirm in the **Delete orphaned resource** dialog by choosing **Delete**. + +You can also select the object to open its detail drawer and use the **Delete** button there. + +### Several objects at once + +1. Choose **Review & delete** on the orphaned-objects card to open the **Delete orphaned resources** drawer. +2. Select the objects to delete. Objects are grouped by deployment target, so you can select a whole target or everything at once. +3. Choose **Next**. +4. Review the **Confirm deletion** step, which lists every selected object grouped by deployment target. +5. Choose **Delete**. + +## The deletion task + +Each request queues one **Delete Kubernetes resources** task, which is linked from the Kubernetes resource being deleted. Deletion tasks can run at the same time as deployments. + +The deletion task runs a script on the Kubernetes Agent that calls `kubectl delete` with a 5 minute timeout for each resource. If some objects delete and others don't, the task deletes everything it can, records every outcome, and then fails. + +Octopus validates your selection when you confirm, and silently skips an object when: + +- The object is no longer orphaned, most likely because a deployment re-added it. +- A deletion task for the object is already queued or running. +- The monitor no longer reports the object in the cluster, so there is nothing to delete. +- Status information for the object is stale, so Octopus cannot safely delete it. + +## Permissions + +Deleting an orphaned object needs permission in two places: in Octopus, to request the deletion, and in your cluster, for the agent that carries it out. + +### Octopus permissions + +Requesting a deletion requires the `DeployedResourceAdminister` permission ("Administer deployed resources managed by Octopus"), scoped to the project, environment, and tenant. It is included by default in the **Project Contributor**, **Project Deployer**, **Project Lead**, and **Space Manager** built-in roles, and is added to those roles on existing instances when you upgrade. + +### Cluster permissions + +Octopus deletes objects by running `kubectl delete` on the Kubernetes agent, so the deletion uses the **agent's** service account. Make sure that account is allowed to delete the kinds and namespaces you expect to clean up. When you use the [permissions controller](/docs/kubernetes/targets/kubernetes-agent/granular-permissions) to scope permissions per step, deletions run under the reserved step slug `octopus-kubernetes-resource-deletion`. + +## Auditing + +Every deletion task writes one audit event to the **Audit** screen in **Configuration**, succeeded or failed, recorded against the user who requested it. Its details list every object with its outcome, grouped by deployment target, with failures first. + +## Known limitations + +- Only top-level orphaned objects can be deleted. Child objects, such as Pods belonging to a Deployment can't be deleted from Octopus to restart a service. +- You cannot stop tracking an orphaned object without deleting it. If the object is still needed by something else, delete it and recreate it from a different project. + +## Learn more + +- [Kubernetes Live Object Status](/docs/kubernetes/live-object-status) +- [Orphaned objects](/docs/kubernetes/live-object-status#orphaned-objects) +- [Troubleshooting Live Object Status](/docs/kubernetes/live-object-status/troubleshooting) +- [Kubernetes agent permissions controller](/docs/kubernetes/targets/kubernetes-agent/granular-permissions) diff --git a/src/pages/docs/kubernetes/live-object-status/index.md b/src/pages/docs/kubernetes/live-object-status/index.md index aaa4497327..a640e415a3 100644 --- a/src/pages/docs/kubernetes/live-object-status/index.md +++ b/src/pages/docs/kubernetes/live-object-status/index.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-03-28 -modDate: 2026-05-24 +modDate: 2026-08-21 navSection: Live Object Status title: Kubernetes Live Object Status navTitle: Overview @@ -84,25 +84,28 @@ Sync Status tracks whether the changes Octopus deployed still matches the resour | In Sync | | Object manifest matches what was applied | | Out of Sync | | Object manifest is not the same as what was applied | | Unknown | | We don't have information about the live status of this object | -| Orphaned | | Object was deployed in a previous release but is no longer part of the latest deployment process | +| Orphaned | | Object was deployed in a previous release but is no longer part of the latest deployment process | +| Deleting | | Octopus is deleting this orphaned object. Select the status to open the task doing the work | Take a look at our [troubleshooting guide](/docs/kubernetes/live-object-status/troubleshooting) for details on why you may see some object statuses ### Orphaned objects -When you deploy a project that no longer includes a resource that was deployed in a previous release (for example, you remove a resource from a YAML manifest or remove a step from the deployment process), Octopus marks the dropped resource as **Orphaned** in the Live Status table. +When a deployment succeeds and no longer includes an object that was deployed in a previous release (for example, you remove an object from a YAML manifest or remove a step from the deployment process), Octopus marks the dropped object as **Orphaned** in the Live Status table. Orphaned objects: - keep their existing Health Status, so Octopus continues tracking them while they remain in your cluster -- have **Orphaned** as their Sync Status (shown with the link-slash icon in the table above) -- are summarized by a total-orphans count shown on the project's Live Status page -- can be narrowed to by using the Live Status table's filter and selecting **Orphaned** +- have **Orphaned** as their Sync Status +- are excluded from the rolled-up Application Health Status and Application Sync Status +- can be deleted directly from Octopus, see [deleting orphaned objects](/docs/kubernetes/live-object-status/deleting-orphaned-objects) -The orphan state clears automatically on the next deployment that re-adds the resource. If you remove an orphaned object from your cluster directly (for example with `kubectl delete`), the Kubernetes monitor detects the removal and Octopus stops tracking the resource, so the orphan entry disappears from the Live Status table without any manual intervention. +Orphans are detected only after a **successful** deployment, so a failed deployment never marks anything as orphaned. + +Skipping a step, or disabling it, does not orphan the objects that step deploys. Octopus only orphans an object when the step that deployed it ran and no longer produces it, or when that step has been removed from the deployment process altogether. :::div{.info} -Orphaned-resource tracking requires every Kubernetes monitor in the application instance to be on agent version 2.38.3 or later (v2) / 3.0.1 or later (v3). On clusters with any older agent, the resource is silently removed from the Live Status table when it is dropped from a deployment, matching the previous behavior. +Orphaned-object tracking requires every Kubernetes monitor in the application instance to be on agent version 2.38.3 or later (v2) / 3.0.1 or later (v3). On clusters with any older agent, the object is silently removed from the Live Status table when it is dropped from a deployment, matching the previous behavior. ::: ### Detailed object information @@ -215,6 +218,8 @@ Viewing the data returned from the Kubernetes monitor from within Octopus requir This data includes the resource and application status, as well as pod logs and events for objects being monitored. This may be a change in security posture that your team should carefully consider. +Deleting an orphaned object additionally requires the `DeployedResourceAdminister` permission, and the Kubernetes agent's own service account must be allowed to delete the object. See [deleting orphaned objects](/docs/kubernetes/live-object-status/deleting-orphaned-objects#permissions). + ## Secrets ### Octopus sensitive variables @@ -256,11 +261,11 @@ This setting defaults to on for all projects, but may change in the future. ## Known issues and limitations -### Excluded steps +### Excluded steps on older agents -The desired object list is compiled from objects that were applied during the last deployment. If steps are excluded during a deployment, then live status will not be shown for objects that were applied in those steps. +Where every Kubernetes monitor in the application is on agent version 2.38.3 or later (v2) / 3.0.1 or later (v3), Octopus keeps showing live status for objects deployed by a step that was skipped or disabled in the latest deployment. -Please avoid skipping steps that deploy Kubernetes objects. +On clusters with any older agent, objects applied by excluded steps are still removed from the Live Status table, so avoid skipping steps that deploy Kubernetes objects until every agent is upgraded. ### Runbooks are not supported diff --git a/src/pages/docs/kubernetes/live-object-status/troubleshooting/index.md b/src/pages/docs/kubernetes/live-object-status/troubleshooting/index.md index 22190f2189..35d56ca8b9 100644 --- a/src/pages/docs/kubernetes/live-object-status/troubleshooting/index.md +++ b/src/pages/docs/kubernetes/live-object-status/troubleshooting/index.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-03-28 -modDate: 2026-01-21 +modDate: 2026-08-21 navTitle: Troubleshooting title: Troubleshooting navSection: Troubleshooting @@ -86,3 +86,33 @@ If possible, we recommend ensuring that - Octopus is the only entity to modify your deployments - You craft your Kubernetes manifests to ensure that there are no invalid fields + +### Why is an object not orphaned after a deployment? \{#object-not-orphaned} + +An object is marked [Orphaned](/docs/kubernetes/live-object-status#orphaned-objects) when a deployment succeeds and no longer deploys an object that a previous release deployed. Common causes are renaming an object, removing the object from a manifest, or removing the step that deployed it from the deployment process. + +Skipping or disabling a step does not orphan its objects. + +If you expected an object to be orphaned and it isn't, check that every Kubernetes monitor referenced by the project is on agent version 2.38.3 or later (v2) / 3.0.1 or later (v3). Orphan tracking is all-or-nothing for a given project/environment/tenant combination. + +### An object stays orphaned after I re-added the step \{#orphan-persists-after-re-adding-a-step} + +Octopus clears the orphan state when a deployment deploys the object again. You must deploy the release after re-adding a step that was removed. + +## Deleting orphaned objects \{#deleting-orphaned-objects} + +### Octopus says an object was deleted but it's still in my cluster \{#deleted-object-still-present} + +Deleting an object deletes its dependents in the background, so a successful deletion returns as soon as the object itself is gone. Kubernetes garbage collects what it owned afterwards, which means deleting a Deployment reports success while its ReplicaSets and Pods are still being removed. + +### A deletion failed \{#deletion-failed} + +Open the deletion task from the warning icon beside the object. The task log contains detailed error messages to help troubleshooting. + +The most common cause is cluster permissions. Deletions run on the Kubernetes agent using the agent's service account, so the agent needs `delete` on that kind and namespace. See [cluster permissions](/docs/kubernetes/live-object-status/deleting-orphaned-objects#cluster-permissions). + +### I can't delete an object because its status is stale \{#cannot-delete-stale-object} + +Octopus refuses to delete an object whose status information is stale, because it can't confirm what is actually in the cluster. Status goes stale after 10 minutes without an update from the Kubernetes monitor. + +Follow [failed to establish connection with Kubernetes monitor](/docs/kubernetes/live-object-status/troubleshooting#failed-to-establish-connection-with-kubernetes-monitor) to get the monitor reporting again, then retry the deletion. diff --git a/src/pages/docs/kubernetes/targets/kubernetes-agent/granular-permissions.md b/src/pages/docs/kubernetes/targets/kubernetes-agent/granular-permissions.md index 7f2bf8d689..4a72201f4a 100644 --- a/src/pages/docs/kubernetes/targets/kubernetes-agent/granular-permissions.md +++ b/src/pages/docs/kubernetes/targets/kubernetes-agent/granular-permissions.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-9-24 -modDate: 2025-9-24 +modDate: 2026-08-21 title: Granular Permissions description: Define Kubernetes RBAC for your Kubernetes agent deployment based on Octopus context navOrder: 22 @@ -119,6 +119,10 @@ Each field adheres to the following rules to match: Each `WorkloadServiceAccount` must have at least one non-empty field. You cannot have a `WorkloadServiceAccount` that matches every scope. +:::div{.info} +[Deleting an orphaned Kubernetes object](/docs/kubernetes/live-object-status/deleting-orphaned-objects) is not run by a deployment step, so it uses the reserved step slug `octopus-kubernetes-resource-deletion`. Use that slug in the `steps` scope to grant the `delete` permissions those cleanups need without widening what your deployments can do. +::: + ##### Permissions The permissions applied for each scope can be configured a couple of ways: diff --git a/src/pages/docs/security/users-and-teams/default-permissions.md b/src/pages/docs/security/users-and-teams/default-permissions.md index c38c46e33a..8056ed62bb 100644 --- a/src/pages/docs/security/users-and-teams/default-permissions.md +++ b/src/pages/docs/security/users-and-teams/default-permissions.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2023-01-01 -modDate: 2026-08-17 +modDate: 2026-08-19 title: Default permissions for built-in user roles description: A listing of the default permissions for each of the built-in user roles. --- @@ -161,6 +161,7 @@ description: A listing of the default permissions for each of the built-in user | CertificateView | View certificates | | DefectReport | Block a release from progressing to the next lifecycle phase | | DefectResolve | Unblock a release so it can progress to the next phase | +| DeployedResourceAdminister | Administer deployed resources managed by Octopus | | DeploymentView | View deployments | | EnvironmentView | View environments | | EventView | View Events, including access to the Audit screen | @@ -214,6 +215,7 @@ description: A listing of the default permissions for each of the built-in user | CertificateView | View certificates | | DefectReport | Block a release from progressing to the next lifecycle phase | | DefectResolve | Unblock a release so it can progress to the next phase | +| DeployedResourceAdminister | Administer deployed resources managed by Octopus | | DeploymentCreate | Deploy releases to target environments | | DeploymentView | View deployments | | EnvironmentView | View environments | @@ -309,6 +311,7 @@ description: A listing of the default permissions for each of the built-in user | CertificateView | View certificates | | DefectReport | Block a release from progressing to the next lifecycle phase | | DefectResolve | Unblock a release so it can progress to the next phase | +| DeployedResourceAdminister | Administer deployed resources managed by Octopus | | DeploymentView | View deployments | | EnvironmentView | View environments | | EventView | View Events, including access to the Audit screen | @@ -498,6 +501,7 @@ description: A listing of the default permissions for each of the built-in user | CertificateView | View certificates | | DefectReport | Block a release from progressing to the next lifecycle phase | | DefectResolve | Unblock a release so it can progress to the next phase | +| DeployedResourceAdminister | Administer deployed resources managed by Octopus | | DeploymentCreate | Deploy releases to target environments | | DeploymentDelete | Delete deployments | | DeploymentView | View deployments |