Skip to content

Orphaned kubernetes pods when api times out #596

Description

@oker1

I've found an old pod that was started by shinyproxy and it has not seen a user request but still ran an outdated spec. I've checked the logs and it seems like shinyproxy had some kubernetes api timeouts while starting the pod and then gave up on it. I would expect these pods to be cleaned up since the instances are saved in redis and they could be compared with a label based listing of pods to find orphans.

2026-06-22T15:50:34.418Z  INFO 1 --- [haringScaler-16] e.o.c.b.kubernetes.KubernetesBackend     : [user=null proxyId=f7c7d365-f6e8-47a8-8235-24ea4539c8d7 specId=test] Waiting: Kubernetes Pod (39/40)
2026-06-22T15:50:34.418Z  WARN 1 --- [haringScaler-16] e.o.c.b.kubernetes.KubernetesBackend     : [user=null proxyId=f7c7d365-f6e8-47a8-8235-24ea4539c8d7 specId=test] Failed: Kubernetes Pod
io.fabric8.kubernetes.client.KubernetesClientException: The timeout period of 10000ms has been exceeded while executing GET /api/v1/namespaces/ns/pods/test-app-f7c7d365-f6e8-47a8-8235-24ea4539c8d7 for server null
Caused by: io.vertx.core.impl.NoStackTraceTimeoutException: The timeout period of 10000ms has been exceeded while executing GET /api/v1/namespaces/ns/pods/test-app-f7c7d365-f6e8-47a8-8235-24ea4539c8d7 for server null
2026-06-22T15:50:34.428Z ERROR 1 --- [haringScaler-16] e.o.c.b.d.p.ProxySharingScaler           : [delegateProxyId=f7c7d365-f6e8-47a8-8235-24ea4539c8d7 specId=test] Failed to start DelegateProxy
2026-06-22T15:50:44.431Z ERROR 1 --- [haringScaler-16] e.o.c.b.d.p.ProxySharingScaler           : [delegateProxyId=f7c7d365-f6e8-47a8-8235-24ea4539c8d7 specId=test] Error while stopping failed DelegateProxy
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: The timeout period of 10000ms has been exceeded while executing GET /api/v1/namespaces/ns/pods/test-app-f7c7d365-f6e8-47a8-8235-24ea4539c8d7 for server null
Caused by: io.vertx.core.impl.NoStackTraceTimeoutException: The timeout period of 10000ms has been exceeded while executing GET /api/v1/namespaces/ns/pods/test-app-f7c7d365-f6e8-47a8-8235-24ea4539c8d7 for server null
2026-06-22T15:50:46.221Z  INFO 1 --- [GlobalEventLoop] e.o.c.b.d.p.ProxySharingScaler           : [specId=test delegateProxyId=f7c7d365-f6e8-47a8-8235-24ea4539c8d7] Stopping DelegateProxy

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions