Dev Tools · 1h ago
Kubernetes controllers silently fail on stale tokens, fix requires custom handler
Kubernetes controllers built on client-go can silently fail when ServiceAccount tokens expire, continuing to show Ready status while doing nothing. The default watch error handler retries 401 Unauthorized errors indefinitely, masking the failure. A fix involves wrapping the HTTP transport to detect and exit on 401s, but controller-runtime's custom handler is opt-in.
Meridian48 take
This is a classic silent failure that undermines Kubernetes' reliability promise, and the fact that the fix is opt-in means many clusters remain vulnerable.
Read the full reporting
Your Kubernetes Controller Is Silently Dead and Nobody Knows →
DEV Community
kubernetesclient-go