Liveness vs Readiness Probes
Cloud infrastructurecloud infrastructure / Kubernetes
Two probes, almost identical YAML, completely different jobs. Mixing them up is one of the most common ways a healthy service ends up in a CrashLoopBackOff death spiral.
The core difference
- Liveness = "Is this container BROKEN and unrecoverable?" → if it fails, k8s KILLS and RESTARTS the pod.
- Readiness = "Can this container serve traffic RIGHT NOW?" → if it fails, k8s pulls it out of the Service load balancer but LEAVES IT RUNNING.
Same check syntax, opposite consequence. One reboots, one just stops sending requests.
What they look like
livenessProbe:
httpGet: { path: /healthz, port: 8080 }
initialDelaySeconds: 10
periodSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet: { path: /ready, port: 8080 }
periodSeconds: 5
Note the DIFFERENT paths. /healthz should be cheap and answer "the process is alive." /ready should answer "deps are warm — DB pool connected, cache loaded, migrations done."
The classic outage
You point liveness at /ready, and /ready checks the database. The DB has a 30s hiccup. Now every pod fails liveness → k8s restarts ALL of them at once → cold pods hammer the recovering DB → they fail readiness too → nothing serves traffic. A 30-second blip became a full outage. The fix: liveness must NOT depend on downstream services. Only readiness should.
The startup trap
Slow-booting app (JVM, big model load)? If liveness fires before boot finishes, the pod gets killed mid-startup — forever. Don't just crank initialDelaySeconds (it delays real crash detection too). Use a startupProbe: it disables liveness/readiness until the app is up, then hands off.
startupProbe:
httpGet: { path: /healthz, port: 8080 }
failureThreshold: 30
periodSeconds: 10 # allows up to 300s to boot
Quick rules
- Liveness: cheap, local, no external deps. Only fail when a RESTART would actually help.
- Readiness: check the things that make THIS instance unable to serve (deps, warmup).
- Slow boot: add a startupProbe instead of inflating delays.
- A failing liveness reboots; a failing readiness just quietly drains. Pick based on which one you want.
Your exercise
Your app exposes ONE /health endpoint that returns 200 only when the Postgres connection works, and you've wired it to BOTH probes. Postgres goes down for 2 minutes. Walk through what k8s does to your pods second by second — and rewrite the probe setup so the same outage degrades gracefully instead of cascading. What goes in /healthz vs /ready?