HOMELAB-973: fix(outline): add startupProbe to prevent liveness race #184

Merged

aaron merged 1 commit from plane/HOMELAB-973-outline-probes into live

2026-04-24 21:23:47 +00:00

aaron commented

2026-04-24 21:20:09 +00:00

Owner

Summary

Root cause: Outline's startup (migrations + throng worker fork + plugin reload + port bind) often exceeds the liveness probe's ~60s window on cold boot, triggering SIGTERM mid-init and exit 0 (graceful shutdown).
Fix: add startupProbe to core/charts/apps/outline/templates/deployment.yaml with a 5-minute window (30 × 10s failureThreshold). Liveness/readiness are suppressed by Kubernetes until the startup probe succeeds.
Verified template renders cleanly with helm template against prod values.

Test plan

ArgoCD sync (blocked on HOMELAB-972 Forgejo↔ArgoCD connectivity)
Outline pod reaches 1/1 Running with stable readiness
curl https://docs.aaron.reynoza.org/_health returns 200

Refs HOMELAB-973.

## Summary - Root cause: Outline's startup (migrations + throng worker fork + plugin reload + port bind) often exceeds the liveness probe's ~60s window on cold boot, triggering SIGTERM mid-init and exit 0 (graceful shutdown). - Fix: add `startupProbe` to `core/charts/apps/outline/templates/deployment.yaml` with a 5-minute window (30 × 10s failureThreshold). Liveness/readiness are suppressed by Kubernetes until the startup probe succeeds. - Verified template renders cleanly with `helm template` against prod values. ## Test plan - [ ] ArgoCD sync (blocked on HOMELAB-972 Forgejo↔ArgoCD connectivity) - [ ] Outline pod reaches 1/1 Running with stable readiness - [ ] `curl https://docs.aaron.reynoza.org/_health` returns 200 Refs HOMELAB-973.

aaron added 1 commit

2026-04-24 21:20:09 +00:00

HOMELAB-973: fix(outline): add startupProbe to prevent liveness race during migrations

CI Review / pr-title (pull_request) Has been cancelled

Details

Lint & Validate / terraform-validate (pull_request) Has been cancelled

Details

Lint & Validate / yaml-lint (pull_request) Has been cancelled

Details

Lint & Validate / shellcheck (pull_request) Has been cancelled

Details

CI Review / helm-validate (pull_request) Has been cancelled

Details

CI Review / ai-review (pull_request) Has been cancelled

Details

9407a0aad5

Outline's master process runs sequelize migrations + data-migration
checks + sponsor message before throng forks a worker that reloads
plugins and binds port 3000. On a cold boot this easily takes 45–90s.

The previous probe config (liveness initialDelay=30s, readiness
initialDelay=10s, both 3×10s failureThreshold) triggered SIGTERM
mid-worker-fork, causing the container to exit 0 (graceful shutdown)
and CrashLoopBackOff repeatedly without the web service ever listening.

Add a startupProbe with a 5-minute window (30 × 10s failureThreshold)
so Kubernetes suppresses liveness/readiness until the server actually
listens. Once startupProbe succeeds, liveness/readiness take over with
short periods to catch real post-startup failures fast.

Root cause diagnosed from pod logs: OIDC plugin registered → migrations
run → sponsor message → exit 0 (never reached "Listening on port 3000").