HOMELAB-1847: LD4.5b — alert intake: episode ledger + the ONE on-call sandbox #596

Merged

aaron merged 9 commits from plane/HOMELAB-1847-oncall into epic/LD4 2026-10-03 04:19:56 +00:00

Conversation 3 Commits 9 Files changed 50 +8892 -94

aaron commented 2026-10-03 03:52:08 +00:00

Owner

Copy link

Closes/configures HOMELAB-1847 (LD4.5b, epic LD4). Do not merge from the agent side — the prod half is E-adj.

Built vs brief

Brief item	Where
Episode model, per-group, fingerprint-set identity, append-not-mint, flap	episode/ (8 files + tests)
Episode ledger (async/durable, FileLedger 0600 + fsync)	episode/ledger.go
Severity-adjusted digest window, ONE issue per window, unowned/owned split, P1 window bypass	digest/
Ownership-aware suppression (open PR/ticket ⇒ P3, zero dispatch)	suppression/
mgmt-VM webhook intake on :8091 (bearer, 202/400/401/413/503), /healthz, window loop, P1 ACT page with the decision-ready package	alertintake/, cmd/alertintake/
claims fix — per-claimant release state + lease TTL by claim class	claims/policy.go, reaper.go, claimer.go
R-03 hygiene ticket for a silent episode claim, deduped on episode id	custodian/hygiene.go, reclaim.go
Registry R-28 + R-10 dep re-point; on-call identity (4 surfaces)	routines/registry.yaml, identity/example-registry.yaml
Lane binding oncall → ld-oncall	config.TemplateKindOncall, cmd/dispatcher/context_plane.go
Class table as data	lab-director/alerts/digest-classes.yaml
mgmt deploy artefacts	lab-director/alerts/deploy/alert-intake.{service,env.example}

The claims fix (the scout's sharpest finding)

claims.ReleaseState was global Todo and the dispatcher watches Todo, so a reaped episode ticket would be spawned as builder work. Now: class-resolved from the ticket's label (episode), LeaseTTLByClass (15m → 2h), ReleaseStateByClass (Todo → Backlog), DispatchableStatesByClass (Todo → Backlog so an episode is claimable from Backlog without making Backlog dispatchable). Unknown class names are refused at load, and the class-less path is byte-identical. Tests: claims/class_test.go (incl. TestReapedEpisodeClaimReturnsToBacklogNeverTodo, TestEpisodeReleaseStateIsNotDispatchable), custodian/hygiene_test.go.

Verify lines (mechanized)

1. alertintake.TestBurstAcrossTwoGroupsProducesOneDigestAndOneEpisodePerGroup (16 alerts, 2 groups → 1 digest, 2 episodes) + episode.TestOneGroupIsExactlyOneEpisode (N=1..64 → 1 row).
2. alertintake.TestClaimedEpisodeIsFilteredFromTheDigestUnownedList, custodian.TestSilentEpisodeClaimFilesExactlyOneHygieneTicket, custodian.TestRepeatedEpisodeReapAppendsObservationNotASecondTicket.
3. suppression.TestOpenFixPROnwardsZeroDispatch, alertintake.TestOpenFixPRSuppressesTheSecondDispatch (open PR ⇒ P3, 0 episode items minted, suppression event logged, digest says "do NOT dispatch").
4. alertintake.TestP1EscalatesImmediatelyAndBypassesTheWindow (ACT page + decision-ready package + escalation ledger row + immediate digest in the same request).

go vet ./... and go test -race ./... green in lab-director/dispatcher.

Deviations to note

• Write ordering: the brief numbers "ledger fsync before 202, then Plane"; the implementation does Plane → ledger → 202. Both satisfy "202 only after a durable append"; this order is chosen because every Plane write is read-before-write guarded on the episode id (a retry cannot double-mint) and because the stored row can then cite its ticket. Stated at length in alertintake/alertintake.go.
• Episode partition follows the orchestrator adjudication (per notification group), not the brief's partition tuple. Row.Groups[] is retained for auditability.
• compiler.Roles() grew to 15 (on-call). The identity registry needs the role for R-28's runsAs, and a page signed by an unattributable identity is not attributable (§8). Role count/pin tests updated in the same commit.
• R-03's in-process pin re-pinned (custodian/reclaim.go) — required by TestInProcessPinsMatchTheCode.
• No context-role per-key binding (design R2 mitigation (a)); v1 uses the documented fallback (b): the digest body carries the on-call instructions and role-lore/on-call.md ships the charter. Flagged as a follow-up.
• No SandboxWarmPool and no new CNP for ld-oncall: both deliberate. Until a CNP exists, ld-oncall pods are not covered by ld-sandbox-class (it selects app.kubernetes.io/name), so merge order matters — see the prod PR.
• Forgejo read requires a read-only token (FORGEJO_TOKEN); the intake fails closed without it.
• Not built: the JetStream mirror for fleet.ledger.alert.> (mgmt cannot reach the bus; the publisher is the in-cluster dispatcher — named follow-up), and the mgmt systemd unit + ufw rule are committed as artefacts but not applied (host config).

E-adj (operator review) — all in the prod PR

Alertmanager receiver + continue: true route (monitoring), the new alert-intake-token secret, the ld-oncall SandboxTemplate, the mgmt systemd unit + ufw 8091 rule, and (deferred, flagged) a CNP for the ld-oncall class.

Shared hunks

claims/ (per-class release state), config/config.go (claim-class keys + TemplateKindOncall), routines/registry.yaml (my entries only: R-03 description+pin, R-10 pausedReason, R-28), identity/example-registry.yaml + compiler/roles.go.

Closes/configures HOMELAB-1847 (LD4.5b, epic LD4). **Do not merge from the agent side — the prod half is E-adj.** ## Built vs brief | Brief item | Where | |---|---| | Episode model, per-group, fingerprint-set identity, append-not-mint, flap | `episode/` (8 files + tests) | | Episode ledger (async/durable, FileLedger 0600 + fsync) | `episode/ledger.go` | | Severity-adjusted digest window, ONE issue per window, unowned/owned split, **P1 window bypass** | `digest/` | | Ownership-aware suppression (open PR/ticket ⇒ P3, zero dispatch) | `suppression/` | | mgmt-VM webhook intake on :8091 (bearer, 202/400/401/413/503), `/healthz`, window loop, P1 ACT page with the decision-ready package | `alertintake/`, `cmd/alertintake/` | | **claims fix** — per-claimant release state + lease TTL by claim class | `claims/policy.go`, `reaper.go`, `claimer.go` | | R-03 hygiene ticket for a silent episode claim, deduped on episode id | `custodian/hygiene.go`, `reclaim.go` | | Registry R-28 + R-10 dep re-point; `on-call` identity (4 surfaces) | `routines/registry.yaml`, `identity/example-registry.yaml` | | Lane binding `oncall` → `ld-oncall` | `config.TemplateKindOncall`, `cmd/dispatcher/context_plane.go` | | Class table as data | `lab-director/alerts/digest-classes.yaml` | | mgmt deploy artefacts | `lab-director/alerts/deploy/alert-intake.{service,env.example}` | ## The claims fix (the scout's sharpest finding) `claims.ReleaseState` was global `Todo` and the dispatcher watches `Todo`, so a reaped **episode** ticket would be spawned as builder work. Now: class-resolved from the ticket's label (`episode`), `LeaseTTLByClass` (15m → **2h**), `ReleaseStateByClass` (`Todo` → **`Backlog`**), `DispatchableStatesByClass` (`Todo` → `Backlog` so an episode is claimable from Backlog without making Backlog dispatchable). Unknown class names are **refused at load**, and the class-less path is byte-identical. Tests: `claims/class_test.go` (incl. `TestReapedEpisodeClaimReturnsToBacklogNeverTodo`, `TestEpisodeReleaseStateIsNotDispatchable`), `custodian/hygiene_test.go`. ## Verify lines (mechanized) 1. `alertintake.TestBurstAcrossTwoGroupsProducesOneDigestAndOneEpisodePerGroup` (16 alerts, 2 groups → 1 digest, 2 episodes) + `episode.TestOneGroupIsExactlyOneEpisode` (N=1..64 → 1 row). 2. `alertintake.TestClaimedEpisodeIsFilteredFromTheDigestUnownedList`, `custodian.TestSilentEpisodeClaimFilesExactlyOneHygieneTicket`, `custodian.TestRepeatedEpisodeReapAppendsObservationNotASecondTicket`. 3. `suppression.TestOpenFixPROnwardsZeroDispatch`, `alertintake.TestOpenFixPRSuppressesTheSecondDispatch` (open PR ⇒ P3, 0 episode items minted, suppression event logged, digest says "do NOT dispatch"). 4. `alertintake.TestP1EscalatesImmediatelyAndBypassesTheWindow` (ACT page + decision-ready package + escalation ledger row + immediate digest in the same request). `go vet ./...` and `go test -race ./...` green in `lab-director/dispatcher`. ## Deviations to note - **Write ordering**: the brief numbers "ledger fsync before 202, then Plane"; the implementation does Plane → ledger → 202. Both satisfy "202 only after a durable append"; this order is chosen because every Plane write is read-before-write guarded on the episode id (a retry cannot double-mint) and because the stored row can then cite its ticket. Stated at length in `alertintake/alertintake.go`. - **Episode partition** follows the orchestrator adjudication (per notification group), not the brief's partition tuple. `Row.Groups[]` is retained for auditability. - **`compiler.Roles()` grew to 15** (`on-call`). The identity registry needs the role for R-28's `runsAs`, and a page signed by an unattributable identity is not attributable (§8). Role count/pin tests updated in the same commit. - **R-03's in-process pin re-pinned** (`custodian/reclaim.go`) — required by `TestInProcessPinsMatchTheCode`. - **No context-role per-key binding** (design R2 mitigation (a)); v1 uses the documented fallback (b): the digest body carries the on-call instructions and `role-lore/on-call.md` ships the charter. Flagged as a follow-up. - **No SandboxWarmPool and no new CNP for `ld-oncall`**: both deliberate. Until a CNP exists, `ld-oncall` pods are not covered by `ld-sandbox-class` (it selects `app.kubernetes.io/name`), so **merge order matters** — see the prod PR. - **Forgejo read requires a read-only token** (`FORGEJO_TOKEN`); the intake fails closed without it. - **Not built**: the JetStream mirror for `fleet.ledger.alert.>` (mgmt cannot reach the bus; the publisher is the in-cluster dispatcher — named follow-up), and the mgmt systemd unit + ufw rule are committed as artefacts but not applied (host config). ## E-adj (operator review) — all in the prod PR Alertmanager receiver + `continue: true` route (`monitoring`), the new `alert-intake-token` secret, the `ld-oncall` SandboxTemplate, the mgmt systemd unit + ufw 8091 rule, and (deferred, flagged) a CNP for the `ld-oncall` class. ## Shared hunks `claims/` (per-class release state), `config/config.go` (claim-class keys + `TemplateKindOncall`), `routines/registry.yaml` (my entries only: R-03 description+pin, R-10 pausedReason, R-28), `identity/example-registry.yaml` + `compiler/roles.go`.

aaron added 8 commits 2026-10-03 03:52:08 +00:00

HOMELAB-1847: feat(episode): per-group alert episodes, fingerprint-set identity, flap + ledger 004e297941

HOMELAB-1847: feat(digest): severity-adjusted window, one digest per window, unowned/owned split f47abab22d

HOMELAB-1847: feat(suppression): ownership-aware zero-redispatch invariant (open PR/ticket => P3, append only) 8cca3a3a4c

HOMELAB-1847: fix(claims): per-claimant release state and lease TTL by claim class ... 4293d87b7e

An alert episode is claimed from Backlog and released back to Backlog. The
previous global ReleaseState (Todo) is the dispatcher's dispatchable state, so a
reaped episode ticket was spawned as builder work — the sharpest hazard in the
ticket. Class resolution is label-driven, unknown classes are refused at load
time, and the class-less path is byte-identical (15m lease, release to Todo).

HOMELAB-1847: feat(custodian): R-03 hygiene ticket for a reaped episode claim, deduped on the episode id ... e41692df67

The reaper releases the lease; R-03 (the reclaim driver, already active in the
registry) files the finding. Dedup is the point: a flapping claim appends one
observation to one ticket rather than minting a ticket per reap. A non-episode
reap files nothing, and a hygiene write failure does not fail the release.

HOMELAB-1847: feat(alertintake): mgmt-VM webhook intake, episode ledger, digest window, P1 window bypass ... 377d444c9a

The component: an HTTP receiver for the Alertmanager v4 payload behind the
existing rail, the episode ledger writer (fsync before 202), the ownership-aware
suppression read, the severity-adjusted digest window (one issue per window), and
the P1 immediate path. 202/400/401/413/503 semantics copied from
pi-alert-receiver, because Alertmanager's retry behaviour is defined by them.

HOMELAB-1847: feat(lane): oncall template key routes alert digests to ld-oncall; R-28 registered; on-call identity activated ... 5c30453715

- config.TemplateKindOncall + laneForIssue routes marker ld:alert-digest to it
- R-28 oncall-window registered (paused dep:routine-executor, like R-25); R-10's
  stale dep re-pointed at 4.5c's disposition enum (the ledger is 4.5b's)
- identity registry: on-call active with four surface bindings, no cluster cred
- compiler roles extended with on-call; R-03's in-process pin re-pinned

HOMELAB-1847: docs(role-lore): the on-call charter (durable position, not distilled records) 80b2667d3c

aaron referenced this pull request 2026-10-03 04:02:33 +00:00

HOMELAB-1690: feat(friction): the L7 mechanism — ld.friction/v1, ld.proposal/v1 state machine, shadow bindings, R-28 triage sink #592

aaron force-pushed plane/HOMELAB-1847-oncall from 80b2667d3c to ca94541f95 2026-10-03 04:02:55 +00:00 Compare

aaron referenced this pull request 2026-10-03 04:07:55 +00:00

HOMELAB-1690: feat(friction): the L7 mechanism — ld.friction/v1, ld.proposal/v1 state machine, shadow bindings, R-28 triage sink #592

aaron commented 2026-10-03 04:11:07 +00:00

Author

Owner

Copy link

Landing review — VERDICT: APPROVE (merge)

Reviewed head: ca94541f9570c9da75f4194621feaf19de0fc2a6. Fresh-context review; primary sources only (PR diff, HOMELAB-1847's four Verify lines, the repo's claims/registry/digest contracts). No prior PR comments existed, so no open blocker findings.

Rebase (stale base corrected)

The branch was cut before #594 (LD4.5d, R-29) merged, so its diff vs epic/LD4 removed the R-29 section. Rebased onto origin/epic/LD4 (2607c6e) by the landing worker. Git auto-merged routines/registry.yaml into the required union:

• R-29 (phase-census) section retained;
• R-28 (oncall-window) entry added (paused for dep:routine-executor);
• header kept from epic/LD4 (documents R-28 as LD4.5b's reservation, R-29, and the corpus/R-24/R-25 entries).

One follow-up commit ca94541: the merged header claimed "Active today (6)" counting R-28, but R-28 ships status: paused — corrected to (5) with R-28 named beside R-25 in the paused sentence. Force-with-lease from 80b2667 → ca94541. Ticket state not moved.

Verification on the new head

• go build ./... clean, go vet ./... clean.
• go test -race -count=1 ./... green across the whole module (incl. routines/identity/config/compiler).
• All four Verify lines' named tests PASS, plus the claims-fix invariants:
  1. alertintake.TestBurstAcrossTwoGroupsProducesOneDigestAndOneEpisodePerGroup (16 alerts/2 groups → 1 digest, 1 episode/group), episode.TestOneGroupIsExactlyOneEpisode.
  2. digest.TestClaimedEpisodeIsFilteredFromUnowned, alertintake.TestClaimedEpisodeIsFilteredFromTheDigestUnownedList, custodian.TestSilentEpisodeClaimFilesExactlyOneHygieneTicket, custodian.TestRepeatedEpisodeReapAppendsObservationNotASecondTicket, claims.TestReapedEpisodeClaimReturnsToBacklogNeverTodo, claims.TestEpisodeReleaseStateIsNotDispatchable.
  3. suppression.TestOpenFixPROnwardsZeroDispatch, alertintake.TestOpenFixPRSuppressesTheSecondDispatch.
  4. alertintake.TestP1EscalatesImmediatelyAndBypassesTheWindow.
  • alertintake.TestLedgerFailureAnswers503AndWritesNoRows (202 only after the durable append).

Review dimensions

• Correctness vs AC: all four Verify lines are mechanized and asserted on behaviour (episode ledger rows, digest plan lists, event ledger, counters), not implementation details.
• Claims fix (the sharpest hazard): per-class resolution is label-driven (episode), LeaseTTLByClass 15m→2h, ReleaseStateByClass Todo→Backlog, DispatchableStatesByClass Todo→Backlog. Backlog is claimable but NOT in dispatchableStates, so a reaped episode cannot be spawned as builder work — asserted by an explicit test. Unknown class names are refused at load; the class-less path is unchanged (TestClassLessTicketKeepsTodaysBehaviour). Writer scope is derived from the config (AllowedClaimStates) so policy and writer cannot disagree at startup.
• 202 ordering (builder's argued deviation): Plane→ledger→202. Both orders satisfy "202 only after a durable append"; the read-before-write guard on every episode/digest Plane write makes a retry non-duplicating. Accepted.
• Codebase fit: reuses direction.WindowKey, episode.Ledger, claims, notifier, comms. Registry pin re-derived by TestInProcessPinsMatchTheCode (green).
• Security: intake binds 10.10.10.5:8091 (VLAN-10 only), bearer token in constant time, fails closed on any missing credential; Forgejo uses a read-only token and counts only OPEN PRs. No new ServiceAccount — the dispatcher's existing Plane writer is reused (interface is the blast-radius boundary).
• Territory: rollup.go untouched; no new SA/RBAC/CNP/secret in this repo. Changes are lab-director/dispatcher/* plus the lab-director/alerts/ data/artefact files. No E-adj in this PR (E-adj lives in prod #887).
• Claim honesty: the diff does what the ticket says; deviations are documented in-code and in the PR body.

Non-blocking findings

• Merge-order hazard (flagged on both PRs, accepted here): ld-oncall pods are not covered by the ld-sandbox-class CNP yet. Merging infra-core first is inert (no unit applied, no Alertmanager receiver) and moves the dispatcher image/binary; the prod PR carries the SandboxTemplate + continue: true route. A CNP for the class remains a named follow-up.
• The commitment "ledger is the durable step before 202" is satisfied, but the P1 escalation + immediate digest run before the 202 is written; a rail/window failure is logged and the 202 still stands (episode recorded). Correct, noted.

Sibling

#592 (R-28→R-30 re-id) had NOT landed on epic/LD4 at review time, so no R-30 in the registry totals.

## Landing review — VERDICT: APPROVE (merge) **Reviewed head: `ca94541f9570c9da75f4194621feaf19de0fc2a6`.** Fresh-context review; primary sources only (PR diff, HOMELAB-1847's four Verify lines, the repo's claims/registry/digest contracts). No prior PR comments existed, so no open blocker findings. ### Rebase (stale base corrected) The branch was cut before #594 (LD4.5d, R-29) merged, so its diff vs `epic/LD4` *removed* the R-29 section. Rebased onto `origin/epic/LD4` (`2607c6e`) by the landing worker. Git auto-merged `routines/registry.yaml` into the required union: - R-29 (`phase-census`) section retained; - R-28 (`oncall-window`) entry added (paused for `dep:routine-executor`); - header kept from `epic/LD4` (documents R-28 as LD4.5b's reservation, R-29, and the corpus/R-24/R-25 entries). One follow-up commit `ca94541`: the merged header claimed "Active today (6)" counting R-28, but R-28 ships `status: paused` — corrected to **(5)** with R-28 named beside R-25 in the paused sentence. Force-with-lease from `80b2667` → `ca94541`. Ticket state not moved. ### Verification on the new head - `go build ./...` clean, `go vet ./...` clean. - `go test -race -count=1 ./...` **green across the whole module** (incl. routines/identity/config/compiler). - All four Verify lines' named tests PASS, plus the claims-fix invariants: 1. `alertintake.TestBurstAcrossTwoGroupsProducesOneDigestAndOneEpisodePerGroup` (16 alerts/2 groups → 1 digest, 1 episode/group), `episode.TestOneGroupIsExactlyOneEpisode`. 2. `digest.TestClaimedEpisodeIsFilteredFromUnowned`, `alertintake.TestClaimedEpisodeIsFilteredFromTheDigestUnownedList`, `custodian.TestSilentEpisodeClaimFilesExactlyOneHygieneTicket`, `custodian.TestRepeatedEpisodeReapAppendsObservationNotASecondTicket`, `claims.TestReapedEpisodeClaimReturnsToBacklogNeverTodo`, `claims.TestEpisodeReleaseStateIsNotDispatchable`. 3. `suppression.TestOpenFixPROnwardsZeroDispatch`, `alertintake.TestOpenFixPRSuppressesTheSecondDispatch`. 4. `alertintake.TestP1EscalatesImmediatelyAndBypassesTheWindow`. - `alertintake.TestLedgerFailureAnswers503AndWritesNoRows` (202 only after the durable append). ### Review dimensions - **Correctness vs AC**: all four Verify lines are mechanized and asserted on behaviour (episode ledger rows, digest plan lists, event ledger, counters), not implementation details. - **Claims fix (the sharpest hazard)**: per-class resolution is label-driven (`episode`), `LeaseTTLByClass` 15m→2h, `ReleaseStateByClass` Todo→**Backlog**, `DispatchableStatesByClass` Todo→Backlog. Backlog is claimable but NOT in `dispatchableStates`, so a reaped episode cannot be spawned as builder work — asserted by an explicit test. Unknown class names are refused at load; the class-less path is unchanged (`TestClassLessTicketKeepsTodaysBehaviour`). Writer scope is derived from the config (`AllowedClaimStates`) so policy and writer cannot disagree at startup. - **202 ordering (builder's argued deviation)**: Plane→ledger→202. Both orders satisfy "202 only after a durable append"; the read-before-write guard on every episode/digest Plane write makes a retry non-duplicating. Accepted. - **Codebase fit**: reuses `direction.WindowKey`, `episode.Ledger`, `claims`, `notifier`, `comms`. Registry pin re-derived by `TestInProcessPinsMatchTheCode` (green). - **Security**: intake binds `10.10.10.5:8091` (VLAN-10 only), bearer token in constant time, fails closed on any missing credential; Forgejo uses a read-only token and counts only OPEN PRs. No new ServiceAccount — the dispatcher's existing Plane writer is reused (interface is the blast-radius boundary). - **Territory**: `rollup.go` untouched; no new SA/RBAC/CNP/secret in this repo. Changes are `lab-director/dispatcher/*` plus the `lab-director/alerts/` data/artefact files. No E-adj in this PR (E-adj lives in prod #887). - **Claim honesty**: the diff does what the ticket says; deviations are documented in-code and in the PR body. ### Non-blocking findings - **Merge-order hazard (flagged on both PRs, accepted here)**: `ld-oncall` pods are not covered by the `ld-sandbox-class` CNP yet. Merging infra-core first is inert (no unit applied, no Alertmanager receiver) and moves the dispatcher image/binary; the prod PR carries the SandboxTemplate + `continue: true` route. A CNP for the class remains a named follow-up. - The commitment "ledger is the durable step before 202" is satisfied, but the P1 escalation + immediate digest run before the 202 is written; a rail/window failure is logged and the 202 still stands (episode recorded). Correct, noted. ### Sibling #592 (R-28→R-30 re-id) had NOT landed on `epic/LD4` at review time, so no R-30 in the registry totals.

aaron force-pushed plane/HOMELAB-1847-oncall from ca94541f95 to 86316feac5 2026-10-03 04:13:53 +00:00 Compare

aaron commented 2026-10-03 04:19:44 +00:00

Author

Owner

Copy link

Landing review (updated) — VERDICT: APPROVE (merge)

Reviewed head: 86316feac522862451a5748e46b20228e7f162f1. This supersedes the earlier verdict; the review below re-validates that head after the second rebase.

Re-rebase note

While the first landing attempt was in flight, #592 (HOMELAB-1690, LD4.6) landed on epic/LD4 (8d8cbd7) and re-id'd its L7 triage sink R-28 → R-30 (R-28 is 4.5b's reservation). The branch was re-based onto 8d8cbd7. The routines/registry.yaml conflict was resolved as a union of all three entries:

• R-28 oncall-window (this PR, paused — dep:routine-executor) at line 791;
• R-29 phase-census (#594) retained at line 707;
• R-30 friction-triage (#592, paused) retained at line 849;
• header takes epic/LD4's reserved-ids paragraph (now naming R-28/R-29/R-30) and corrects "Active today" from (6) to (5) — R-28 and R-30 both ship paused, so neither is active.

The diff vs epic/LD4 is now purely additive for #594/#592's entries (no R-29/R-30 removal). epic/LD4 is an ancestor of the head; Mergeable: true. Force-with-lease ca94541 → 86316fe.

Verification on 86316fe

• go build ./... + go vet ./... clean.
• go test -race -count=1 ./... green across the whole module (all 28 packages; includes dispatcher/friction/ from #592).
• All four Verify-line named tests PASS plus the claims/hygiene invariants:
  1. TestBurstAcrossTwoGroupsProducesOneDigestAndOneEpisodePerGroup, TestOneGroupIsExactlyOneEpisode.
  2. TestClaimedEpisodeIsFilteredFromTheDigestUnownedList, TestSilentEpisodeClaimFilesExactlyOneHygieneTicket, TestRepeatedEpisodeReapAppendsObservationNotASecondTicket, TestReapedEpisodeClaimReturnsToBacklogNeverTodo, TestEpisodeReleaseStateIsNotDispatchable.
  3. TestOpenFixPROnwardsZeroDispatch, TestOpenFixPRSuppressesTheSecondDispatch.
  4. TestP1EscalatesImmediatelyAndBypassesTheWindow.
  • TestLedgerFailureAnswers503AndWritesNoRows.
• CI run #1793 on this head: all four checks success (alert-receiver-test, dispatcher-test, helm-validate, receiver-test).

Review dimensions (unchanged from the earlier verdict; re-checked on the rebased content)

Correctness vs the four ACs; claims per-class release fix (episode → Backlog, non-dispatchable, asserted); 202-after-durable-append ordering accepted (the argued Plane→ledger→202 deviation); codebase fit (reuses direction.WindowKey/episode.Ledger/claims/notifier/comms); test adequacy (behavioural asserts, not implementation); security (VLAN-10 bind, constant-time bearer, fail-closed credentials, read-only Forgejo token, no new SA/RBAC/CNP/secret); territory (rollup.go untouched; code under lab-director/dispatcher/ + lab-director/alerts/ artefacts only; no E-adj in this PR).

Non-blocking findings

• Merge-order hazard (flagged on both PRs): ld-oncall is not yet covered by the ld-sandbox-class CNP. Landing infra-core first is inert at runtime (no host unit applied, no Alertmanager receiver); the prod PR (#887) carries the SandboxTemplate + continue: true route. A CNP for the class remains a named follow-up, as does the JetStream mirror.
• The P1 escalation + immediate digest run before the 202 is written; a rail/window failure is logged and the 202 stands with the episode recorded. Correct and intentional.

No open blocker findings.

## Landing review (updated) — VERDICT: APPROVE (merge) **Reviewed head: `86316feac522862451a5748e46b20228e7f162f1`.** This supersedes the earlier verdict; the review below re-validates that head after the second rebase. ### Re-rebase note While the first landing attempt was in flight, **#592 (HOMELAB-1690, LD4.6) landed on `epic/LD4` (`8d8cbd7`) and re-id'd its L7 triage sink R-28 → R-30** (R-28 is 4.5b's reservation). The branch was re-based onto `8d8cbd7`. The `routines/registry.yaml` conflict was resolved as a union of **all three** entries: - **R-28** `oncall-window` (this PR, `paused` — `dep:routine-executor`) at line 791; - **R-29** `phase-census` (#594) retained at line 707; - **R-30** `friction-triage` (#592, `paused`) retained at line 849; - header takes `epic/LD4`'s reserved-ids paragraph (now naming R-28/R-29/**R-30**) and corrects "Active today" from (6) to **(5)** — R-28 and R-30 both ship paused, so neither is active. The diff vs `epic/LD4` is now purely additive for #594/#592's entries (no R-29/R-30 removal). `epic/LD4` is an ancestor of the head; `Mergeable: true`. Force-with-lease `ca94541` → `86316fe`. ### Verification on `86316fe` - `go build ./...` + `go vet ./...` clean. - `go test -race -count=1 ./...` **green across the whole module** (all 28 packages; includes `dispatcher/friction/` from #592). - All four Verify-line named tests PASS plus the claims/hygiene invariants: 1. `TestBurstAcrossTwoGroupsProducesOneDigestAndOneEpisodePerGroup`, `TestOneGroupIsExactlyOneEpisode`. 2. `TestClaimedEpisodeIsFilteredFromTheDigestUnownedList`, `TestSilentEpisodeClaimFilesExactlyOneHygieneTicket`, `TestRepeatedEpisodeReapAppendsObservationNotASecondTicket`, `TestReapedEpisodeClaimReturnsToBacklogNeverTodo`, `TestEpisodeReleaseStateIsNotDispatchable`. 3. `TestOpenFixPROnwardsZeroDispatch`, `TestOpenFixPRSuppressesTheSecondDispatch`. 4. `TestP1EscalatesImmediatelyAndBypassesTheWindow`. - `TestLedgerFailureAnswers503AndWritesNoRows`. - CI run #1793 on this head: all four checks success (alert-receiver-test, dispatcher-test, helm-validate, receiver-test). ### Review dimensions (unchanged from the earlier verdict; re-checked on the rebased content) Correctness vs the four ACs; claims per-class release fix (episode → Backlog, non-dispatchable, asserted); 202-after-durable-append ordering accepted (the argued Plane→ledger→202 deviation); codebase fit (reuses `direction.WindowKey`/`episode.Ledger`/`claims`/`notifier`/`comms`); test adequacy (behavioural asserts, not implementation); security (VLAN-10 bind, constant-time bearer, fail-closed credentials, read-only Forgejo token, no new SA/RBAC/CNP/secret); territory (`rollup.go` untouched; code under `lab-director/dispatcher/` + `lab-director/alerts/` artefacts only; no E-adj in this PR). ### Non-blocking findings - **Merge-order hazard** (flagged on both PRs): `ld-oncall` is not yet covered by the `ld-sandbox-class` CNP. Landing infra-core first is inert at runtime (no host unit applied, no Alertmanager receiver); the prod PR (#887) carries the SandboxTemplate + `continue: true` route. A CNP for the class remains a named follow-up, as does the JetStream mirror. - The P1 escalation + immediate digest run before the 202 is written; a rail/window failure is logged and the 202 stands with the episode recorded. Correct and intentional. No open blocker findings.

aaron merged commit 7496e07fe9 into epic/LD4 2026-10-03 04:19:56 +00:00

aaron referenced this pull request from a commit 2026-10-03 04:19:56 +00:00

Merge pull request 'HOMELAB-1847: LD4.5b — alert intake: episode ledger + the ONE on-call sandbox' (#596) from plane/HOMELAB-1847-oncall into epic/LD4

atlantis commented 2026-10-03 04:19:57 +00:00

Collaborator

Copy link

Plan Error

running git clone --depth=1 --branch plane/HOMELAB-1847-oncall --single-branch https://atlantis:<redacted>@forgejo.aaron.reynoza.org/Homelab/infra-core.git /atlantis-data/repos/Homelab/infra-core/596/default: Cloning into '/atlantis-data/repos/Homelab/infra-core/596/default'...
fatal: could not open '/atlantis-data/repos/Homelab/infra-core/596/default/.git/objects/pack/tmp_pack_FOlsfu' for reading: No such file or directory
fatal: fetch-pack: invalid index-pack output
: exit status 128

**Plan Error** ``` running git clone --depth=1 --branch plane/HOMELAB-1847-oncall --single-branch https://atlantis:<redacted>@forgejo.aaron.reynoza.org/Homelab/infra-core.git /atlantis-data/repos/Homelab/infra-core/596/default: Cloning into '/atlantis-data/repos/Homelab/infra-core/596/default'... fatal: could not open '/atlantis-data/repos/Homelab/infra-core/596/default/.git/objects/pack/tmp_pack_FOlsfu' for reading: No such file or directory fatal: fetch-pack: invalid index-pack output : exit status 128 ```

aaron referenced this pull request 2026-10-03 05:18:58 +00:00

HOMELAB-1731: LD4.11 · maintainability fitness routine (weekly, Sendrast first) #598

aaron referenced this pull request 2026-10-03 18:17:54 +00:00

LD4 (HOMELAB-1713): The Loops — intake, El Director, routines, readiness, plan-freshness, meetings, alert audit → live #591

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

operator-review

Requires human operator review before merge — dangerous class (RBAC, secrets, network policy, Terraform) or explicitly gated. Never auto-merge.

No labels

operator-review

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Homelab/infra-core!596

No description provided.