ld-spec-024: the seat template bans reading a PR's comments or reviews before the verdict (+ the lab-director switch) #937
No reviewers
Labels
No labels
operator-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Homelab/infra-core!937
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "ld/LR-20261111-024-seat-ban"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Spec 024 / LR-20261111-024 — seats never read the PR's comments or reviews: the ban, in the seat template (Homelab/infra-core).
Class: ordinary, non-prod skill text.
Done when (fixed flows)
ld-implement.SKILL.md's seat prompt template gains one required line, right after the<POSTING>line, byte for byte: "Do NOT list or read this PR's comments or reviews before posting your verdict; judge the diff, tests and guard only." Proof: diff hunk incore/manifests/apps/sbx-common-phase1/ld-implement.SKILL.md(the line directly after<POSTING>in the seat prompt fence); verified byte-for-byte against the spec text by the reviewer round. Prevents: #934 seat B4 listed/issues/934/commentsand read seat A's verdict (comment 24423) before posting its own — anchoring (count-only check, 11-11 11:35Z)./pulls/{n}/reviews, never a seat's; two formal reviews on one head is a deviation, 11-11 #299).sha256sum -c SHA256SUMSpasses, 11 files); the full guardscripts/gits-sbx-epic-test.shshows NEW FAILURES: 0: live baseline and this head, same runner, 722 checks / 20 known live-drift FAILs each, FAIL lists identical (the only textual difference is one ephemeral tmpdir path inside a pre-existing H11 failure).Also in this PR (Amendment 2 — the lab-director switch is declared done when this lands)
<POSTING>filling becomes ONE filling for every seat on every repo, verbatim: "Post your verdict as a comment only. Do NOT run verdict-cli." The verdict-cli formal-review step is removed from the skill; no seat ever runs it.Homelab/lab-directorthe verdict comment's line 1 isLD-VERDICT <APPROVED|REQUEST_CHANGES> head=<40-hex> session=proj=sbx-lab;role=seat;sid=<sid>(the FULL gateway tag; the relay refused the baresession=<sid>form as tag-malformed for both #309 seats, 11-11); the relay files the head's formal review from it (6b), as ld-reviewer./run/secrets/ld/identities/named exception is gone from the hard limits: no seat reads anything there.Amendment 3 (in this head)
One guard check so template and relay cannot drift:
p1_verdict_example_checkinscripts/gits-sbx-epic-test.shverifies the template's LD-VERDICT example line (sample sid filled in) against the relay's anchored patterns, copied with a pointer atlab-director/dispatcher/relay/relay.go:272(verdictLineRE) andrelay/seat.go:38,55(seatSIDStamp, SeatSIDPattern). Mutation 24 re-stamps SHA256SUMS after the edit so it isolates exactly this invariant (seat A's HOLD on the previous head); it proves the bare form goes red and the full form passes.Proof summary
sha256sum -c SHA256SUMSincore/manifests/apps/sbx-common-phase1: OK (11 files).bash scripts/gits-sbx-epic-test.shon live (baseline) and on head3afb81ebfc0ac8dd5fa6ac5b243fc2a4c2b0a6bb, same runner; FAIL lists identical; NEW FAILURES: 0.no independent walk: implementer-run checks only
SESSION: LR-20261111-024-seatA-20261111T1301Z
MODEL: omniroute/deepseek-flash
HEAD:
ed820b2761VERDICT: APPROVE
FINDINGS: none
TEST: cd /home/ld/work/epics/ld-spec-024/seatA/infra-core/core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> exit 0
Basis (diff only, no comments/reviews read):
ed820b2761(matches the PR head). Base09c6eb13(origin/live).<POSTING>is exactly "Do NOT list or read this PR's comments or reviews before posting your verdict; judge the diff, tests and guard only." — the spec's sentence, byte for byte./issues/...#/commentsstring is incident narrative outside the fence. 6b moves the one-formal-review-per-head check to the implementer (/pulls/{n}/reviews), and the<POSTING>filling collapses to the single comment-only line for every seat on every repo./run/secrets/ld/identities/named exception is removed from the hard limits, consistent with 6b.Non-blocking observations for the implementer (not findings): the retained verbatim seat-rule sentence still names verdict-cli ("If verdict-cli or the named tool refuses or fails..."), which now sits beside the ban line — harmless but slightly confusing; and "the relay" is referenced in 6b without being defined in this skill.
SESSION: LR-20261111-024-seatB-20261111T1301Z
MODEL: omniroute/deepseek-flash
HEAD:
ed820b2761VERDICT: APPROVE
FINDINGS: none
TEST: cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> 0 ; diff guard-live-fails.txt guard-head-fails.txt -> 0 (20 known live-drift FAILs on both sides, 722 checks each, sole textual difference the ephemeral tmpdir path)
NOTE (non-blocking): the verbatim seat rule (unchanged, as spec 024 NEVER-TRY requires) still names verdict-cli ("If verdict-cli or the named tool refuses or fails"), which sits oddly beside 6b's "no seat ever runs verdict-cli"; that is the spec's own limit, not this diff's. On Homelab/lab-director, the prompt now carries both the LD-VERDICT line-1 requirement and the template's closing "Answer in exactly this form and nothing else" (which begins at VERDICT:); worth one clarifying word when the relay side lands.
657b114f0a36593636aaReview seat verdict — spec 024 / LR-20261111-024, infra-core #937.
SESSION: LR-20261111-024-seatB-20261111T1325Z
MODEL: omniroute/deepseek-flash
HEAD:
36593636aaVERDICT: APPROVE
FINDINGS: none
TEST: cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> exit 0 (11 files OK)
What I checked (diff, tests, guard only — no comments/reviews read):
git rev-parse HEAD=36593636aa, base09c6eb13; diff touches exactly 3 files (SHA256SUMS, ld-implement.SKILL.md, scripts/gits-sbx-epic-test.sh).<POSTING>(line 71) inside the seat-prompt fence./pulls/{n}/reviewsappears only as the implementer's own 6b step; verdict-cli formal-review step removed from 6b; no seat reads/run/secrets/ld/identities/(limits line rewritten, no seat exception left).sha256sum -c SHA256SUMSis green (11 files). Additionally verified without running the full suite:bash -n scripts/gits-sbx-epic-test.shOK; newp1_verdict_example_checkreturns empty on the head template (so check (f) is green) and reddens on both a drifted LD-VERDICT line (linea-no-coincide-con-relay.go:272 sid-no-anclado(seat.go:55)) and on mutation (24)'s bare-sid edit (session-sin-tag-completo(bare)); thepassstring now says 24 mutations and the p1_mut block holds exactly 24.SESSION: LR-20261111-024-seatA-20261111T1325Z
MODEL: omniroute/deepseek-flash
HEAD:
36593636aaVERDICT: HOLD
FINDINGS:
veredicto-sid-bare, scripts/gits-sbx-epic-test.sh:12294) does not isolate the new invariant (f): it sed-editssbx-common-phase1/ld-implement.SKILL.mdwithout re-stamping SHA256SUMS, andp1_checkalso enforces the hash of exactly that file (loop over SHA256SUMS ->sha256-$f, line 11919). After the mutation the file's sha256 is 35b32e135105459b0b5d703e4fe825fdf98485a60a8bf10e22a130b093584d13 while SHA256SUMS carries d401618ccc7518cb12645bf1fd23129cdfa3f245d6d72d99207f826626bce247, sop1_mutsees non-emptyp1_checkoutput whatever invariant (f) does. Spec 024 Amendment 3 requires "test it both ways: the bare form FAILS and the full form passes"; the guard's own test does not demonstrate that. Fix: re-stamp SHA256SUMS inside the mutation snippet (sed -i ... && sha256sum ... > SHA256SUMS), or add a both-ways test that callsp1_verdict_example_checkdirectly.ed820b2761(the PR body's Proof summary names that head). The reviewed head36593636supersedes it and its delta adds precisely the code that run could not exercise:p1_verdict_example_check(lines 11829-11871), invariant (f) call site (11923-11928) and mutation 24. At 2026-11-11T13:27Z the head's guard run (guard-head3.log, pid 1816821) had not yet reached the20[phase1]block, so no completed guard run at the reviewed head exists; Amendment 3 requires the full guard again at the new head.p1_check,vbad(line 11927) is assigned without alocaldeclaration, unlike every other helper variable of that function (local tree=$1 bad='' comp gits lab send ck f n want got s old new seed items, plus thep1sk/p1_lout/p1_ln/p1_ld/p1_jesusline), so it leaks into global scope.CHECKED AND CORRECT: Done-when #1 — the ban line sits immediately after
<POSTING>in the seat-prompt fence, byte for byte: "Do NOT list or read this PR's comments or reviews before posting your verdict; judge the diff, tests and guard only." Done-when #2 — no template step tells a seat to fetch reviews or comments: the only/pulls/{n}/reviewsmention (line 44) assigns it to the implementer; no seat-side verdict-cli step remains. The template's LD-VERDICT example line (line 89) passes the copied relay patterns (relay.go:272 + seat.go:38,55) and the bare-sid form is rejected: I ran the extracted helper on this head's template (empty = pass) and on the bare-sid mutant (returnssession-sin-tag-completo(bare)).bash -n scripts/gits-sbx-epic-test.shexit 0;p1_mutcount is 24, matching the claim. Done-when #4 is not verifiable by this seat by rule (I did not read this PR's comments or reviews).TEST:
cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS-> exit 0 (11 files OK); auxiliary: extractedp1_verdict_example_checkon head template -> pass, on bare-sid mutant ->session-sin-tag-completo(bare),bash -n scripts/gits-sbx-epic-test.sh-> exit 036593636aa69a9fb7624SESSION: LR-20261111-024-seatA-20261111T1333Z
MODEL: omniroute/deepseek-flash
HEAD:
69a9fb7624VERDICT: APPROVE
FINDINGS: none
TEST: cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> 0
What I checked (seat A, comment only; no verdict-cli run):
git rev-parse HEAD=69a9fb7624, unchanged; diff base09c6eb13(origin/live), 3 files.ld-implement.SKILL.md, directly after<POSTING>(line 71) inside the seat-prompt fence. Matches the PR body text exactly./pulls/{n}/reviewsmention (line 44, 6b) and the/issues/934/commentsmention (line 86) are the implementer's step and the cost anecdote, explicitly not a seat's. The/run/secrets/ld/identities/exception is gone from the hard limits (line 133: no seat reads anything there).sha256sum -c SHA256SUMS= 11 files OK, exit 0; the recorded sha256 ofld-implement.SKILL.mdmatches the file (d401618c…ce247).bash -n scripts/gits-sbx-epic-test.sh= 0.p1_verdict_example_checkon the head's template returns empty (green). The head's example line matches the copied relay pattern (40-hex head, fullsession=proj=sbx-lab;role=seat;sid=<run>-seatA-<stamp>Z). Adversarial runs: line removed ->ejemplo-no-unico(0) ejemplo-ausente;seatC->sid-no-anclado(seat.go:55); baresession=<sid>(mutation 24) ->session-sin-tag-completo(bare), i.e. red. Mutation 24's snippet, replayed literally from the script on a copy of the apps tree, does what it claims: the bare form goes red AND the SHA256SUMS re-stamp leaves the hash loop green (ld-implement.SKILL.md: OK), so the mutation isolates this invariant exactly as the PR body says (previous seat A's HOLD was about exactly that isolation).SESSION: LR-20261111-024-seatB-20261111T1333Z
MODEL: omniroute/deepseek-flash
HEAD:
69a9fb7624VERDICT: HOLD
FINDINGS: core/manifests/apps/sbx-common-phase1/ld-implement.SKILL.md:86 makes the lab-director verdict comment's line 1 the
LD-VERDICT <APPROVED|REQUEST_CHANGES> head=<40-hex> session=proj=sbx-lab;role=seat;sid=<sid>line, but the seat prompt fence at line 73 still saysAnswer in exactly this form and nothing else:whose line 1 isVERDICT: APPROVE | HOLD— a seat that obeys line 73 posts a comment whose first line is not LD-VERDICT, and lab-director/dispatcher/relay/relay.go ParseClaim reads only the first non-empty line against verdictLineRE, so that comment is refused tag-malformed; state the composite form explicitly (LD-VERDICT line 1 first, then the VERDICT/MODEL/HEAD/FINDINGS/TEST block) and note that p1_verdict_example_check only validates the example line at line 89, never this conflict.TEST: cd /home/ld/work/epics/ld-spec-024/seatB/infra-core/core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> exit 0
Notes (not findings): head confirmed at
69a9fb7624by git rev-parse HEAD; base merge-base09c6eb13; the ban line at line 72 is byte-for-byte the spec sentence and sits directly after<POSTING>;bash -n scripts/gits-sbx-epic-test.shexit 0; p1_mut count is 24, matching the pass message; the copied relay regexes match relay.go:272 and seat.go:38,55 verbatim, and p1_verdict_example_check returns empty on the head,session-sin-tag-completo(bare)on the bare-sid mutation, andejemplo-no-unico(2)on a duplicated line; mutation 24's SHA256SUMS re-stamp is correct, and ld-implement.SKILL.md is not among the six YAML-parsed SKILL.md, so with the hash loop satisfied (f) is the only invariant that mutation can trip.69a9fb76243afb81ebfcSESSION: LR-20261111-024-seatA-20261111T1341Z
MODEL: omniroute/deepseek-flash
HEAD:
3afb81ebfcVERDICT: APPROVE
FINDINGS: none
TEST: cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> exit 0 (11 files OK)
Evidence (independent, this seat, comment only — no verdict-cli run):
git rev-parse HEAD=3afb81ebfc, worktree clean; base09c6eb13, diff is 3 files (+72/-14) and nothing else.<POSTING>now has one filling only; 6b carries the relay-files-the-review wording; the/run/secrets/ld/identities/named exception is gone from the hard limits./pulls/{n}/reviewsmention is the implementer's own step in 6b, and/issues/.../commentsappears only in the historical cost note.p1_verdict_example_checkextracted and run against this tree returns EMPTY (good line accepted);bash -n scripts/gits-sbx-epic-test.shexits 0; 24p1_mutcalls match the "24 mutaciones" pass text.sha256sum -cstill OK for the template, and the check goes RED (session-sin-tag-completo(bare)) — the mutation isolates the invariant as claimed.no independent walk: implementer-run checks onlyline is present (flow 3's full-guard numbers are declared implementer-run, not claimed as independent).SESSION: LR-20261111-024-seatB-20261111T1341Z
MODEL: omniroute/deepseek-flash
HEAD:
3afb81ebfcVERDICT: APPROVE
FINDINGS: none
TEST: cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> exit 0 (11 files: OK)
NOTE (non-blocking, not a finding): the new guard checks only the one
^LD-VERDICTexample line. The tag form is also stated in prose at ld-implement.SKILL.md:44 (6b) and :86 (after<POSTING>); I confirmed that drifting either to the baresession=<sid>form (the #309 refusal) leavesp1_verdict_example_checkgreen. That matches the spec's wording ("the template's LD-VERDICT example line"), so it is a limit of the check, not of this head.NOTE (non-blocking, not a finding): the guard's separator class is
[[:space:]]where the relay's is[ \t](relay.go:272), so a \v or \f separator between fields would pass the guard and be refused by the relay;[[:blank:]]is the exact ERE counterpart. The tail[[:space:]]*$is harmless because the relay'sfirstLine()TrimSpaces the line first.Checks behind the APPROVE: head is the verbatim sha and
sha256sum -c SHA256SUMSis green; the ban line sits directly after<POSTING>in the seat prompt fence, byte for byte; the<POSTING>filling is now the single comment-only form and the verdict-cli step plus the/run/secrets/ld/identities/exception are gone consistently (no dangling "prerequisite 4b"); the relay the doc points to exists on live (lab-director/dispatcher/relay/, polled by ld-runtime viaLD_RELAY_POLL_INTERVAL, repoHomelab/lab-director) and itsParseClaim/ParseSessionTag/SeatSIDPatternaccept exactly the documented form. I re-ran the guard function against the template and against mutations: current line green; bare sid red; missing line red; duplicate line red; wrong role red; uppercase head red; sid without stamp red. Mutation 24's re-stamp was verified to leave the hash loop green while the verdict check goes red, so it isolates the invariant. PR body marks all four "Done when" flows PROVEN with their proofs and carries theno independent walk: implementer-run checks onlyline.APPROVE — ld-reviewer — head
3afb81ebfcSESSION: LR-20261111-024-seatB-20261111T1401Z
MODEL: omniroute/deepseek-flash
HEAD:
3afb81ebfcFINDINGS: none
TEST: cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> exit 0 (11 files OK)
What I checked (diff 09c6eb1...3afb81eb, three files, +72/-14):
git rev-parse HEADin my clone =3afb81ebfc, clean tree, base09c6eb133d.ld-implement.SKILL.md:72is byte-identical tospec.md:15("Do NOT list or read this PR's comments or reviews before posting your verdict; judge the diff, tests and guard only.") and sits directly after<POSTING>in the fenced seat prompt./pulls/{n}/reviewsreference is the implementer's own step in 6b; no seat step tells a seat to fetch comments or reviews.sha256sum -c SHA256SUMSgreen, 11 files, exit 0; the re-stamped ld-implement hash matches.p1_verdict_example_checkin isolation. Real template -> empty (pass; exactly one^LD-VERDICTline at line 89); same file with the bare-sid edit ->session-sin-tag-completo(bare); absent file ->template-ausente. So the check is non-vacuous and red exactly where the relay's tag parser refuses.p1_mutcalls, 24 claimed, no duplicate names;bash -nclean.lab-director/dispatcher/relay/relay.go:272verdictLineRE andrelay/seat.go:38seatSIDStamp at this repo's ownorigin/livematch the guard's comment verbatim (40-hex head, non-empty session tag,\d{8}T\d{4}Z, run id QuoteMeta-anchored).NOTES (non-blocking, nothing to resolve before landing): (a) the guard comment points at
relay/seat.go:55for SeatSIDPattern, which is line 54 at this live head — the doc comment sits at 55, so a reader still lands inside the function; (b)[[:space:]]in the copied line is a documented, fractionally wider stand-in for the relay's[ \t]; (c) the verbatim seat rule still names verdict-cli even though no seat may run it — intentional (rule kept byte-for-byte), reads as a hypothetical.APPROVE — ld-reviewer — head
3afb81ebfcSESSION: LR-20261111-024-seatA-20261111T1412Z
MODEL: omniroute/deepseek-flash
HEAD:
3afb81ebfcFINDINGS: none
TEST: cd core/manifests/apps/sbx-common-phase1 && sha256sum -c SHA256SUMS -> 0