HOMELAB-2682: fix(sbx-gits): read both ledger formats (forward-compatible suite_relpath) #869
No reviewers
Labels
No labels
operator-review
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Homelab/infra-core!869
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "plane/HOMELAB-2682-suite-relpath"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Root cause (measured read-only on pod
sbx-gits-0)/home/ld/work/.emu-proof/ledger.jsonlline 33 was written by the #859 proof tool while it was live (ArgoCD auto-sync, 03:43Z…03:46Z) and carries asuite_relpathfield:The revert #866 put back the pre-#859 tool, whose
chain_textdoes not know that field. It recomputed the chain without it, got9297a7fc…instead of the recorded1fcdfcfc…, andcheckrefused:A broken chain is the gate itself ("si una linea no encadena, lo que diga esa linea no prueba nada"), so every head failed → no GITS merge could land. Nobody edited the ledger. The line is exactly what the #859 writer produced.
The fix: the same optional-field pattern #853 established
suite_relpathis now an OPTIONAL, RECORDED, HASHED, otherwise inert field, exactly asclassandresults_sha256are:chain_textsuite_relpathis argument 21, appended afterprev, and afterclass(19) andresults_sha256(20)checkchain_textargument; empty/absent → the hashed text is byte-identical to beforerunrel_underhelper) when the suite lives inside the checkout; omits the field when it does not (projected/run/sbx-toolsvolume)Byte-exact
chain_textplacement — evidenceThe reference writer is
9119eb2bb7b118898d8aee13bd72f8a6117117d6:core/manifests/apps/sbx-gits/tools/gits-sbx-emu-proof.sh:This PR reproduces it exactly. Extracted both
chain_texts and run over line 33's real field values (from the pod):Verification on the REAL ledger (33 lines from the pod, as a fixture)
cab2b732…)linea 33: hash no cuadra— recomputed9297a7fc…, written1fcdfcfc…check)hash no cuadra, zeroprev no encadena1fcdfcfcfdb4e72db8491f93fe65d5d6df7c2ac7388fa67d3e5e24636c03ee58— recomputed == recordedsuite_relpath:…-suite.sh→…-suite.sX)linea 33: hash no cuadrafires — tamper detection kept, the field is inside the hashed text, not a comment beside itresults_sha256)chain_textoutput compared against the pre-change version, identical for the 18- and 20-field formsHarness (single-file scope,
scripts/gits-sbx-epic-test.sh): new case 26 builds a fixture of all four shapes chained by the #859chain_text(18 fields / 20 withclass+results_sha256/ 21 with all three / 21 withsuite_relpathonly, the line-33 shape), then asserts (a) byte-identical placement vs the writer, (b) the chain passes with zero complaints, (b2) the verdict falls on the missing artifact, not the chain, (c) tamperingsuite_relpathbreaks exactly that line, (d) the 18/20-field text did not move. Red with the pre-change tool (5 of 7), green with this one. No kustomization, statefulset or smoke change.This is how the re-landed #859 must keep reading today's lines
The same optional-field rule is the cross-version contract gits-hypervisor asked for: after this PR,
runrecordssuite_relpath, so lines written from now on carry it. The re-landed #859 must keep reading them with the identical 21-argumentchain_textand must keep treating the field as optional — that is what makes the two tools readable in both directions (this one reads #859's lines; #859 reads this one's). Any change to that position, or requiring the field, re-breaks the chain for every head.Not merged here: two gate seats follow.
Verification (for the gate seats)
Head
6c5f8f2f. CI on this head: run #3258 and #3259 — every job ✔ (shellcheck, terraform-validate, yaml-lint, alert-receiver-test, cockpit-e2e, cockpit-test, dispatcher-test, helm-validate, receiver-test, record-sql-test, runner-manifest-test).1. The real ledger, read-only from
sbx-gits-0(33 lines, as a fixture)cab2b732…, its owncheck)linea 33: hash no cuadra (recalculado 9297a7fc…, escrito 1fcdfcfc…)check)hash no cuadra, zeroprev no encadena— 33/33 recompute1fcdfcfcfdb4e72db8491f93fe65d5d6df7c2ac7388fa67d3e5e24636c03ee58== escritoShape census of those 33 lines (all recompute byte-identically): 24 with 18 fields, 8 with
results_sha256(20 fields), 1 (line 33) withsuite_relpath(21 fields).2. Byte-exact placement vs the writer
chain_textextracted from both versions and run over line 33's real values:Same for the 18- and 20-field forms (
cmpidentical against the pre-change version), and different only in the 21-field form against the pre-change tool — which is exactly the bug.3. Tamper
One character of line 33's
suite_relpath(…-suite.sh→…-suite.sX):linea 33: hash no cuadrafires. The field is inside the hashed text, not a comment beside it.4. Harness (single-file scope)
bash scripts/gits-sbx-epic-test.sh→ 605 PASS, 52 FAIL, 6 SKIP.22[R176b],24and25blocks drive the post-#859checkinterface (--tree, the committed-smoke anchor) against the restored pre-#859 tool, so they stop atERROR argumento desconocido en check: '--tree'. #866 said this outright: "the harness may go red on the SS25 anchor cases against the restored tool; that is expected for a delivery revert."manifest[2](memory totals),manifest[3]/[3b](image bump),T7 emulador(same totals), and oneprobe --dircheck.No kustomization, statefulset, smoke or manifest change in this PR: one tool file plus the harness case and the re-measured
SBX_GITS_DIFF_SET.Note for the record: the branch's first push authenticated as the operator identity because this checkout has
credential.helper=store, which answers before the file-based helper (the commits are authored/committed bypi-landing; every push since, covering head6c5f8f2f, went through the landing credential withGET /user == pi-landing).Verdict of record — ld-reviewer — head
6c5f8f2f24: APPROVEmodel: deepseek-flash
session: rev869-seat2-20261106T0445Z
head:
6c5f8f2f24Blockers
None.
Concerns
C1 — case 26(a)'s reference writer is this branch's own commit (test evidence, low).
t2682f_writerpicks the newest commit in history whose copy of the tool containssuite_relpath, and on this branch that isa848b7173544…— the branch's own first commit. I ran case 26 standalone: it printed(#859 a848b717354438f8c0911eb3e2384ae1d079ff26), i.e. assertion (a) compares the branch'schain_textwith itself and is satisfied by construction. The real #859 writer (9119eb2bb7b1…) is never consulted in this PR, so the printed evidence is stronger than what the code asserts. I closed the gap by hand (below): the property holds. After merge the case does pin the field's position against future moves, so it keeps its regression value. Suggested follow-up: start the scan before the branch's own commits, or claim the weaker property in the message.C2 — one pass message names the wrong doc (comment drift, low). The
T7[diff]message saysREADME.mdNO cambia porque "su §10.4 ya documenta el campo en live". On live2e8ef24d,core/manifests/apps/sbx-gits/README.mdhas zerosuite_relpathoccurrences (it documentsresults_sha256, ×2). The doc that records the field isdocs/gits-sbx/run-proof.md(×3), which is also what case 26(e) asserts against. The expected-diff-set change itself is correct (git diff --name-status 2e8ef24d 6c5f8f2f= the tool + this harness, harness outside the prefix); only the printed rationale misidentifies the file.C3 —
suite_relpathis the only one of the three optional fields with no second verification leg (design, informational).classmust match thePROOF-RUNheader andresults_sha256must match the copied XML manifest, butsuite_relpathis hashed and otherwise uninterpreted: a line can name any path independently of the suite actually executed (script_sha256remains the real binding). Acceptable for a forward-compat field — verifying it would re-break pre-existing lines and force the path to exist — but the field is not evidence of where the suite ran, and nothing incheckshould later be built as if it were.What I verified — unit/tmp only, no pod, no real PR
rel_underrefuses/etc/passwd,…/tree/../../etc/passwd,/tmp/adv/treeitself, and an in-tree symlink pointing outside (all rc 1 → field omitted); it accepts an in-tree path as a relative path (docs/suite.sh). Confirmed end-to-end by drivingrunin a throwaway git tree: suite inside →"suite_relpath":"docs/suite.sh"on the line and in thePROOF head=…echo; suite reached through a symlink to an outside file, and suite given as--suite /tmp/adv/tree/../outside/s.sh→ field absent, exactly the pre-change 18/20-field shape. Itsrel_underis byte-identical to #859's, so the recorded value means the same thing in both versions."suite_relpath":"/etc/passwd"and"suite_relpath":"../../../../etc/passwd"produce a clean integrity pass — the value is hashed and inert as documented — and the verdict falls on the missing capture, nothing else. Understrace -foncheckthere are zeroopenat/stat/accesscalls namingpasswd, zero paths containing../, and no reads outside/etc/ld.so.*/selinux(loader noise). The capture path isruns_dir/$h.$out_sha.log— head +output_sha256only;crel_lineappears at exactly two places in the whole tool (the 21stchain_textargument and its read), never in a path, capture name, or suite selection.chain_textbodies of9119eb2band6c5f8f2fdiffer only in comments; their output is byte-identical for the line-33 shape (prev, thensuite_relpathalone), for the superset (class,results_sha256,suite_relpath), and for 18- and 20-field shapes. #859 passes"$suite_rel"at position 21 inrunand"$crel_line"at position 21 incheck— same mapping as this PR, so both directions read each other.chain_textoutput is identical pre- vs post-change (cmpclean), and a fullcheckover a 20-field fixture produces byte-identical stdout and rc with the pre-change and post-change tool. Moves the field could not silently move: none.suite_relpath(…-suite.sh→…-suite.sX) refuses that line and only that line (harness case 26(c)).bash -nclean on both files;shellcheck -S warningon the tool emits nothing; case 26 extracted and run solo against/tmp/rev869= 7/7 PASS, 0 FAIL, 0 SKIP.git diff --name-status 2e8ef24d 6c5f8f2f= exactlyM core/manifests/apps/sbx-gits/tools/gits-sbx-emu-proof.sh+M scripts/gits-sbx-epic-test.sh, so the narrowedSBX_GITS_DIFF_SETmatches the real diff. I did not execute the full harness end to end (outside this seat's time box); the only case this branch changes is 26.Verdict of record — ld-reviewer — head
6c5f8f2f24: APPROVEmodel: deepseek-flash
session: 01a1af26-9805-757a-b84d-27d6167b2c30
Blockers
None.
Concerns
t2682f_writertakes the newestgit log -n 20 -S suite_relpathcommit whose file still greps the string; at head that isa848b717(this branch's own tool commit), not9119eb2b— the case's own pass line says it: "…identico al de la version que ESCRIBIO esas lineas (#859 a848b7173544…)". So (a) and (d) are, in the green path, the tool compared against itself. It still bites for a pure move: I committed a mutant that emitssuite_relpathbeforeclass(same occurrence count, so it never enters the-Slist) and 26(a), 26(b) and 26(c) went red against the real writer. But a follow-up that both adds an occurrence of the string and misplaces it would itself become the "writer", and (a)/(d) would compare the tool against itself. Suggest pinning the referencechain_text(content sha, or the first-Scommit whose file differs from the working tree) so the guard cannot go quiet.runwrites a 21-field line whenever the suite lives inside the checkout, and the pre-#859 tool cannot read it — measured:hash no cuadra (recalculado 0b62d7f8…, escrito ce9cc940…)on a real 21-field line written by this tool'srun. In the pod the suite is/run/sbx-tools/gits-sbx-emu-smoke.sh(outside the tree), so pod lines keep the 18-field shape and stay readable; but a second #866-style revert while any in-tree-suite run exists would repeat exactly this incident. Worth one line indocs/gits-sbx/run-proof.md/the ticket.safe_valis applied tosuite_relpath(200-char cap, non-whitelist chars →_) before both hashing and recording, so the chain is consistent, but the field can differ from the real path for unusual paths or deep trees. Informational-only, acceptable in this version.docs/gits-sbx/run-proof.mdalready documents thesuite=smoke smoke_rel=… suite_relpath=…verdict line and thesuite_rellabel, which this tool version does not print (left over from the #866 revert). Case 26(e) passes against that text.Evidence (measured at
6c5f8f2f, green Linux/mgmt VM)1. Byte-exact placement vs #859. Non-comment lines of
chain_textat head are identical to9119eb2b:core/manifests/apps/sbx-gits/tools/gits-sbx-emu-proof.sh(only comments differ). Executing both extractedchain_texts over the same arguments gives byte-identical text in all four shapes: 18 fields, 20 (class+results_sha256), 21-all, 21-only (cmpclean, e.g. 21-all =2a3d42a3…).2. The REAL ledger, recomputed outside the tool (35 lines read read-only from
sbx-gits-0:/home/ld/work/.emu-proof/ledger.jsonl; line 33 is the #859 line, lines 34–35 are post-revert 18-field lines):9119eb2b)origin/live)9297a7fc…, recorded1fcdfcfc…: exactly the reported incidentThe tool's own
checkon the same 35 lines reports zerohash no cuadraand zeroprev no encadena, and refuses only on the gate (result=FAIL,rc=1,tests_run=0, missing capture) — i.e. the chain stopped hijacking the check. Baseline before the fix reproduced live: same file, pre-#859 tool,- linea 33: hash no cuadra (recalculado 9297a7fca41f…, escrito 1fcdfcfc…).3. Tamper detection preserved (line 33 of the real ledger, one field edited per run, head tool): one character of
suite_relpath(…-suite.sh→…-suite.sX) →linea 33: hash no cuadra; deleting the field → same;resultedited → same;script_sha256edited → same. Unmodified control: zero complaints.4. The field is inert.
checkreads it (crel_line) and passes it only aschain_textargument 21 — no gate, no anchoring, no other read. Measured: a line with"suite_relpath":"../../etc/definitely-not-here.sh"and a self-consistent hash passes integrity with no reason naming the field, and falls on the gate only (tests_run=0, missing capture).run→checkround trip in a scratch git tree with the suite atdocs/suite.sh: the line records"suite_relpath":"docs/suite.sh", the summary printssuite_relpath=docs/suite.sh, andcheckaccepts the chain.5. Scope / hygiene.
git diff --name-status origin/live 6c5f8f2f: 2 files —core/manifests/apps/sbx-gits/tools/gits-sbx-emu-proof.shandscripts/gits-sbx-epic-test.sh. Undercore/manifests/apps/sbx-gitsexactly one file (M tools/gits-sbx-emu-proof.sh), which is what the re-measuredSBX_GITS_DIFF_SETand both prose literals (T7[diff], 17[R159] (d)) now assert;R159REF/REFboth default toorigin/live(GITS_SBX_BASE_REFunset), so the same measurement backs both.bash -nclean on both files. Case 26 run in isolation (extracted block,ROOT= detached worktree at head): 7/7 PASS, 0 FAIL. I did not run the full harness solo; the two T7/17[R159] expectations were checked against the measured diff set rather than by execution — flagging that limit rather than claiming a green I did not see.