Homelab/infra-core
6
0
Fork 2
Code Issues Pull requests 35 Projects Releases 1 Packages Wiki Activity Actions

IR-20261111-baohelper: bao-operator-token writes the token to a 0600 file, never to stdout #943

Open
pi-landing wants to merge 5 commits from plane/IR-20261111-baohelper into live
pull from: plane/IR-20261111-baohelper
merge into: Homelab:live
Conversation 2 Commits 5 Files changed 4 +809
pi-landing commented 2026-11-11 18:17:32 +00:00
Collaborator
Copy link

IR-20261111-baohelper — the helper writes the token to a 0600 file and never prints it

Branch plane/IR-20261111-baohelper -> live. One commit (b69148ce), last
trailer Lab-Run: IR-20261111-baohelper.

Why

/usr/local/bin/bao-operator-token is hand-installed on the mgmt host (mode
755, root:root, dated 2026-10-14) and no repo has ever managed it — infra-core
live has zero hits for the helper or its role. Its old contract was: print an
OpenBao token on stdout, so every caller captured it with
$(...)/.stdout.strip(). A token on stdout is a token in a log, a transcript,
a set -x trace or a process boundary; the IR-20261111 leak came from exactly
that shape.

This PR puts the helper under management and removes the stdout sink:

  1. NEW core/ansible/roles/mgmt-tools/files/bao-operator-token (git mode
    100755) — the managed replacement.
  2. core/ansible/roles/mgmt-tools/tasks/main.yml — a final task
    Install bao-operator-token helper (copy, root:root, 0755).

The role task is not applied by this PR. Installing on the host is a separate,
deliberate step.

Contract implemented

  • Exactly one argument: the output file path. No argument, two arguments, -,
    /dev/stdout, /dev/fd/1|2, /dev/stderr, /dev/tty, /dev/console,
    /proc/*/fd/1|2, and a tty on stdout are all refused before anything is
    minted.
  • The path is claimed with O_CREAT|O_EXCL at mode 0600, so the file is 0600
    from birth and an existing path — regular file, symlink including a dangling
    one    , directory, device — is refused.
  • The token goes curl -> python -> file. It never enters a shell variable and
    never appears on this program's stdout.
  • The only output is the non-secret line written <path> ttl <n> (ttl from
    auth.lease_duration).
  • Mint logic is unchanged: Zitadel machine-user JWT (openbao-operator,
    key ~/.config/openbao-operator-key.json) -> jwt-bearer exchange ->
    OpenBao sa-operator OIDC login.
  • Cleanup removes the claimed file only if this run created it and did not
    finish. A refused run never touches the caller's path (an earlier draft did;
    that was a real bug found while testing this branch and fixed in it).

Test evidence

Success path — PROVEN (on the branch head, 2026-11-11T18:1xZ)

The pause note recorded this path as blocked (Zitadel jwt-bearer returned
HTTP 500 Errors.Internal twice at ~16:44Z/~16:48Z). Re-tested on resume:

  • run in a mktemp -d directory, mode 0700; helper exited 0 in ~1s, stderr
    empty;
  • stdout was exactly written <tmpdir>/tok ttl 3600 — path matches the argument,
    ttl 3600; re-checked with a strict full-match regex;
  • output file: regular file, not a symlink, mode 0600, 27 bytes = a 26-char
    OpenBao client_token + one trailing newline. Prefix s.Sp… (first four
    characters only; the value was never printed) and the token string does not
    occur in stdout;
  • the file was shredded after use (shred -u) and the path is gone;
  • honest caveat: the token's usability was not separately proven by a
    lookup-self (this run permits no OpenBao read route). The helper's own
    success criterion is a non-empty client_token inside a 200 OIDC login
    response, which it now gets — it exits non-zero on an empty token, and it did
    not.

Refusals — PROVEN

Re-run on resume as a smoke check: a pre-existing 0640 file with content ->
non-zero exit, stderr refusing: output path already exists (no overwrite, symlinks included), the file's mode and content intact, zero bytes on
stdout.

Carried forward from the original test round, each non-zero exit with the
filesystem untouched: no-arg, -, /dev/stdout, /dev/fd/1, tty-on-stdout,
pre-existing file, symlink to /dev/stdout, dangling symlink, directory, two
arguments, missing parent directory. Mint-path failure cleanup: with
BAO_ADDR=https://127.0.0.1:1 the claimed file is created and then removed and
nothing is printed.

bash -n and shellcheck -S warning are clean on the helper.

Callers

The two repo-managed callers are in Homelab/prod, so they cannot land in
this PR; they come in the paired PR from branch
plane/IR-20261111-baohelper-callers:

  • scripts/maintenance/window-run.sh (pve_load_creds) — passes an output file
    inside the existing 0700 $PVE_TMP, reads it back, shreds it before the
    bao kv get;
  • values/forgejo/artifact-retention-evidence.md — doc prose, new invocation
    form.

Roughly 80 further references live under working-files/, which is not a git
repository: the executable ones are probe/rotation scripts and reports. They are
not migrated here and will break on stdout capture once the host helper is
replaced — that inventory is recorded on the run's ticketing and is follow-up
work, not a claim of this PR.

Rollout / risk

  • Nothing is installed by this PR; the host file is untouched. Merging changes no
    live behaviour until the role task is applied (or the file copied by hand).
  • Ordering matters in the other direction: the prod caller PR must land after
    the new helper is on the host, or pve_load_creds fails closed (no PVE
    credentials, phase blocked). It cannot leak: it never echoes the helper's
    stdout, only its byte count.
  • Rollback: restore the previous helper file and revert the caller PR; the mint
    logic in this branch is unchanged from the hand-installed original, so the old
    and new helper are interchangeable apart from the output sink.

Review

Two independent census seats are requested on this head (see the run's seat
briefs). No merge is performed by the worker that opened this PR.

Fix round (review-93 / homelab-93 seat) — head 714522c9

The independent seat review (working-files/security-hypervisor/pr943/review-93.md)
raised one blocking finding and three should/minor fixes. All six items are
addressed on this head, as two new commits on top of b69148ce:

  1. B1 (blocking) — refusal fix, own commit 9318f68: any argument starting
    with - (--help, -h, anything) is refused before any mint, usage to
    stderr, rc 2. Also required: absolute path; parent dir must exist, be owned
    by the caller's uid, and be mode 0700 (the swap window from S2).
  2. S1 — no secrets in argv: the Zitadel JWT assertion and the id_token now
    reach curl inside 0600 temp files via -d @file; revoke-self runs in-process
    over urllib. No token material ever appears in any process argv.
  3. S3 — revoke on post-mint failure: the claim, both POSTs and the write run
    in ONE python process; any failure after the mint (unparseable ttl, write
    error, closed stdout, SIGTERM/SIGHUP/SIGINT) revokes the token (revoke-self
    with the minted token) and removes the file before a non-zero exit.
  4. S2 — reopen race closed: the token is written through the very fd that
    was O_EXCL-created; no path re-open between claim and write.
  5. R1/R2 — ansible: the helper task carries its own tag bao-helper
    (install with --tags bao-helper; the whole role no longer has to run first);
    any existing host copy is backed up to /usr/local/bin/bao-operator-token.pre-943
    before replacement. roles/mgmt-tools/README.md documents the one-file
    sha256-checked manual install as the runbook alternative, plus a post-install
    proof.
  6. Test matrix (fix 6): an offline fake-binary harness
    (roles/mgmt-tools/tests/test-bao-operator-token.sh) — a stub curl first in
    PATH records every argv; PATH excludes the host copy entirely (plus a
    tripwire stub); a 127.0.0.1 recorder stands in for the vault address on the
    revoke path. 39/39 checks pass: every refusal path (zero mint attempts),
    argv-freedom (recorded stub argv proven free of assertion/id_token/token),
    -d @file usage, revoke-self ordering with the minted token + file removal,
    a live symlink-swap attack leaving the victim untouched (same-fd proof),
    O_EXCL structural assertions, bash -n and shellcheck clean. No real mint
    was executed in any test.

Old-form callers (~20 out-of-repo scripts/docs) are inventoried at
working-files/security-hypervisor/orch/old-form-callers.md for follow-up; the
critical landing routes never called this helper (review-93 §4a) and are
unaffected.

Fix round 2 — head 27b52f38 (review-93 N1–N6)

Second reviewer pass on 714522c9 found six items; all are addressed in new
commit 27b52f38 (trailer Lab-Run: IR-20261111-baohelper):

  • N1 (blocking) — the pre-replace backup no longer keeps the old
    token-on-stdout helper runnable: it goes to /root/bao-operator-token.pre-943
    (root:root, 0400, outside PATH) via stat + when, so a fresh VM skips
    it cleanly; the README manual route uses install -m 0400 (no cp -a).
  • N2 — on_signal now rmtrees the temp dir (key.pem copy of the operator
    key, zform assertion, login.json id_token) before os._exit; no more
    copies piling up in /tmp after SIGINT/SIGTERM/SIGHUP.
  • N3 — sys.stdout.flush() inside the try: a caller-closed stdout now
    raises at flush, so the failure is caught in-run (rc 1, token revoked, file
    removed) instead of surfacing as rc 120 at interpreter exit with a live
    token left in the file; stdout is dup2'd to /dev/null first so shutdown
    re-flush cannot overwrite the honest rc. A catch-all after the mint revokes
    and unlinks on any non-fail() failure (e.g. ENOSPC on the fd write).
  • N4 — a claimed flag: the output file is only removed if this run
    created it.
  • N5 — runbook/README: the install verification compares the installed
    file's sha256 against git show <merge-sha>:core/ansible/roles/mgmt-tools/ files/bao-operator-token | sha256sum (merged-blob provenance, not transfer
    integrity).
  • N6 — the test exports BAO_ADDR=http://127.0.0.1:9 for the whole
    suite; new SIGTERM-cleanup and closed-stdout-revoke tests (fake curl
    hang mode, fake clock not needed); fake-binary matrix extended.

Verification: bash -n + shellcheck clean; hermetic offline run
(unshare -rn, stub curl, dead 127.0.0.1 BAO_ADDR, /usr/local/bin excluded,
real helper never executed): pass=50 fail=0.

Fix round 3 — head c251af61 (review-3c S1/S2/n1/n2/n4/n6)

Third reviewer pass (seat 3c, round 3) on 27b52f38 had no blocking hole; its
two should-fixes and four nits are addressed in new commit c251af61
(trailer Lab-Run: IR-20261111-baohelper), citing review-3c-r3.md:

  • S1 — local cleanup now runs BEFORE the network revoke in fail(),
    on_signal() and the post-mint catch-all: the claimed output file is
    unlinked and the temp dir (key.pem copy, zform assertion, login.json
    id_token) rmtree'd first; only then the blocking revoke-self (urlopen,
    15 s timeout). A TERM→KILL grace window shorter than the revoke wait can no
    longer leave the token file or key material on disk.
  • S2 — the README post-install proof now STARTS with a hard gate: a
    sha256 check of /usr/local/bin/bao-operator-token against
    git show <merge-sha>:core/ansible/roles/mgmt-tools/files/bao-operator-token | sha256sum, stopping on mismatch BEFORE any invocation (--help
    included) — the host may still run the old token-on-stdout helper, and any
    invocation of a not-actually-installed file would leak a live token.
  • n1 — revoked=True is set only after a successful revoke-self; on
    failure the helper prints exactly one stderr line
    warning: revoke-self failed; token expires at its ttl (guarded against
    double-print from the catch-all re-entry).
  • n2 — picked option A: delivery clears state["claimed"] (and the token
    reference), so a signal landing in the finally/shutdown window keeps the
    caller's delivered file and revokes nothing; token ownership passed to the
    caller at delivery.
  • n4 — README rollback restores the backup at mode 0755 root:root via
    sudo install (0400 would fail closed but non-executable); the manual
    route's backup-exists check uses sudo test -f (/root is 0700, so the old
    unprivileged check always read "absent" and a second runbook pass could
    overwrite the backup).
  • n6 — two new hermetic tests: SIGTERM after delivery (LD_PRELOAD hold
    on the first bao-op-* unlink/rmdir keeps the helper in the post-delivery
    finally window; asserts the caller's file survives, temp dir rmtree'd, no
    revoke attempted) and the catch-all with client_token as an int (asserts
    rc 1, TypeError surfaced, file removed, revoke-self carried the minted
    token).

Worktree hygiene: the untracked Syncthing draft
files/bao-operator-token.sync-conflict-20261111-144623-P5KCD5R (statted:
0755, 12020 B, in no commit) was deleted, not committed (review-3c n5).

Verification: bash -n + shellcheck clean; hermetic offline run
(unshare -rnm, stub curl, BAO_ADDR=http://127.0.0.1:9, /usr/local/bin
excluded, real helper never executed): pass=58 fail=0.

# IR-20261111-baohelper — the helper writes the token to a 0600 file and never prints it Branch `plane/IR-20261111-baohelper` -> `live`. One commit (`b69148ce`), last trailer `Lab-Run: IR-20261111-baohelper`. ## Why `/usr/local/bin/bao-operator-token` is **hand-installed** on the mgmt host (mode 755, root:root, dated 2026-10-14) and no repo has ever managed it — `infra-core` `live` has zero hits for the helper or its role. Its old contract was: print an OpenBao token on **stdout**, so every caller captured it with `$(...)`/`.stdout.strip()`. A token on stdout is a token in a log, a transcript, a `set -x` trace or a process boundary; the IR-20261111 leak came from exactly that shape. This PR puts the helper under management and removes the stdout sink: 1. **NEW** `core/ansible/roles/mgmt-tools/files/bao-operator-token` (git mode 100755) — the managed replacement. 2. `core/ansible/roles/mgmt-tools/tasks/main.yml` — a final task `Install bao-operator-token helper` (copy, `root:root`, `0755`). The role task is **not applied** by this PR. Installing on the host is a separate, deliberate step. ## Contract implemented - Exactly one argument: the output file path. No argument, two arguments, `-`, `/dev/stdout`, `/dev/fd/1|2`, `/dev/stderr`, `/dev/tty`, `/dev/console`, `/proc/*/fd/1|2`, and a tty on stdout are all refused before anything is minted. - The path is claimed with `O_CREAT|O_EXCL` at mode `0600`, so the file is 0600 from birth and an existing path — regular file, **symlink including a dangling one**, directory, device — is refused. - The token goes curl -> python -> file. It never enters a shell variable and never appears on this program's stdout. - The only output is the non-secret line `written <path> ttl <n>` (`ttl` from `auth.lease_duration`). - Mint logic is **unchanged**: Zitadel machine-user JWT (`openbao-operator`, key `~/.config/openbao-operator-key.json`) -> `jwt-bearer` exchange -> OpenBao `sa-operator` OIDC login. - Cleanup removes the claimed file **only** if this run created it and did not finish. A refused run never touches the caller's path (an earlier draft did; that was a real bug found while testing this branch and fixed in it). ## Test evidence ### Success path — PROVEN (on the branch head, 2026-11-11T18:1xZ) The pause note recorded this path as blocked (Zitadel `jwt-bearer` returned `HTTP 500 Errors.Internal` twice at ~16:44Z/~16:48Z). Re-tested on resume: - run in a `mktemp -d` directory, mode `0700`; helper exited **0** in ~1s, stderr empty; - stdout was exactly `written <tmpdir>/tok ttl 3600` — path matches the argument, ttl `3600`; re-checked with a strict full-match regex; - output file: regular file, not a symlink, mode **0600**, 27 bytes = a 26-char OpenBao `client_token` + one trailing newline. Prefix `s.Sp…` (first four characters only; the value was never printed) and the token string does **not** occur in stdout; - the file was shredded after use (`shred -u`) and the path is gone; - honest caveat: the token's *usability* was not separately proven by a `lookup-self` (this run permits no OpenBao read route). The helper's own success criterion is a non-empty `client_token` inside a 200 OIDC login response, which it now gets — it exits non-zero on an empty token, and it did not. ### Refusals — PROVEN Re-run on resume as a smoke check: a pre-existing `0640` file with content -> non-zero exit, stderr `refusing: output path already exists (no overwrite, symlinks included)`, the file's mode and content **intact**, zero bytes on stdout. Carried forward from the original test round, each non-zero exit with the filesystem untouched: no-arg, `-`, `/dev/stdout`, `/dev/fd/1`, tty-on-stdout, pre-existing file, symlink to `/dev/stdout`, dangling symlink, directory, two arguments, missing parent directory. Mint-path failure cleanup: with `BAO_ADDR=https://127.0.0.1:1` the claimed file is created and then removed and nothing is printed. `bash -n` and `shellcheck -S warning` are clean on the helper. ## Callers The two repo-managed callers are in **`Homelab/prod`**, so they cannot land in this PR; they come in the paired PR from branch `plane/IR-20261111-baohelper-callers`: - `scripts/maintenance/window-run.sh` (`pve_load_creds`) — passes an output file inside the existing 0700 `$PVE_TMP`, reads it back, shreds it before the `bao kv get`; - `values/forgejo/artifact-retention-evidence.md` — doc prose, new invocation form. Roughly 80 further references live under `working-files/`, which is not a git repository: the executable ones are probe/rotation scripts and reports. They are **not** migrated here and will break on stdout capture once the host helper is replaced — that inventory is recorded on the run's ticketing and is follow-up work, not a claim of this PR. ## Rollout / risk - Nothing is installed by this PR; the host file is untouched. Merging changes no live behaviour until the role task is applied (or the file copied by hand). - Ordering matters in the other direction: the `prod` caller PR must land **after** the new helper is on the host, or `pve_load_creds` fails closed (no PVE credentials, phase blocked). It cannot leak: it never echoes the helper's stdout, only its byte count. - Rollback: restore the previous helper file and revert the caller PR; the mint logic in this branch is unchanged from the hand-installed original, so the old and new helper are interchangeable apart from the output sink. ## Review Two independent census seats are requested on this head (see the run's seat briefs). No merge is performed by the worker that opened this PR. ## Fix round (review-93 / homelab-93 seat) — head `714522c9` The independent seat review (working-files/security-hypervisor/pr943/review-93.md) raised one blocking finding and three should/minor fixes. All six items are addressed on this head, as two new commits on top of `b69148ce`: 1. **B1 (blocking) — refusal fix, own commit `9318f68`:** any argument starting with `-` (`--help`, `-h`, anything) is refused before any mint, usage to stderr, rc 2. Also required: absolute path; parent dir must exist, be owned by the caller's uid, and be mode 0700 (the swap window from S2). 2. **S1 — no secrets in argv:** the Zitadel JWT assertion and the id_token now reach curl inside 0600 temp files via `-d @file`; revoke-self runs in-process over urllib. No token material ever appears in any process argv. 3. **S3 — revoke on post-mint failure:** the claim, both POSTs and the write run in ONE python process; any failure after the mint (unparseable ttl, write error, closed stdout, SIGTERM/SIGHUP/SIGINT) revokes the token (revoke-self with the minted token) and removes the file before a non-zero exit. 4. **S2 — reopen race closed:** the token is written through the very fd that was O_EXCL-created; no path re-open between claim and write. 5. **R1/R2 — ansible:** the helper task carries its own tag `bao-helper` (install with `--tags bao-helper`; the whole role no longer has to run first); any existing host copy is backed up to `/usr/local/bin/bao-operator-token.pre-943` before replacement. `roles/mgmt-tools/README.md` documents the one-file sha256-checked manual install as the runbook alternative, plus a post-install proof. 6. **Test matrix (fix 6):** an offline fake-binary harness (`roles/mgmt-tools/tests/test-bao-operator-token.sh`) — a stub `curl` first in PATH records every argv; PATH excludes the host copy entirely (plus a tripwire stub); a 127.0.0.1 recorder stands in for the vault address on the revoke path. 39/39 checks pass: every refusal path (zero mint attempts), argv-freedom (recorded stub argv proven free of assertion/id_token/token), `-d @file` usage, revoke-self ordering with the minted token + file removal, a live symlink-swap attack leaving the victim untouched (same-fd proof), O_EXCL structural assertions, `bash -n` and `shellcheck` clean. No real mint was executed in any test. Old-form callers (~20 out-of-repo scripts/docs) are inventoried at `working-files/security-hypervisor/orch/old-form-callers.md` for follow-up; the critical landing routes never called this helper (review-93 §4a) and are unaffected. --- ## Fix round 2 — head `27b52f38` (review-93 N1–N6) Second reviewer pass on `714522c9` found six items; all are addressed in new commit `27b52f38` (trailer `Lab-Run: IR-20261111-baohelper`): - **N1 (blocking)** — the pre-replace backup no longer keeps the old token-on-stdout helper runnable: it goes to `/root/bao-operator-token.pre-943` (root:root, **0400**, outside PATH) via `stat` + `when`, so a fresh VM skips it cleanly; the README manual route uses `install -m 0400` (no `cp -a`). - **N2** — `on_signal` now `rmtree`s the temp dir (key.pem copy of the operator key, zform assertion, login.json id_token) before `os._exit`; no more copies piling up in `/tmp` after SIGINT/SIGTERM/SIGHUP. - **N3** — `sys.stdout.flush()` inside the `try`: a caller-closed stdout now raises at flush, so the failure is caught in-run (rc 1, token revoked, file removed) instead of surfacing as rc 120 at interpreter exit with a live token left in the file; stdout is dup2'd to /dev/null first so shutdown re-flush cannot overwrite the honest rc. A catch-all after the mint revokes and unlinks on any non-`fail()` failure (e.g. ENOSPC on the fd write). - **N4** — a `claimed` flag: the output file is only removed if this run created it. - **N5** — runbook/README: the install verification compares the installed file's sha256 against `git show <merge-sha>:core/ansible/roles/mgmt-tools/ files/bao-operator-token | sha256sum` (merged-blob provenance, not transfer integrity). - **N6** — the test exports `BAO_ADDR=http://127.0.0.1:9` for the whole suite; new SIGTERM-cleanup and closed-stdout-revoke tests (fake curl `hang` mode, fake clock not needed); fake-binary matrix extended. Verification: `bash -n` + `shellcheck` clean; hermetic offline run (`unshare -rn`, stub curl, dead 127.0.0.1 BAO_ADDR, /usr/local/bin excluded, real helper never executed): **pass=50 fail=0**. --- ## Fix round 3 — head `c251af61` (review-3c S1/S2/n1/n2/n4/n6) Third reviewer pass (seat 3c, round 3) on `27b52f38` had no blocking hole; its two should-fixes and four nits are addressed in new commit `c251af61` (trailer `Lab-Run: IR-20261111-baohelper`), citing `review-3c-r3.md`: - **S1** — local cleanup now runs BEFORE the network revoke in `fail()`, `on_signal()` and the post-mint catch-all: the claimed output file is unlinked and the temp dir (key.pem copy, zform assertion, login.json id_token) rmtree'd first; only then the blocking revoke-self (urlopen, 15 s timeout). A TERM→KILL grace window shorter than the revoke wait can no longer leave the token file or key material on disk. - **S2** — the README post-install proof now STARTS with a hard gate: a sha256 check of `/usr/local/bin/bao-operator-token` against `git show <merge-sha>:core/ansible/roles/mgmt-tools/files/bao-operator-token | sha256sum`, stopping on mismatch BEFORE any invocation (`--help` included) — the host may still run the old token-on-stdout helper, and any invocation of a not-actually-installed file would leak a live token. - **n1** — `revoked=True` is set only after a successful revoke-self; on failure the helper prints exactly one stderr line `warning: revoke-self failed; token expires at its ttl` (guarded against double-print from the catch-all re-entry). - **n2** — picked option A: delivery clears `state["claimed"]` (and the token reference), so a signal landing in the finally/shutdown window keeps the caller's delivered file and revokes nothing; token ownership passed to the caller at delivery. - **n4** — README rollback restores the backup at mode **0755 root:root** via `sudo install` (0400 would fail closed but non-executable); the manual route's backup-exists check uses `sudo test -f` (/root is 0700, so the old unprivileged check always read "absent" and a second runbook pass could overwrite the backup). - **n6** — two new hermetic tests: SIGTERM **after** delivery (LD_PRELOAD hold on the first `bao-op-*` unlink/rmdir keeps the helper in the post-delivery `finally` window; asserts the caller's file survives, temp dir rmtree'd, no revoke attempted) and the catch-all with `client_token` as an int (asserts rc 1, TypeError surfaced, file removed, revoke-self carried the minted token). Worktree hygiene: the untracked Syncthing draft `files/bao-operator-token.sync-conflict-20261111-144623-P5KCD5R` (statted: 0755, 12020 B, in no commit) was deleted, not committed (review-3c n5). Verification: `bash -n` + `shellcheck` clean; hermetic offline run (`unshare -rnm`, stub curl, `BAO_ADDR=http://127.0.0.1:9`, /usr/local/bin excluded, real helper never executed): **pass=58 fail=0**.
IR-20261111-baohelper: fix(bao-operator-token): file-only token sink, never stdout
All checks were successful
CI Review / runner-manifest-test (pull_request) Successful in 6s
Details
CI Review / receiver-test (pull_request) Successful in 5s
Details
CI Review / helm-validate (pull_request) Successful in 9s
Details
Build Lab Director Dispatcher Image / preflight (pull_request) Successful in 7s
Details
Lint & Validate / shellcheck (pull_request) Successful in 10s
Details
CI Review / alert-receiver-test (pull_request) Successful in 21s
Details
Lint & Validate / yaml-lint (pull_request) Successful in 26s
Details
CI Review / record-sql-test (pull_request) Successful in 58s
Details
Lint & Validate / terraform-validate (pull_request) Successful in 2m29s
Details
Build Lab Director Dispatcher Image / build-and-push (pull_request) Successful in 2m24s
Details
CI Review / dispatcher-test (pull_request) Successful in 2m54s
Details
CI Review / cockpit-test (pull_request) Successful in 3m7s
Details
CI Review / cockpit-e2e (pull_request) Successful in 3m21s
Details
b69148ce30 
The host helper /usr/local/bin/bao-operator-token (hand-installed, 755,
root-owned) minted a live sa-operator OpenBao token (1h TTL, read/write all
of kv) and printed it to stdout. On 2026-11-11 a worker captured that stdout
into a transcript; harness redaction contained the value, but the helper's
contract is the defect.

New contract, enforced in the helper:

- exactly one argument: the output file path (required);
- the token is written to that path, created mode 0600 with O_EXCL, so an
  existing path is refused (regular file, symlink including a dangling one,
  directory or device) and the caller's file is never touched on refusal;
- `-`, /dev/stdout, /dev/fd/1, /proc/*/fd/1 and a tty are refused outright;
- stdout is refused as a sink, and a run with a tty on stdout refuses;
- the program prints only the non-secret line `written <path> ttl <n>`;
- the mint logic (machine-user JWT -> Zitadel jwt-bearer -> OpenBao
  sa-operator OIDC login) is otherwise unchanged.

The role file is the managed replacement for the hand-installed host copy:
core/ansible/roles/mgmt-tools/files/bao-operator-token is installed to
/usr/local/bin/bao-operator-token by a new copy task. Nothing is applied
here; the host copy is replaced when the mgmt-tools role next runs.

Proven by test: no-argument, `-`, /dev/stdout, /dev/fd/1, tty-on-stdout,
pre-existing file (content preserved), symlink, dangling symlink, directory,
two arguments, missing parent — all refuse with a non-zero status and leave
the filesystem untouched; a mint whose OpenBao login fails removes only the
file it claimed. The success path could not be exercised at build time:
Zitadel's jwt-bearer exchange returns HTTP 500 Errors.Internal for the
`openbao-operator` machine user (upstream identity issue, retried, not this
change); the same flow minted successfully earlier the same day.

Lab-Run: IR-20261111-baohelper
ld-reviewer commented 2026-11-11 18:24:14 +00:00
Collaborator
Copy link

APPROVE — ld-reviewer — head b69148ce30

VERDICT: APPROVED. Independent census read: fetched refs/pull/943/head over the seat read route → b69148ce303523e34b8f6e924d71066e6e02f9d3, exact match to the pin (no HEAD-MISMATCH). Single commit, parent 35a336157, last non-empty line of the message is exactly Lab-Run: IR-20261111-baohelper. CI on this sha: 13/13 contexts success, combined state success (incl. Lint & Validate / shellcheck and yaml-lint). Scope: 2 files, +139/−0, new file mode 100755, tasks/main.yml at 100644. I ran bash -n (parse only) and the helper was NOT executed — nothing minted by this seat.

(a) Sink contract — PASS. Lines 48–58 run in this order: arg count == 1, non-empty, -, then the case glob /dev/stdout|/dev/fd/1|/dev/fd/2|/dev/stderr|/dev/tty|/dev/console|/proc/*/fd/1|/proc/*/fd/2, then [ -t 1 ] — all before any network call or key read, so no reachable path mints before sink validation. The claim (lines 64–81) is the real backstop: islink(path) or exists(path) refuses, then os.open(O_WRONLY|O_CREAT|O_EXCL, 0o600). A symlink (dangling or not), directory and device all trip exists/islink or FileExistsError; -t 1 refuses the tty-on-stdout case; missing parent falls into the caught OSError (non-zero, nothing created). OUT_CLAIMED is set only after the claim returns 0, so a refusal cannot arm cleanup.

(b) Token to stdout — PASS for the stated contract. Whole chain walked: JWT build (py, writes only $TMPD/in and $TMPD/key.pem), openssl dgst -sign with -out (no stdout), IDT captured from the jwt-bearer response via $( ), and the login response piped curl -> python -> file with no intermediate shell var. Only stdout-producing statements are the written %s ttl %d line (123) and $( )-internal prints; die and the $KEY not found message go to >&2. Under set -euo pipefail: openssl failure exits, jwt-bearer failure empties python's stdin (traceback on stderr, pipefail exits), and a mid-write or print failure leaves OUT_WRITTEN=0 so cleanup deletes the token file — fail-closed throughout.

(c) Only output — PASS: print("written %s ttl %d" % (path, ttl)) with ttl = int(auth.get("lease_duration") or 0) (122–123), no other stdout on the success path.

(d) Cleanup safety — PASS: TMPD is only ever a mktemp -d value and is -n-guarded before rm -rf -- "$TMPD" (35), so it cannot escape its own dir; the removal (38) requires OUT_CLAIMED -eq 1 AND OUT_WRITTEN -ne 1, so a refused run leaves a pre-existing path untouched (traced: every die precedes OUT_CLAIMED=1) and a completed run leaves the caller's file in place. trap - EXIT INT TERM HUP PIPE before re-exit prevents recursion; INT/TERM/HUP/PIPE map to 130/143 and still land in cleanup.

(e) Mint logic — UNCHANGED as described: $HOME/.config/openbao-operator-key.json → RS256 machine-jwt → POST /oauth/v2/token grant_type=…jwt-bearer → POST ${BAO_ADDR:-https://openbao.aaron.reynoza.org}/v1/auth/oidc/login with role: sa-operator. No change to auth flow, role, audience or key path.

(f) Ansible task — PASS: src: bao-operator-token resolves to the role's files/ dir (that tree already existed), dest: /usr/local/bin/bao-operator-token matches the hand-installed path, owner/group: root, mode: "0755" quoted, copy is checksum-idempotent, appended as the last of the now-24 tasks with no when/command, so it cannot mint at playbook time and depends on nothing the earlier tasks must provide. Honestly NOT applied: no workflow, ArgoCD app or scheduled job runs it — mgmt-tools is included only by core/ansible/playbooks/setup-mgmt-vm.yml, which is invoked from a manual runbook.

(g) Test evidence vs what the code allows — consistent, no overstatement. written <path> ttl 3600 is exactly what 123 emits from lease_duration; 27 bytes = token + "\n" (119); 0600 from birth holds because the open mode is masked by umask but line 79 (and 118) chmod it back up to exactly 0600; the carried-forward refusal matrix maps 1:1 onto the case/-t 1/claim branches; the 0640 pre-existing-file case is exactly the islink or exists refusal with OUT_CLAIMED=0; the BAO_ADDR=127.0.0.1:1 cleanup case is the OUT_CLAIMED=1, OUT_WRITTEN=0 branch. The "usability not proven by lookup-self" caveat is present and correctly scoped. bash -n clean (re-verified) and shellcheck green in CI.

Blast radius / callers — honest, no false claim. Nothing is installed by this merge, and the body explicitly does NOT claim all callers are migrated: it names the ~80 working-files/ references as un-migrated follow-up that will break on stdout capture. I confirmed this repo has zero references to the helper outside the two added files, so "callers live in Homelab/prod" checks out. Ordering (prod caller PR after the helper is on the host) is stated, not glossed.

FLAGS — non-blocking, for follow-up (none of these change this verdict):

  1. IDT (99–101) and the signed JWT assertion ASSERTION (98) do enter shell variables and, worse, curl argv via -d "assertion=$ASSERTION" (101) and -d "…\"jwt\":\"$IDT\"" (105) — visible in /proc/<pid>/cmdline and ps for the life of those two POSTs, on the host where this mints a kv-wide token. The id_token is exchangeable 1:1 for that token, so it is credential-equivalent, and the commit message itself condemns "a process boundary". The body's "The token … never enters a shell variable" is accurate for the client_token only. Not blocking here because this is the unchanged original mint chain (leg e puts auth-flow changes out of scope) and the incident vector was stdout capture; the fix is transport-level only — feed curl from stdin (--data @-) and the exposure disappears. Worth a ticket on HOMELAB-1893.
  2. TOCTOU at 116: the writer re-opens path by name with O_WRONLY|O_TRUNC and no O_NOFOLLOW, after the claim closed its fd at 78. Harmless in the documented mktemp -d 0700 dir, but in a dir writable by another user the claimed file can be swapped for a symlink or FIFO and the token lands on an attacker-selected target (the fchmod bounds mode, not the target). Cleanest fix: keep the claim fd and hand it to the writer (os.fdopen(int(sys.argv[2]), "w")).
  3. Doc nit, line 7: the usage example teaches export BAO_TOKEN="$(cat …)", which keeps the token in the caller's environment for the process lifetime (children, env dumps, core files). A one-line caveat preferring a short-lived read would be more honest about the residual exposure.

Operational note (disclosed, not a defect): because the copy task now sits in mgmt-tools, the next manual setup-mgmt-vm.yml run swaps the host helper — at that instant the ~80 un-migrated callers fail closed. The body says exactly this and it matches the role's inclusion graph; just sequence the caller migration before the next mgmt-vm playbook run.

model: omniroute/deepseek-flash
session: IR-20261111-baohelper-seatB-20261111T1820Z

**APPROVE — ld-reviewer — head b69148ce303523e34b8f6e924d71066e6e02f9d3** VERDICT: APPROVED. Independent census read: fetched `refs/pull/943/head` over the seat read route → `b69148ce303523e34b8f6e924d71066e6e02f9d3`, exact match to the pin (no HEAD-MISMATCH). Single commit, parent `35a336157`, last non-empty line of the message is exactly `Lab-Run: IR-20261111-baohelper`. CI on this sha: 13/13 contexts `success`, combined state `success` (incl. `Lint & Validate / shellcheck` and `yaml-lint`). Scope: 2 files, +139/−0, new file mode 100755, `tasks/main.yml` at 100644. I ran `bash -n` (parse only) and the helper was NOT executed — nothing minted by this seat. (a) Sink contract — PASS. Lines 48–58 run in this order: arg count == 1, non-empty, `-`, then the case glob `/dev/stdout|/dev/fd/1|/dev/fd/2|/dev/stderr|/dev/tty|/dev/console|/proc/*/fd/1|/proc/*/fd/2`, then `[ -t 1 ]` — all before any network call or key read, so no reachable path mints before sink validation. The claim (lines 64–81) is the real backstop: `islink(path) or exists(path)` refuses, then `os.open(O_WRONLY|O_CREAT|O_EXCL, 0o600)`. A symlink (dangling or not), directory and device all trip `exists`/`islink` or `FileExistsError`; `-t 1` refuses the tty-on-stdout case; missing parent falls into the caught `OSError` (non-zero, nothing created). `OUT_CLAIMED` is set only after the claim returns 0, so a refusal cannot arm cleanup. (b) Token to stdout — PASS for the stated contract. Whole chain walked: JWT build (py, writes only `$TMPD/in` and `$TMPD/key.pem`), `openssl dgst -sign` with `-out` (no stdout), `IDT` captured from the jwt-bearer response via `$( )`, and the login response piped `curl -> python -> file` with no intermediate shell var. Only stdout-producing statements are the `written %s ttl %d` line (123) and `$( )`-internal prints; `die` and the `$KEY not found` message go to `>&2`. Under `set -euo pipefail`: openssl failure exits, jwt-bearer failure empties python's stdin (traceback on stderr, pipefail exits), and a mid-write or `print` failure leaves `OUT_WRITTEN=0` so cleanup deletes the token file — fail-closed throughout. (c) Only output — PASS: `print("written %s ttl %d" % (path, ttl))` with `ttl = int(auth.get("lease_duration") or 0)` (122–123), no other stdout on the success path. (d) Cleanup safety — PASS: `TMPD` is only ever a `mktemp -d` value and is `-n`-guarded before `rm -rf -- "$TMPD"` (35), so it cannot escape its own dir; the removal (38) requires `OUT_CLAIMED -eq 1` AND `OUT_WRITTEN -ne 1`, so a refused run leaves a pre-existing path untouched (traced: every `die` precedes `OUT_CLAIMED=1`) and a completed run leaves the caller's file in place. `trap - EXIT INT TERM HUP PIPE` before re-`exit` prevents recursion; INT/TERM/HUP/PIPE map to 130/143 and still land in cleanup. (e) Mint logic — UNCHANGED as described: `$HOME/.config/openbao-operator-key.json` → RS256 machine-jwt → `POST /oauth/v2/token` `grant_type=…jwt-bearer` → `POST ${BAO_ADDR:-https://openbao.aaron.reynoza.org}/v1/auth/oidc/login` with `role: sa-operator`. No change to auth flow, role, audience or key path. (f) Ansible task — PASS: `src: bao-operator-token` resolves to the role's `files/` dir (that tree already existed), `dest: /usr/local/bin/bao-operator-token` matches the hand-installed path, `owner/group: root`, `mode: "0755"` quoted, copy is checksum-idempotent, appended as the last of the now-24 tasks with no `when`/`command`, so it cannot mint at playbook time and depends on nothing the earlier tasks must provide. Honestly NOT applied: no workflow, ArgoCD app or scheduled job runs it — `mgmt-tools` is included only by `core/ansible/playbooks/setup-mgmt-vm.yml`, which is invoked from a manual runbook. (g) Test evidence vs what the code allows — consistent, no overstatement. `written <path> ttl 3600` is exactly what 123 emits from `lease_duration`; 27 bytes = token + `"\n"` (119); 0600 from birth holds because the `open` mode is masked by umask but line 79 (and 118) chmod it back up to exactly 0600; the carried-forward refusal matrix maps 1:1 onto the `case`/`-t 1`/claim branches; the `0640` pre-existing-file case is exactly the `islink or exists` refusal with `OUT_CLAIMED=0`; the `BAO_ADDR=127.0.0.1:1` cleanup case is the `OUT_CLAIMED=1, OUT_WRITTEN=0` branch. The "usability not proven by lookup-self" caveat is present and correctly scoped. `bash -n` clean (re-verified) and shellcheck green in CI. Blast radius / callers — honest, no false claim. Nothing is installed by this merge, and the body explicitly does NOT claim all callers are migrated: it names the ~80 `working-files/` references as un-migrated follow-up that will break on stdout capture. I confirmed this repo has zero references to the helper outside the two added files, so "callers live in Homelab/prod" checks out. Ordering (prod caller PR after the helper is on the host) is stated, not glossed. FLAGS — non-blocking, for follow-up (none of these change this verdict): 1. `IDT` (99–101) and the signed JWT assertion `ASSERTION` (98) do enter shell variables and, worse, curl argv via `-d "assertion=$ASSERTION"` (101) and `-d "…\"jwt\":\"$IDT\""` (105) — visible in `/proc/<pid>/cmdline` and `ps` for the life of those two POSTs, on the host where this mints a kv-wide token. The id_token is exchangeable 1:1 for that token, so it is credential-equivalent, and the commit message itself condemns "a process boundary". The body's "The token … never enters a shell variable" is accurate for the `client_token` only. Not blocking here because this is the unchanged original mint chain (leg e puts auth-flow changes out of scope) and the incident vector was stdout capture; the fix is transport-level only — feed curl from stdin (`--data @-`) and the exposure disappears. Worth a ticket on HOMELAB-1893. 2. TOCTOU at 116: the writer re-opens `path` by name with `O_WRONLY|O_TRUNC` and no `O_NOFOLLOW`, after the claim closed its fd at 78. Harmless in the documented `mktemp -d` 0700 dir, but in a dir writable by another user the claimed file can be swapped for a symlink or FIFO and the token lands on an attacker-selected target (the `fchmod` bounds mode, not the target). Cleanest fix: keep the claim fd and hand it to the writer (`os.fdopen(int(sys.argv[2]), "w")`). 3. Doc nit, line 7: the usage example teaches `export BAO_TOKEN="$(cat …)"`, which keeps the token in the caller's environment for the process lifetime (children, `env` dumps, core files). A one-line caveat preferring a short-lived read would be more honest about the residual exposure. Operational note (disclosed, not a defect): because the copy task now sits in `mgmt-tools`, the next manual `setup-mgmt-vm.yml` run swaps the host helper — at that instant the ~80 un-migrated callers fail closed. The body says exactly this and it matches the role's inclusion graph; just sequence the caller migration before the next mgmt-vm playbook run. model: omniroute/deepseek-flash session: IR-20261111-baohelper-seatB-20261111T1820Z
ld-reviewer commented 2026-11-11 18:24:47 +00:00
Collaborator
Copy link

APPROVE — ld-reviewer — head b69148ce30

APPROVED (legs a–g pass; four non-blocking follow-ups).

Preflight — head is b69148ce30, one commit, last trailer exactly Lab-Run: IR-20261111-baohelper. Combined commit status on this exact sha: success, 13 contexts (Lint & Validate / shellcheck, yaml-lint, terraform-validate, CI Review suite). New blob bao-operator-token is git mode 100755, 5385 B, 125 lines — two files only. Review was read-only: I parsed the fetched blob (bash -n rc 0, grep) and never ran the helper or minted anything.

(a) Sink contract — PASS. [ "$#" -eq 1 ] rejects 0 and 2+; -, /dev/stdout, /dev/fd/1, /dev/fd/2, /dev/stderr, /dev/tty, /dev/console, /proc/*/fd/1|2 are refused, plus [ -t 1 ]. The claim step runs before the key read, mktemp and any network call: python islink/exists pre-check, then os.open(path, O_WRONLY|O_CREAT|O_EXCL, 0o600) + chmod 0600. islink catches a dangling symlink; exists catches regular file, directory, device (and every stdout/tty alias, so an EXISTS refusal is a genuine backstop even for an alias the case list missed); O_EXCL covers a path created between the pre-check and the open. There is no mint-then-validate ordering anywhere, so "minted before sink validation" is unreachable. Refusals are non-zero (die=2, python=1).

(b) Token never on stdout — PASS. Walked the whole chain: JWT built in a quoted heredoc python (writes $TMPD/in, $TMPD/key.pem 0600 — no prints); openssl dgst -sha256 -sign -out "$TMPD/sig" (nothing on stdout); assertion/id_token captured by $(...); login response piped curl → python → file, and the writer prints only the written … line. No set -x/-v, no tee, no echo of any token; die and all error paths go to stderr; a python traceback (e.g. fsync failure) prints source lines, never values; fsync precedes the print and OUT_WRITTEN=1 follows the pipeline, so a failure after the write deletes the claimed file and exits non-zero; with pipefail a curl failure aborts before anything is printed.

(c) Only output — PASS. Exactly one print("written %s ttl %d" % (path, ttl)), ttl = int(auth.get("lease_duration") or 0).

(d) Cleanup does not damage the caller — PASS. EXIT trap clears the traps, preserves and re-exits $?; rm -rf -- "$TMPD" is guarded on non-empty (a failed mktemp -d can never expand to /); the output file is removed only when OUT_CLAIMED=1 && OUT_WRITTEN != 1 && -f "$OUT". A refused run leaves OUT_CLAIMED=0, so a pre-existing path is untouched — and mktemp/key failures happen after the claim but before a write, which correctly removes the empty claimed file.

(e) Mint logic — PASS against the description: self-signed RS256 machine-user JWT (iss/sub = userId, aud Zitadel, exp +600) from ~/.config/openbao-operator-key.json → jwt-bearer grant → /v1/auth/oidc/login role sa-operator. No auth-flow, role, audience or key-path change in the diff. Caveat recorded: the hand-installed original is not in the repo, so "unchanged" is verified against the PR's description, not against a literal old file.

(f) Ansible task — PASS. copy src: bao-operator-token (the file this same commit adds, so the role cannot run ahead of its source) → /usr/local/bin/bao-operator-token, owner/group root, mode: "0755" quoted; no shell, no handler, no mint at playbook time, idempotent. It is the final task, so prerequisites run first. "Not applied" is true: core/ansible/playbooks/setup-mgmt-vm.yml (hosts: mgmt) does include role: mgmt-tools, but no .forgejo/workflows/* job runs ansible-playbook/that playbook, so merging installs nothing.

(g) Evidence vs what the code allows — PASS, no overstatement. Output shape/ttl, 27 B = 26-char s.* token + newline, the refusal stderr string quoted in the body matches the code byte-for-byte, every entry of the carried-forward refusal matrix is reachable in the code (missing-parent = ENOENT on the claim open, OUT_CLAIMED=0, zero stdout), and the BAO_ADDR=https://127.0.0.1:1 case matches the cleanup rule. The usability caveat is stated honestly. Callers: the body does not claim migration or zero breakage — it says the ~80 working-files/ references will break and are follow-up, and that the prod caller PR must land after the helper is on the host. Ordering is correct, blast-radius is accurate (no manifests, ArgoCD, systemd or timer).

Non-blocking observations (follow-up, none block this head):

  1. The write step uses os.open(path, O_WRONLY|O_TRUNC) without O_NOFOLLOW, so a writer in the parent directory could swap the claimed file for a symlink between claim and write (TOCTOU). Callers use a 0700 mktemp -d, so the exposure needs a hostile user in a shared parent; one flag (or keeping the claim fd) closes it.
  2. The assertion and Zitadel id_token do pass through shell variables and curl argv (-d "assertion=$ASSERTION", -d '…"jwt":"'"$IDT"'"'), visible in ps for those calls. That is the short-lived JWT, not the OpenBao client_token; --data-binary @- would remove it but would change mint plumbing, which this PR deliberately does not touch. The body's "the token … never enters a shell variable" is accurate as written about the client_token.
  3. No CI lint coverage: Lint & Validate / shellcheck scans only core/scripts/*.sh and chart scripts, so this extension-less role file is never shellchecked. The body's bash -n / shellcheck -S warning clean is a manual self-report and consistent with the code (my bash -n on the fetched blob: rc 0); adding the file to the lint set is a worthwhile follow-up.
  4. The rollback sentence ("old and new helper interchangeable apart from the output sink") holds for file-reading callers; the stdout-capturing callers are exactly what is not interchangeable — already disclosed in the body, so the risk is recorded rather than hidden.

Blocker legs: none.

model: omniroute/deepseek-flash
session: IR-20261111-baohelper-seatA-20261111T1820Z

**APPROVE — ld-reviewer — head b69148ce303523e34b8f6e924d71066e6e02f9d3** APPROVED (legs a–g pass; four non-blocking follow-ups). Preflight — head is b69148ce303523e34b8f6e924d71066e6e02f9d3, one commit, last trailer exactly `Lab-Run: IR-20261111-baohelper`. Combined commit status on this exact sha: **success**, 13 contexts (Lint & Validate / shellcheck, yaml-lint, terraform-validate, CI Review suite). New blob `bao-operator-token` is git mode 100755, 5385 B, 125 lines — two files only. Review was read-only: I parsed the fetched blob (`bash -n` rc 0, grep) and never ran the helper or minted anything. (a) Sink contract — PASS. `[ "$#" -eq 1 ]` rejects 0 and 2+; `-`, /dev/stdout, /dev/fd/1, /dev/fd/2, /dev/stderr, /dev/tty, /dev/console, /proc/*/fd/1|2 are refused, plus `[ -t 1 ]`. The claim step runs *before* the key read, `mktemp` and any network call: python `islink`/`exists` pre-check, then `os.open(path, O_WRONLY|O_CREAT|O_EXCL, 0o600)` + `chmod 0600`. `islink` catches a dangling symlink; `exists` catches regular file, directory, device (and every stdout/tty alias, so an EXISTS refusal is a genuine backstop even for an alias the case list missed); `O_EXCL` covers a path created between the pre-check and the open. There is no mint-then-validate ordering anywhere, so "minted before sink validation" is unreachable. Refusals are non-zero (die=2, python=1). (b) Token never on stdout — PASS. Walked the whole chain: JWT built in a quoted heredoc python (writes `$TMPD/in`, `$TMPD/key.pem` 0600 — no prints); `openssl dgst -sha256 -sign -out "$TMPD/sig"` (nothing on stdout); assertion/id_token captured by `$(...)`; login response piped curl → python → file, and the writer prints only the `written …` line. No `set -x`/`-v`, no `tee`, no echo of any token; `die` and all error paths go to stderr; a python traceback (e.g. fsync failure) prints source lines, never values; `fsync` precedes the print and `OUT_WRITTEN=1` follows the pipeline, so a failure after the write deletes the claimed file and exits non-zero; with `pipefail` a curl failure aborts before anything is printed. (c) Only output — PASS. Exactly one `print("written %s ttl %d" % (path, ttl))`, `ttl = int(auth.get("lease_duration") or 0)`. (d) Cleanup does not damage the caller — PASS. EXIT trap clears the traps, preserves and re-exits `$?`; `rm -rf -- "$TMPD"` is guarded on non-empty (a failed `mktemp -d` can never expand to `/`); the output file is removed only when `OUT_CLAIMED=1 && OUT_WRITTEN != 1 && -f "$OUT"`. A refused run leaves `OUT_CLAIMED=0`, so a pre-existing path is untouched — and `mktemp`/key failures happen after the claim but before a write, which correctly removes the empty claimed file. (e) Mint logic — PASS against the description: self-signed RS256 machine-user JWT (iss/sub = userId, aud Zitadel, exp +600) from `~/.config/openbao-operator-key.json` → `jwt-bearer` grant → `/v1/auth/oidc/login` role `sa-operator`. No auth-flow, role, audience or key-path change in the diff. Caveat recorded: the hand-installed original is not in the repo, so "unchanged" is verified against the PR's description, not against a literal old file. (f) Ansible task — PASS. `copy src: bao-operator-token` (the file this same commit adds, so the role cannot run ahead of its source) → `/usr/local/bin/bao-operator-token`, `owner/group root`, `mode: "0755"` quoted; no shell, no handler, no mint at playbook time, idempotent. It is the final task, so prerequisites run first. "Not applied" is true: `core/ansible/playbooks/setup-mgmt-vm.yml` (hosts: mgmt) does include `role: mgmt-tools`, but no `.forgejo/workflows/*` job runs `ansible-playbook`/that playbook, so merging installs nothing. (g) Evidence vs what the code allows — PASS, no overstatement. Output shape/ttl, 27 B = 26-char `s.*` token + newline, the refusal stderr string quoted in the body matches the code byte-for-byte, every entry of the carried-forward refusal matrix is reachable in the code (missing-parent = ENOENT on the claim open, `OUT_CLAIMED=0`, zero stdout), and the `BAO_ADDR=https://127.0.0.1:1` case matches the cleanup rule. The usability caveat is stated honestly. Callers: the body does **not** claim migration or zero breakage — it says the ~80 `working-files/` references will break and are follow-up, and that the prod caller PR must land **after** the helper is on the host. Ordering is correct, blast-radius is accurate (no manifests, ArgoCD, systemd or timer). Non-blocking observations (follow-up, none block this head): 1. The write step uses `os.open(path, O_WRONLY|O_TRUNC)` without `O_NOFOLLOW`, so a writer in the parent directory could swap the claimed file for a symlink between claim and write (TOCTOU). Callers use a 0700 `mktemp -d`, so the exposure needs a hostile user in a shared parent; one flag (or keeping the claim fd) closes it. 2. The assertion and Zitadel id_token do pass through shell variables and curl argv (`-d "assertion=$ASSERTION"`, `-d '…"jwt":"'"$IDT"'"'`), visible in `ps` for those calls. That is the short-lived JWT, not the OpenBao client_token; `--data-binary @-` would remove it but would change mint plumbing, which this PR deliberately does not touch. The body's "the token … never enters a shell variable" is accurate as written about the client_token. 3. No CI lint coverage: `Lint & Validate / shellcheck` scans only `core/scripts/*.sh` and chart scripts, so this extension-less role file is never shellchecked. The body's `bash -n` / `shellcheck -S warning` clean is a manual self-report and consistent with the code (my `bash -n` on the fetched blob: rc 0); adding the file to the lint set is a worthwhile follow-up. 4. The rollback sentence ("old and new helper interchangeable apart from the output sink") holds for file-reading callers; the stdout-capturing callers are exactly what is not interchangeable — already disclosed in the body, so the risk is recorded rather than hidden. Blocker legs: none. model: omniroute/deepseek-flash session: IR-20261111-baohelper-seatA-20261111T1820Z
The previous helper treated a leading-dash argument as an output path:
`bao-operator-token --help` (the exact 16:37Z leak habit) minted a live
sa-operator token into ./--help in the caller's cwd — often a git worktree,
one 'git add -A' from committing it, with nothing in the transcript.

Refusals now run BEFORE any mint, all rc 2 with usage on stderr:
- any argument starting with '-' (--help, -h, --anything), not just bare '-';
- relative paths (an absolute path is required);
- parent directory must exist, be owned by the caller's uid, and be mode 0700
  (the mint.sh rule — closes the claim-to-write symlink swap, S2);
- the existing stdout/tty/existing-path refusals are unchanged.

The helper body is restructured in the same commit: the O_EXCL claim, both
curl POSTs and the token write now run in ONE python process so the token is
written through the very fd that was O_EXCL-created (no path re-open), the
Zitadel assertion and id_token ride to curl in 0600 temp files via -d @file
(never argv, review-93 S1), and any failure after the mint revokes the token
(revoke-self, in-process) and removes the file before a non-zero exit (S3).

Lab-Run: IR-20261111-baohelper
IR-20261111-baohelper: test(bao-operator-token): fake-binary matrix; ansible tag, backup and install runbook (review-93 R1/R2, fix 5/6)
All checks were successful
CI Review / receiver-test (pull_request) Successful in 5s
Details
CI Review / runner-manifest-test (pull_request) Successful in 6s
Details
CI Review / helm-validate (pull_request) Successful in 11s
Details
Build Lab Director Dispatcher Image / preflight (pull_request) Successful in 8s
Details
Lint & Validate / shellcheck (pull_request) Successful in 9s
Details
CI Review / alert-receiver-test (pull_request) Successful in 26s
Details
Lint & Validate / yaml-lint (pull_request) Successful in 33s
Details
CI Review / record-sql-test (pull_request) Successful in 1m49s
Details
Lint & Validate / terraform-validate (pull_request) Successful in 3m23s
Details
Build Lab Director Dispatcher Image / build-and-push (pull_request) Successful in 4m37s
Details
CI Review / cockpit-test (pull_request) Successful in 5m37s
Details
CI Review / dispatcher-test (pull_request) Successful in 7m40s
Details
CI Review / cockpit-e2e (pull_request) Successful in 9m3s
Details
714522c951 
Ansible (review-93 R1/R2):
- the helper install task gets its own tag 'bao-helper' so
  'ansible-playbook playbooks/setup-mgmt-vm.yml --tags bao-helper' installs
  only the helper, instead of running the whole role first (an unrelated apt
  pin can stop the play before the helper task is reached);
- any existing host copy is backed up to /usr/local/bin/bao-operator-token.pre-943
  before replacement, so the rollback plan has a real file to restore;
- roles/mgmt-tools/README.md documents the one-file sha256-checked manual
  install as the runbook alternative, plus a post-install proof.

Tests (review-93 fix 6) — offline fake-binary harness, no real mint possible:
- a stub 'curl' first in PATH records every argv; a tripwire stub catches any
  PATH-by-name resolution of the helper; PATH excludes /usr/local/bin so the
  host copy is unreachable;
- refusal matrix: --help/-h/--anything/bare '-'/no-args/two-args (rc 2, usage
  on stderr, zero mint attempts), relative path, /dev/stdout, non-0700 parent,
  unowned parent, existing file (content preserved), symlink, dangling
  symlink, tty stdout;
- success path: 0600 mode, token in the file and never on stdout, stub-curl
  argv proven free of assertion/id_token/token (inspected log), -d @file usage;
- revoke-on-failure: a 127.0.0.1 HTTP recorder stands in for BAO_ADDR and
  asserts revoke-self carried the minted token, and the file is removed;
- same-fd write: a symlink swap between claim and write leaves the victim
  untouched (the write went through the O_EXCL fd, no path re-open);
- static structural assertions + bash -n + shellcheck. 39/39 pass.

Lab-Run: IR-20261111-baohelper
IR-20261111-baohelper: fix(bao-operator-token): signal-safe tempdir cleanup, in-try stdout flush, post-mint catch-all (review-93 N2/N3/N4)
Some checks failed
CI Review / receiver-test (pull_request) Successful in 5s
Details
CI Review / runner-manifest-test (pull_request) Successful in 6s
Details
CI Review / helm-validate (pull_request) Successful in 11s
Details
Build Lab Director Dispatcher Image / preflight (pull_request) Successful in 8s
Details
Lint & Validate / shellcheck (pull_request) Successful in 9s
Details
CI Review / alert-receiver-test (pull_request) Successful in 22s
Details
Lint & Validate / yaml-lint (pull_request) Successful in 23s
Details
CI Review / record-sql-test (pull_request) Successful in 53s
Details
Build Lab Director Dispatcher Image / build-and-push (pull_request) Successful in 1m44s
Details
Lint & Validate / terraform-validate (pull_request) Successful in 2m22s
Details
CI Review / cockpit-test (pull_request) Successful in 2m56s
Details
CI Review / dispatcher-test (pull_request) Successful in 3m24s
Details
CI Review / cockpit-e2e (pull_request) Failing after 6m7s
Details
27b52f38d9 
Helper (review-93 N2/N3/N4, found on head 714522c9):
- on_signal now rmtree's the temp dir before os._exit: SIGINT/SIGTERM/SIGHUP
  no longer leave key.pem (operator private key copy), zform (signed
  assertion) and login.json (id_token) behind in /tmp — the old bash
  helper's trap cleaned its temp dir and this must not regress (N2);
- sys.stdout.flush() INSIDE the try: stdout on a pipe is block-buffered, so
  print() alone never raised and a caller-closed stdout surfaced only at
  interpreter exit (rc 120) AFTER the token was given up — a live token in
  the file with zero revoke calls (N3). The handler also dup2s /dev/null
  over stdout before failing so shutdown re-flush cannot overwrite the
  honest rc 1;
- catch-all after the mint: any non-fail() failure (OSError/ENOSPC on the fd
  write, ...) now revokes the token and removes the file;
- claimed flag: remove_out() only unlinks when this run created the file, so
  a signal in the pre-claim window cannot delete a caller's file (N4).

Ansible (review-93 N1):
- the pre-replace backup goes OUT of PATH to /root/bao-operator-token.pre-943
  (root:root 0400) instead of a mode-preserving copy inside /usr/local/bin,
  where the old token-on-stdout helper stayed one tab-completion away;
- stat + when: skipped cleanly on a fresh VM with no old helper, so the play
  no longer aborts on 'Cannot copy invalid source'.

Tests (review-93 N2/N3/N6):
- export BAO_ADDR=http://127.0.0.1:9 for the whole suite: even an unexpected
  post-mint failure can no longer reach a real OpenBao (N6);
- new SIGTERM-cleanup test: fake curl in 'hang' mode, TMPDIR scratch, asserts
  rc 143 + output removed + bao-op-* dir rmtree'd (key copy/assertion gone);
- new closed-stdout test: pipe read end closed by ':', asserts rc 1 (not 120),
  in-try error on stderr, file removed, and an independent revoke recorded;
- recorder kept across both revoke tests to count independent revokes; lo
  brought up in case the suite runs in a fresh network namespace; structural
  assertions extended (flush-in-try, on_signal rmtree, catch-all, claimed);
- bash -n + shellcheck clean; hermetic fake-binary run: pass=50 fail=0.

Runbook (review-93 N5): the README manual route backs up via
'install -m 0400' to /root (no cp -a) and the expected sha256 now comes from
'git show <merge-sha>:core/ansible/roles/mgmt-tools/files/bao-operator-token
| sha256sum' — provenance of the merged blob, not transfer integrity — with
an installed-file digest comparison step after the scp route.

Lab-Run: IR-20261111-baohelper
IR-20261111-baohelper: fix(bao-operator-token): local-first cleanup, honest revoke, delivery-cleared claim, sha-gated proof (review-3c S1/S2/n1/n2/n4/n6)
All checks were successful
CI Review / receiver-test (pull_request) Successful in 5s
Details
CI Review / runner-manifest-test (pull_request) Successful in 9s
Details
CI Review / helm-validate (pull_request) Successful in 11s
Details
Build Lab Director Dispatcher Image / preflight (pull_request) Successful in 7s
Details
CI Review / alert-receiver-test (pull_request) Successful in 20s
Details
Lint & Validate / shellcheck (pull_request) Successful in 11s
Details
Lint & Validate / yaml-lint (pull_request) Successful in 29s
Details
CI Review / record-sql-test (pull_request) Successful in 59s
Details
Lint & Validate / terraform-validate (pull_request) Successful in 1m57s
Details
CI Review / cockpit-e2e (pull_request) Successful in 2m53s
Details
Build Lab Director Dispatcher Image / build-and-push (pull_request) Successful in 2m41s
Details
CI Review / dispatcher-test (pull_request) Successful in 3m0s
Details
CI Review / cockpit-test (pull_request) Successful in 2m58s
Details
c251af615e 
S1: fail(), on_signal() and the post-mint catch-all now do LOCAL cleanup
first (unlink the claimed output file, rmtree the temp dir holding the
key copy) and only then the blocking network revoke (urlopen, timeout
15s). A TERM->KILL grace window shorter than the revoke wait can no
longer leave the token file or key.pem/zform/login.json on disk.

S2: the README post-install proof now STARTS with a hard gate — a
sha256 check of /usr/local/bin/bao-operator-token against
`git show <merge-sha>:core/ansible/roles/mgmt-tools/files/bao-operator-token
| sha256sum` — and stops on mismatch BEFORE any invocation (--help
included), so a skipped/failed install can't leak a live token from the
old token-on-stdout helper still on the host.

n1: revoked=True is now set only after a successful revoke-self; on
failure the helper prints exactly one stderr line
"warning: revoke-self failed; token expires at its ttl" (guarded so the
catch-all re-entry cannot repeat it).

n2: picked option A — delivery clears state["claimed"] (and drops the
token reference), so a signal landing in the finally/shutdown window
keeps the caller's delivered file and revokes nothing; ownership of the
token passed to the caller at delivery.

n4: README rollback restores the backup with mode 0755 root:root via
sudo; the manual route's backup-exists check now uses `sudo test -f`
(/root is 0700, so the unprivileged test always read "absent" and a
second runbook pass could overwrite the backup).

n6: two new hermetic tests — SIGTERM after delivery (LD_PRELOAD hold on
the first bao-op-* unlink/rmdir keeps the helper in the post-delivery
finally window; asserts the caller's file survives, temp dir is
rmtree'd, no revoke attempted) and the catch-all with client_token as
an int (asserts rc 1, TypeError surfaced, file removed, revoke-self
carried the minted token).

Hermetic suite: 58/58 pass (unshare -rnm, BAO_ADDR=127.0.0.1:9, stub
curl, tripwires); bash -n + shellcheck clean.

Lab-Run: IR-20261111-baohelper
All checks were successful
CI Review / receiver-test (pull_request) Successful in 5s
Details
CI Review / runner-manifest-test (pull_request) Successful in 9s
Details
CI Review / helm-validate (pull_request) Successful in 11s
Required
Details
Build Lab Director Dispatcher Image / preflight (pull_request) Successful in 7s
Details
CI Review / alert-receiver-test (pull_request) Successful in 20s
Details
Lint & Validate / shellcheck (pull_request) Successful in 11s
Required
Details
Lint & Validate / yaml-lint (pull_request) Successful in 29s
Required
Details
CI Review / record-sql-test (pull_request) Successful in 59s
Details
Lint & Validate / terraform-validate (pull_request) Successful in 1m57s
Required
Details
CI Review / cockpit-e2e (pull_request) Successful in 2m53s
Details
Build Lab Director Dispatcher Image / build-and-push (pull_request) Successful in 2m41s
Details
CI Review / dispatcher-test (pull_request) Successful in 3m0s
Details
CI Review / cockpit-test (pull_request) Successful in 2m58s
Details
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin plane/IR-20261111-baohelper:plane/IR-20261111-baohelper
git switch plane/IR-20261111-baohelper
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
operator-review

Requires human operator review before merge — dangerous class (RBAC, secrets, network policy, Terraform) or explicitly gated. Never auto-merge.

No labels
operator-review
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Homelab/infra-core!943
No description provided.