HOMELAB-1735: ci: env-block Harbor push secrets (act substitutes textually) #522
No reviewers
Labels
No labels
operator-review
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Homelab/infra-core!522
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "plane/HOMELAB-1735-env-fix"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Defect B — the publish path cannot carry a
$-bearing usernameThe Harbor robot credentials are now correct in the repo (OpenBao-sourced, byte-verified, registry probe 200), yet the publish still failed. Run 5675 / index 1491 (
workflow_dispatchonlive@dd99f5c):Forgejo/act substitutes
${{ secrets.X }}textually into the generated shell script. The step was:so the script line became
-u "robot$forgejo-runner-push"and bash expanded$forgejo— a hard abort underset -u(run 1491), or a silent degradation torobot-runner-push→401 unauthorizedwithout it (defect A, run 1490). Robot names always contain$, so this is the first credential in this repo that cannot be safely inlined. Previous run of the same workflow on 2026-09-22 loggedLogin Succeededonly because that credential was$-free.Fix
Move both secrets into a step-level
env:block and reference$HARBOR_PUSH_USERNAME/$HARBOR_PUSH_PASSWORDin the shell. Env values are handed to the step process verbatim and the shell never re-expands them, so the secret value never enters the script text.Applied to
.forgejo/workflows/runner-image-build.yamland.forgejo/workflows/docs-build.yaml(both carry the identicalHARBOR_PUSH_*login). Nothing else changes — the mktemp-DOCKER_CONFIG/ no-logout structure from #521 is untouched. In-repo precedent:release.yamlalready passes an arbitrary PR body (github.event.pull_request.body) through a stepenv:block.Why this is safe (mechanism, not assumption)
Forgejo runner v12 host environment executes steps via
exec.CommandContextwithcmd.Env = getEnvListFromMap(env)(act/container/host_environment.go); step-levelenv:is merged into that map (act/runner/step.go: mergeEnv). Env values areKEY=valueentries in the child process environment — no shell parses them.Verify plan
forgejo: unbound variable; fixed form logsLogin Succeededwith the byte-exactrobot$forgejo-runner-pushand the rendered script contains no secret value.yamllintclean (one pre-existing long line removed)..forgejo/workflows/).on.push.paths, so merging toliveself-triggers the runner-image publish at the new merge sha. That run must logLogin Succeeded, push, andcrane digest harbor.aaron.reynoza.org/platform/forgejo-runner:<new-sha>must resolve.docs-build.yamldoes not self-trigger (itspaths:excludes itself) — it will prove out on the nextdocs/**push. Identical env block; the rendered-script proof covers it.Local evidence
No cluster mutation, no pod restart, no secret value printed.
Note on a stale assumption:
pi-image-build.yamlis the precedent for the mktemp-DOCKER_CONFIG/no-logout structure, but it still inlines the secrets — it only works becauseHARBOR_USERNAMEis$-free. Theenv:block is the new part.Forgejo/act substitutes `${{ secrets.X }}` TEXTUALLY into the generated step script, so inlining `robot$forgejo-runner-push` produced a script line `-u "robot$forgejo-runner-push"` and bash expanded `$forgejo`: "forgejo: unbound variable" under `set -u` (run 1491, Push to Harbor step). Without `set -u` it silently degrades to `robot-runner-push` -> 401. Pass the two secrets through a step-level `env:` block and reference `$HARBOR_PUSH_USERNAME` / `$HARBOR_PUSH_PASSWORD` in the shell instead. Env values are handed to the step process verbatim (runner v12 host env: `cmd.Env = getEnvListFromMap(env)`) and are never re-expanded by the shell, so the secret value never enters the script text. Same structure as `.forgejo/workflows/release.yaml` (which already passes an arbitrary PR body through a step `env:` block). Both workflows' Harbor login/push steps; nothing else changes.The reviewed wording said env values are handed to the step 'verbatim'. They are passed as process environment variables (never re-parsed by the shell), but the runner does interpolate a value's ${{ }} expressions once first. State the accurate, narrower claim. Comment-only; no behavioural change.Independent review — VERDICT: CLEAN (no blockers)
Fresh-context reviewer, separate process (
pi -p --no-session, different model from the author), briefed only on the diff + the defect story + the binding constraints, working from an isolated clone at the PR head. It was not the implementer and was not handed the implementer's summary as its brief.Mechanism claim verified against a primary source, not asserted
The reviewer read the runner source (
code.forgejo.org/forgejo/runner/v12@v12.6.4) and confirmed the fix works for the code path these jobs actually use (:hostmode →HostEnvironment.exec):step.Runis interpolated into the generated script textact/runner/step_run.gosetupShellCommandenv:is merged last (wins over job/workflow env) and its value interpolated onceact/runner/step.gosetupEnv/mergeEnvact/container/host_environment.go:330cmd.Env = getEnvListFromMap(env)Interpolateis single-pass (the result is never re-scanned)act/runner/expression.go:377env:are still masked in logsact/runner/logger.go:119,185valueMasker(… Config.Secrets)— no new leakageVerification the reviewer performed
git diff -widentical to plain diff; no mode changes, no whitespace errors, no unrelated reformatting.DOCKER_CONFIG="$(mktemp -d)", the absence ofdocker logout, the digest step and the docs deploy steps are byte-identical to the base.run:→ zero secret expressions in any script; all remainingsecrets.*occurrences are the 2env:values per file.env:is step-level, sibling ofrun:, attached only toPush to Harbor; no workflow/job-levelenv:exists to shadow it.$HARBOR_*refs in the scripts are exactly the declaredenv:keys — no typos. Repo-widegrep HARBOR_PUSH→ only these two files.set -udoes not fire — old and new both produce-u "". No new abort path; strictly an improvement.Concerns (non-blocking)
pi-image-build.yaml:131-132still inlinessecrets.HARBOR_PASSWORD/HARBOR_USERNAME— same latent defect class. Latent, not live: that credential is the Harbor admin ($-free), unlikerobot$…. Out of scope for this PR (constraint: two files only) → follow-up.${{ }}expressions once. Addressed inf031bf8, which restates the accurate narrower claim (passed as process environment variables, so the shell never re-parses them). Comment-only delta.echo "$PW" | docker login— unchanged semantics.Delta after review
f031bf8is a comment-only delta over the reviewed51ee8bf— asserted programmatically: every changed line ingit diff 51ee8bf f031bf8is a YAML comment (zero non-comment lines). Behaviour is identical, so the reviewed verdict carries.No cluster mutation, no pod restart, no secret value printed by the review or the change.
Landed — merge
7597371, publish green, image live at a new tagMerged:
75973717f0dddcb183c34a8b64bf279058fc34ba(merge commit, PR headdc2c33f).CI on the PR head: 5/5 green (terraform-validate 2m42s, yaml-lint, shellcheck, helm-validate, receiver-test).
GitOps continuation: the merge self-triggered the publish, and it succeeded
.forgejo/workflows/runner-image-build.yamllists itself inon.push.paths, so the push toliveran the workflow at the new merge sha. Run index 1499, jobbuild-and-push, eventpush, job succeeded:Note
Login Succeeded— the exact line whose absence was the evidence for defect A in run 1490. And the login wrote to/tmp/tmp.RpNZv1BMrR/config.json, i.e. the throwawayDOCKER_CONFIG, not the pod-shared/home/runner/.docker(the HOMELAB-1735 footgun half is preserved).Registry verification (read-only
crane, independent of the job log)crane digest …/platform/forgejo-runner:7597371sha256:964448352603bd7038e9d60896dc6e583444bb613232c9a56ad5cb73f3401d17— byte-identical to the digest the job printed (log ↔ registry cross-check)crane ls7597371), 9 tags totalapplication/vnd.docker.distribution.manifest.v2+jsoncrane digest …:dd99f5cNOT_FOUND(as expected — the workflow atdd99f5cis the broken one)crane digest …:latestsha256:c7239a38…(this workflow only publishes the short-sha tag)One CI hiccup, diagnosed and cleared
First CI run on
f031bf8went red onterraform-validate— unrelated to this PR (proof:git diff dd99f5c f031bf8 -- core/is empty andlint.yamlis untouched). Log evidence: in the last dir,terraform init -backend=falseinstalled 6/7 providers and stopped atInstalling bpg/proxmox v0.114.0...with no error line (the job discards init's stderr via2>/dev/null). It was transient: the identical job succeeded in 2m49s on4646fd7, all 8 dirs pass locally minutes later includingbpg/proxmox v0.114.0, and the re-trigger passed. Cleared with an emptyci: re-trigger CIcommit (dc2c33f), matching this repo's existing convention (d5f51db,37ee5ee). Diff vsdd99f5cis unchanged: the same two files.Scope as merged
.forgejo/workflows/runner-image-build.yaml+.forgejo/workflows/docs-build.yaml, 3 commits (2 substantive + 1 CI re-trigger). No other repo, no cluster mutation, no pod restart, no secret value printed.docs-build.yamlwas not triggered by this merge — itson.push.pathsaredocs/**,mkdocs.yml,Dockerfile.docs,Caddyfileand do not include the workflow file itself. It carries the identical env block and the same rendered-script proof, and will prove out on the next docs push.Remaining (operator)
Cycle the 8 runner pods at a maintenance moment, then the ticket's mtime/
.authscheck on a restarted pod.