HOMELAB-2479: ci(emu): gate emulator builds on their own inputs + pin both bases #887
No reviewers
Labels
No labels
operator-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Homelab/infra-core!887
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "plane/HOMELAB-2479-emulator-gate"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why (measured, 2026-11-07T03:35–03:50Z)
The emulator image is the single biggest thing this registry is asked to store, and part of that is a trigger bug, not a real change.
crane ls harbor.aaron.reynoza.org/platform/android-emulator5c1407d10-30,3e59ad710-31,39725f611-02,230756a11-02,e54f03b11-07) over 7.82 d = 0.64 builds/dayregistry.artifact.size(platform/android-emulator, 24 h window)kubelet_volume_stats_used_bytes{pvc="harbor-registry"}delta(...[24h])registry.artifact.push_time> 24 h, sumsizeplatform/android-emulator1 artifact 2.52 GiB (61 %),docker-cache/diegosouzapw(omniroute proxy fill) 6 artifacts 1.56 GiB,platform/docs2 artifacts 51 MiBkubelet_volume_stats_available_bytesThe 2.52 GiB in the last 24 h was pushed by
e54f03b— a fixture-only commit (re-sync of the vendoredsshd-fixture.sh). It changed no emulator input, and the emulator was rebuilt and pushed anyway, because the two images share one workflow and the workflow-levelpaths:filter cannot say which image a push affects.3e59ad7(2026-10-31) was the same story.Of the 5 builds in the sample, 2 were fixture-only (
3e59ad7,e54f03b), 2 emulator-only (39725f6,230756a), 1 both (5c1407d, the commit that created both images).What changes (3 files, +100/−3, two commits)
Commit 1 — gating (
de08fa4).changesjob diffs the pushed range (push:github.event.before..github.sha;pull_request:base.sha..HEAD) and publishes two outputs.platform/android-emulator/**→emulator=trueonly.platform/ghostty-ssh-fixture/**→fixture=trueonly. An image path wins, so a fixture fix that also edits the sha256 pin insideemu-images.yamlno longer drags the emulator along (that is exactly whate54f03band3e59ad7did — every one of the 5 commits touched the workflow file).emu-images.yamlalone, with no image path → both, so a workflow-only change still proves both images.build-emulator/build-fixturegate onneeds.changes.outputs.<image> == 'true'.workflow_dispatchgainsbuild_emulator(defaultfalse) andbuild_fixture(defaulttrue), astype: choicestrings — a boolean dispatch input is coerced to a real JSON boolean and the engine's string-schema dispatch validation rejects it with 422 (GITS-314, measured in the app repo). The condition also matches the literal'true', so a real boolean would still work.if: github.event_name != 'pull_request'on both push steps is untouched).Dry run of the shipped decision block against the 5 real live commits (the exact
casestatement that runs in CI):Commit 2 — base pins (
9d1ef2b), separate so it can be dropped alone. Both Dockerfiles wereFROM debian:bookworm-slim— a floating tag, so an upstream debian rebuild changes the base of every layer below it with no commit in this repo. Pinned to the multi-arch index digest read withcrane digestat 03:43Z:(amd64 child
c67e92c8…; the index keeps the manifest list, so a non-amd64 build still resolves.) Those twoFROMlines are the only floating base references in either Dockerfile; every other download (JDK, cmdline-tools, herdr, the vendored script) was already sha256-pinned.Math: what the gate is worth
docker-cacheproxy fill driven by pulls, which this PR does not touch, and node content stores hold a copy of every image the registry does too. This is a rate cut, not a cure.Dispatch: two examples
(the header is fed through a 0600 curl config so the token never lands in argv — HOMELAB-2656/GITS-380.)
Consumers: does anything need a fresh emulator per push? No.
Every consumer pins a digest, so freshness does not matter to any of them:
emulaneHomelab/ghostty-in-the-shell.forgejo/workflows/emu.yaml:73EMU_IMAGEplatform/android-emulator@sha256:66430940…(build230756a, no tag).forgejo/workflows/probe-emu.yaml:61,.forgejo/workflows/release-smoke.yaml:38EMU_IMAGE@sha256:8b0a6b49…(= build5c1407d, 10-30)sbx-gitsHomelab/infra-corecore/manifests/apps/sbx-gits/statefulset.yaml:1073.spec.template.spec.containers[].image:230756a@sha256:66430940…emu.yaml:86,probe-emu.yaml:71FIXTURE_IMAGEghostty-ssh-fixture:3e59ad7@sha256:31ee38da…The only floating reference anywhere is a human recipe in
docs/gits-sbx/emulador-host.md:207(crane ls … | tail -1) — documentation, not CI.Known red on
live, not introduced herebuild-fixturefailed onliveate54f03b(run #3333, 8 s) and no fixture tag was pushed for that commit (crane ls platform/ghostty-ssh-fixture=3e59ad7,5c1407d,gits23-sync-18bbafe7). The failure is in the stepVerify the control API at runtime (add-key + ssh login): afteradd-keysucceeds, the step'sssh -p "${FIX_PORT}"printsBad port '', i.e.sshd-fixture.sh envreturned noFIX_PORTfor a container the entrypoint started withoutFIX_SESSION_NAME— the re-synced shard'sfixture discovery (stop / env)section (platform/ghostty-ssh-fixture/fixtures/sshd-fixture.sh:157+) does not find the session the entrypoint started. Symptom and location are measured; the root cause is unproven (would need the run's full log around discovery, which the 8 s failure excerpt does not carry). It will make this PR's ownbuild-fixturered, exactly as it is red onlivetoday. Filed as the next step of HOMELAB-2479, not fixed here — this PR is deliberately trigger-shaped only.Rollback
git revert 9d1ef2b && git revert de08fa4(orgit revertof the range) — no state, no data, no retention or registry change is involved, so a revert restores today's behaviour exactly.Open questions (for the reviewer and the next ticket)
probe-emu.yaml:61/release-smoke.yaml:38pin8b0a6b49…(5c1407d, 2026-10-30), whileemu.yaml:73andsbx-gitspin66430940…(230756a, 2026-11-02). Re-pinning the two to230756a/e54f03bis what makes the old emulator artifacts reclaimable at all (retention cannot delete a pinned digest). That is the sequenced next read, not this PR.emu-images.yamlwhile also touching a fixture path will build only the fixture. The next emulator-path change proves it, andbuild_emulator=trueforces it by hand. If that is too loose, the alternative is a hunk-range test on the workflow file — a bigger diff than this ticket allows.$GITHUB_OUTPUT/ joboutputs:are used by the newchangesjob. The runner is current, but the first run of this PR is the proof (::set-outputis the fallback form if the engine refuses it).apt-get installpackage versions in both Dockerfiles are still unpinned (a re-pin of those needs a version sweep, not a one-line change).No merge. Reviewer first.
Verdict: APPROVE
9d1ef2b48cmodel: omniroute/deepseek-flash
session: 01a1b5c8-851e-76f0-828e-7f80b7d1a776
Seat 1 of 2 (independent; wrote this without reading the other seat).
Verified (measured, re-run myself)
caseblock againstgit diff --name-only <c>^ <c>for all five commits:5c1407demu=T fix=T ·3e59ad7emu=F fix=T ·39725f6emu=T fix=F ·230756aemu=T fix=F ·e54f03bemu=F fix=T. All five are merge commits, so per-commit diff equals the push rangebefore..sha.crane digest debian:bookworm-slimtoday =sha256:734a72b3…b179bc— the pinned value. Multi-arch claim holds: amd64 childsha256:c67e92c8…, arm64sha256:ec8534cd…, so the index digest keeps non-amd64 builds resolvable.FROMper Dockerfile, both pinned; nothing else floats inside those two files.docker pushand bothcrane digeststeps still carryif: github.event_name != 'pull_request'(lines 284, 302, 419, 431);HARBOR_PUSH_*appears only inside them. PR builds push nothing.changes✓ 9s,build-emulator✓ 5m39s,build-fixture✓ 1m28s; yaml-lint, shellcheck, terraform-validate and the unit/e2e workflow also ✓. That also settles the PR's own open question 3:$GITHUB_OUTPUT+ joboutputs:work, because an absent output would have skipped both gated jobs.core/manifests/apps/sbx-gits/statefulset.yamlpins:230756a@sha256:66430940…— at line 1084, not 1073. Only floating reference is the docs recipedocs/gits-sbx/emulador-host.md:207(documentation, not CI). GITS-repo pins: unknown, I did not readHomelab/ghostty-in-the-shell.Concerns
emu-images.yamland any fixture path builds only the fixture — a broken emulator verify step can land on live unproven. Bounded (build_emulator=trueforces it), so acceptable, but it should be written down where the next editor will read it.changedis empty,emuandfixboth stay false and both jobs are skipped — the run is green with zero builds and reads as a pass unless you open the job list. A[ -z "$changed" ] && build bothguard closes it.set -euo pipefailwith an unresolvablebefore(force-push rewriting live) turnschangesred and skips both builds. Classifier error ⇒ build both is the safer default.pull_requestrange:"${BASE_SHA}..HEAD"is a tree comparison. Measured on this very branch, base..head lists 12 files (it shows live-only commits as reversions). Over-build only, so fail-safe —"${BASE_SHA}...HEAD"states the intent without depending on branch freshness.build-fixturered" did not reproduce (✓ 1m28s). Consistent with the PR run building the merge ref, which carries live's #886FIX_PORTgate that this branch's merge-base lacks; since the failure is timing-dependent, the non-reproduction proves nothing either way. Safe: the PR diff does not touch that verify step, so the merge result is unaffected.Suggestions
build_emulatorgone.fetch-depth: 0full clone per run on thek8spool, in a ticket about runner/storage saturation — worth a shallow alternative.Evidence:
~/repos/homelab/working-files/infra-stability/evidence-20261107/seat1-887.mdVerdict: APPROVE
9d1ef2b48cmodel: omniroute/deepseek-flash
session: 01a1b5d0-986b-76f0-828e-7f86ed9ed4ce
Adversarial/threat-model pass, primary sources only. No hold-level finding. Evidence:
working-files/infra-stability/evidence-20261107/seat2-887.md.Verified
crane digest debian:bookworm-slimtoday =sha256:734a72b3…b179bc, exactly the pin; amd64 childc67e92c8…matches. Both Dockerfiles are single-stage (oneFROMeach), so no second floating build stage.false/truematch the body's examples.$GITHUB_OUTPUT(open question 3) is proven, not hypothetical: had the writes been refused,set -uwould have aborted the step and both gated jobs would have been skipped. Run #3338changessucceeded in 9 s and both built.git merge origin/live, then aborted): clean. The merged tree keeps the fixture sha256 identical at all three sites (workflow constant, Dockerfile ARG, file) and preserves #886's non-empty-FIX_PORTgate and 601a425c's unique container name. No pin conflict, no re-introduced broken script.concern —
changesfailure fails to nothing. A bad range, checkout failure or force-pushedbeforemakesgit diffexit 128; the step dies underset -e, outputs stay empty, andneeds: changesskips both builds. Whether that surfaces as a missing required check is unknown:GET /branch_protectionsis refused for this credential. Missing evidence: that API as an admin, and one run with a deliberately bad range.concern — the PR range is two-dot.
base.sha..HEADis a tree diff, not "what the PR changed": on this head it lists 12 files (body says 3), including-259lines of asshd-fixture.shthe PR never touches. Over-report on PRs costs builds only. The permissive direction is measured:--name-onlyprints only a rename's destination, so moving an input out of an image dir under-reports and skips a needed rebuild.concern — the body's "known red on live" is stale. Live #3346 (#886) shows
build-fixture success 1m8s; the red run was #3333, and #886 postdates the branch point. This head carries the other script version, so the prediction is wrong both ways. Green here proves the gate, not live's fixture job.concern —
actions/checkout@v4is now the only floating reference in this workflow (a fourth use is added here), while every download and both bases are pinned.suggestion A
false/falsedispatch is a successful no-op run; consider refusing it. suggestion State a debian pin-bump cadence, or the fresh pin rots silently; the unpinnedapt-getversions (open question 4) are the largest remaining float, correctly out of scope.Open questions 1: agree it is the sequenced next read. 2: accept — forcing both on any workflow edit would cost 2.5 GiB on 5 of 5 sampled commits; the dispatch input is a real escape hatch. 3: answered above. 4: out of scope.