HOMELAB-1686: LD4.2 meeting pipeline — dispatcher convening, whisper-asr image, mgmt ingest #577
No reviewers
Labels
No labels
operator-review
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Homelab/infra-core!577
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "plane/HOMELAB-1686-meetings"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
LD4.2 · HOMELAB-1686 (epic LD4, slot 6). Builder PR. Do not merge: this is a
landing-pipeline PR and three of its slices are E-adj (see below).
Brief:
.rpiv/artifacts/epic-ld4/design-LD4.2-meetings.md. Orchestrator rulings onthe ticket supersede it where they differ.
1. First slice — the roundtable-class verdict (commit 1)
The orchestrator assigned the brief's riskiest assumption to the builder, verified
first: does a roundtable sandbox class exist, and can it reach Outline?
Verdict: DISPROVED on both halves. Full evidence in
lab-director/dispatcher/docs/ld4.2-roundtable-class-verdict.md; the grep-verifiedfacts are:
SandboxTemplates exist (ld-sandbox,ld-build-test,ld-browser,ld-firmware,ld-meet-ai,ld-claude-code) andsandbox-init/roles.jsondeclares two roles (
light,builder-core). No roundtable class, no plannerclass, no role.
OTel and the sandbox MCP. Outline appears nowhere, and neither do Plane or
the NATS bus for the record's write path.
this slice builds on.
Consequence (the documented fallback, stated loudly): the roundtable's Outline
write is not wired. The record's durable home in v0 is the meeting-record Plane
item. Adding a class means a new
SandboxTemplate+ aroles.jsonentry + a CNP —all on AGENTS.md's dangerous list, none verifiable by a builder without cluster
access. That is an operator decision, not a builder's, and it is recorded in the
verdict doc, in
prod'sdispatcher-config.yaml, and on the ticket.A second finding fell out of the same read and is more consequential than the
verdict: no sandbox class reaches Plane either. A convened roundtable would run
in a sandbox whose only output channel is the model, and v1's lane watcher closes a
lane with
ReasonNoResult(nothing submits a result bundle yet). So themeetingsarm is shipped OFF in prod with both reasons written out, rather than turned on
to look complete: a spawn whose product cannot come back is visibly busy and
silently empty.
2. What is built
prod/manifests/meet-ai/whisper-asr/+prod/apps/whisper-asr.yamllab-director/whisper-asr/+.forgejo/workflows/whisper-asr-image-build.yamllab-director/dispatcher/{director,intake,config,cmd/dispatcher}director/meeting_record.go,intake.MarkerMeetingRecord, cross-language fixturecore/services/pi-meeting-ingest/(mgmt-local, systemd user unit)cmd/dispatcher/meetings.go(meetingTrigger)Verify:
go test -race ./...+go vet ./...green inlab-director/dispatcher;node --test→ 21/21 incore/services/pi-meeting-ingest.The convening, end to end (fixtures)
cmd/dispatcher/roundtable_pipeline_test.godrives the REAL packages throughfixture seams (Plane read/write, MCP, clock — no socket):
Todo→ exactly ONE MCPCreate, templateld-roundtable, package keyed on the item, one convening stamp;slice exposed:
filesContextSourceignored a resolved lane, so a roundtabledispatch would have compiled class
defaultwhile the spawn arm was handedroundtable, andspawn's preflight refuses that pair.director/plane_convener_test.gocovers find-or-create (one meeting never gets twoticket surrogates), a finder failure (refused, never "not found"), and the marker
grammar.
3. Deviations from the brief, and why
Attendeesis required config on the ingest (MEETING_ATTENDEES), notderived.
events.MeetingCompleted.Gaps()treats an empty attendee list as a gap("an unattributed roundtable is not attributable"), so an ingest without the
series' seats would produce a meeting that never convenes — and that failure
would look like a broken roundtable rather than missing configuration.
Diarization is deferred (ruling), so the seats are stated once by the operator
instead of guessed from the audio.
arms.meetingsships OFF in prod. See §1.dispatcher has no ingress, JetStream is not wired (
direction/ledger.go:386),and the DM arm is a chat trigger. The brief left the transport open; this is the
one channel both ends already share.
asks for both, but requiring the label would let a missing label silently
demote a meeting to ordinary ticket work — the codebase's existing posture
(
intake.PlaneReader.ReadUndigested: "Filtering is by MARKER, not by label") ismarker-only, and the writer still asserts the label.
syncPolicy. The image does not existuntil the build workflow publishes it, and an auto-synced app would
ImagePullBackOff on every clone. Two-edit activation is written out in the app,
the Deployment header, the PR body and the ticket.
orchestrator ruled mgmt drop-dir + sweep; the brief's in-cluster NFS watcher was
rejected because no workload in this cluster mounts the media share, so it would
have meant inventing a storage contract to satisfy an ingest. Precedent:
core/services/pi-signal-bridge/.4. Honest gaps (named, not papered over)
creates the target Outline collection and sets
OUTLINE_COLLECTION_ID. NoMeetingscollection exists today; I did not create one unilaterally in a shareddocs platform. Recommended: create
Meetings, then set the env var.end to end today. The wiring is (fixtures prove one convening per meeting,
keyed on the item), but the roundtable's product has no way back: no class reaches
Plane or Outline, and no result-submission path exists. Unlock order is (a) a
roundtable class + its egress, (b) a result path, (c) the ingest on the prod side.
meetingTrigger.seen) plus theitem's state and a comment stamp. A dispatcher restart between the convening and
the item moving out of
Todowould convene a second run. The durable ledger isthe JetStream follow-up; the find-or-create in
PlaneConvenerkeeps "one meeting,one record item" true regardless.
v1.9.4, bakesggml-large-v3-turbo-q5_0and verifies its sha256(
394221709cd5ad1f40c46e6031ca61bce88931e6e088c188294c6d5a55ffa7e2, read off themgmt VM's own copy), and
GGML_NATIVE=OFFso a CI build runs on wk-01. The CIsmoke test exercises
/healthand/inferenceon a generated WAV — but that hasnot run yet, and the first build is the real test.
5. E-adj (dangerous class — REQUIRING OPERATOR REVIEW)
prod/manifests/meet-ai/whisper-asr/cilium-network-policy.yaml— a newCiliumNetworkPolicy (meet-ai pods on :8171; DNS egress only).
prod/manifests/lab-director/dispatcher-config.yaml— adds theroundtabletemplate binding, a
meetings:block andarms.meetings: false.harbor-robot-meet-aiSecret; the ingest's two tokens live in the mgmt VM's envfile (documented, not committed). No secret manifest is added by this PR.
6. Shared, localized hunks (other slices' territory)
lab-director/dispatcher/cmd/dispatcher/app.go— the convener wiring + themeetingsarm registration (director/ and events/ neighbours).lab-director/dispatcher/config/config.go+config/load.go—TemplateKindRoundtable,arms.meetings, themeetingsblock,Config.MeetingsState().lab-director/dispatcher/cmd/dispatcher/context_plane.go— one rule inlaneForIssue.lab-director/dispatcher/cmd/dispatcher/pipeline.go—filesContextSourcegainsPackageForLane(the defect fix in §2).lab-director/dispatcher/director/fixture.go— aRefs()accessor.No new command, no new DB, no state outside Plane/Outline/files. Nothing in this PR
merges, hand-dispatches, or writes to a cluster.
First slice of LD4.2, per the orchestrator's assignment: prove or disprove the brief's riskiest assumption (§5.3 / R-5) before anything is built on it. Verdict: DISPROVED on both halves. * prod/manifests/agent-sandbox/ declares six SandboxTemplates (ld-sandbox, ld-build-test, ld-browser, ld-firmware, ld-meet-ai, ld-claude-code) and sandbox-init/roles.json declares two roles (light, builder-core). There is no roundtable class and no planner class. * The union of every class's egress allowlist is DNS, Forgejo, OmniRoute, the NATS bus and (light only) OTel. Outline appears nowhere in cilium-network-policies.yaml, and the dispatcher's own policy omits it too. The unratified Gateway-VIP egress hole (HOMELAB-1786) is explicitly NOT a path this slice may build on. Consequence: the documented fallback applies. The meeting-record Plane item is the record's durable home in v0; a roundtable class and its Outline egress are an operator decision (new SandboxTemplate + role + CNP, all on AGENTS.md's dangerous list), recorded in the PR body and the ticket's verification comment.The routing half of LD4.2. A meeting has no ticket, so the meeting-record Plane item IS the ticket: marker ld:meeting-record (+ label roundtable for humans), routed by laneForIssue to a new template kind. * intake: MarkerMeetingRecord + LabelRoundtable, and MarkerLinePayload so director/ parses a marker payload the same way intake/ delimits one. * config: TemplateKindRoundtable (the third item-kind key) so the loader accepts it; the shipped binding is ld-sandbox, because there is no roundtable SandboxTemplate (see docs/ld4.2-roundtable-class-verdict.md). * laneForIssue: marker ld:meeting-record -> roundtable, checked before the LD4.1 markers. The MARKER routes, not the label - the same posture ReadUndigested takes; requiring the label too would let a missing label silently demote a meeting to ordinary ticket work. * config: arms.meetings (Bool, unset=OFF) + a meetings block (interval/state/pageSize/maxPages). meetings.state defaults to the watch's first dispatchableStates entry, so the arm's view of "open" and the watch's view of "dispatchable" cannot drift; the loader requires it when the arm is on.The image half of S1. The image does not exist yet, so this is what makes it buildable and publishable through the normal pipeline: * lab-director/whisper-asr/Dockerfile — multi-stage whisper.cpp v1.9.4 build (GGML_NATIVE=OFF: the CI runner and prod-wk-01 are different CPUs), with ggml-large-v3-turbo-q5_0 baked in and its sha256 VERIFIED at build time (the hash was read off the mgmt VM's own copy, so a drift fails the build rather than silently baking a different file). CPU only; no ffmpeg (the caller sends 16 kHz mono WAV, which is the contract); curl is present so the smoke test and an operator's in-pod check need no second image. * .forgejo/workflows/whisper-asr-image-build.yaml — PR: build + smoke test (no credentials); live: publish <short-sha> + the 0.1.0/latest aliases and print the digest. The smoke test exercises the real contract on a generated WAV. * lab-director/whisper-asr/README.md — the contract, the consumers it copies (meeting-media.py, voice-whisper.json), and what is deliberately absent. Also fixes a real defect this slice exposed: the FILE context author ignored a resolved lane, so a roundtable dispatch under context.author=files (prod's setting) would compile class "default" while the spawn arm was handed "roundtable" — and spawn's preflight refuses a package whose class disagrees with its spec. filesContextSource now implements PackageForLane, with the regression test at cmd/dispatcher/roundtable_pipeline_test.go:TestTheFileAuthorHonoursTheRoundtableLane.LD4.2's producer: the operator-mediated half of the recording-ingest convention. Proton Meet has no server-side recording API (E2E MLS), so the pipeline's input is an exported MP4 an operator drops. This component runs on the mgmt VM as a systemd USER unit — the pi-signal-bridge precedent (no npm deps, env file, loopback tunnel) — because the orchestrator's ingest-v1 ruling is exactly that shape, and because no workload in this cluster mounts the media NFS (the brief's in-cluster watcher would have meant inventing a storage contract to satisfy an ingest). inbox/<recording> --ffmpeg 16kHz mono--> POST whisper-asr:8171/inference --> Outline transcript doc (Context Block + #t=HH:MM:SS cues) --> ONE meeting-record Plane item (marker ld:meeting-record, label roundtable, Todo) carrying the meeting.completed payload The last step is the trigger the dispatcher's meetings arm sweeps. Design points that are load-bearing rather than incidental: * The wire is the SAME as meeting-media.py's: hand-assembled multipart, field "file" + response_format/temperature/language, 16 kHz mono pcm_s16le. A test asserts the exact bytes. * The Plane payload is the Go marker grammar: marker line + ONE JSON line whose field names are director.MeetingRecord's. A Node-rendered item is committed to director/testdata/ and PARSED BY THE GO READER in a test, so a drift on either side breaks a test instead of a roundtable. * MEETING_ATTENDEES is REQUIRED: events.MeetingCompleted.Gaps() treats an empty attendee list as a gap, so an ingest without the series' seats would produce a meeting that never convenes, and that failure would look like a broken roundtable rather than missing configuration. Diarization is deferred, so the seats are stated once by the operator, not guessed from the audio. * Evidence is never deleted: sources are moved (inbox -> done/failed) by an atomic rename claim, and a failure leaves the file plus a .error.txt beside it. * One writer per item class: this component creates meeting-record items only. Direction items are the roundtable's. Also adds whisper-asr-tunnel.service (the loopback port-forward that reaches the in-cluster ClusterIP without a new network surface). Evidence: node --test -> 21 tests, 0 sockets opened; go test -race ./... and go vet ./... green.Landing review — verdict: CLEAN (merge)
Fresh-context landing worker (previous worker died in a session crash; no prior verdict existed). CI green on head
f3e6b9325b83a3abaeb372c4a6f0412be4a1d6ce(run #1696;dispatcher-testrunsgo vet ./... && go test -race ./...perci.yaml:81). Baseepic/LD4is still at1519dd2= this branch's merge-base, so the merge is clean with no conflict resolution. Review used primary sources only: the diff, the ticket's AC + rulings, the brief, and read-onlyprod/infra-coresources.Independently reproduced evidence
prod@epic/LD4: exactly six classSandboxTemplates (ld-sandbox,ld-build-test,ld-browser,ld-firmware,ld-meet-ai,ld-claude-code; the*-poolobjects are pools, not classes);agent-sandbox/pi-image/sandbox-init/roles.jsondeclares exactlylight+builder-core; the egress union is agent-sandbox-system:8888 (MCP), kube-system:53, forgejo:3000/:2222, omniroute:20128, lab-director:4222 (NATS), monitoring:4317/4318 —grep -c outline= 0, notoFQDNs/toEntities/world, and no Plane namespace.ReasonNoResultlane closure is pre-existing v1 (teardown/errors.go:88,pipeline.go:234) and is not bypassed here.filesContextSourcelane defect — real, correctly diagnosed, fixed, tested.events.ClassUnknown == ""→TemplateKeyFor("")→ the"default"key whileroundtableDispatchhands the spawn arm"roundtable";spawn.go:306refuses that pair.PackageForLaneon the file author closes it, andTestTheFileAuthorHonoursTheRoundtableLanedrives prod's actual author (context.author: files) and asserts the spawn happens with the roundtable template — it fails pre-fix.PlaneConvenercreates at most one Plane item (only when the finder finds none) and one comment;director.PlaneSinkis pinned at two methods byboundary_test.go. The diff contains no merge, force-push, delete,kubectlor cluster write.MEETING_ATTENDEESfail-closed.config.validate()adds it to the missing list and throws at startup (test/ingest.test.js:61), matchingMeetingCompleted.Gaps()("an unattributed roundtable is not attributable").renderMeetingRecordItemand diffed it againstdirector/testdata/meeting-record-from-node.html: byte-identical.go vetclean,go test -race ./...green across all 22 dispatcher packages;node --test21/21. Nolisten/createServerin the ingest (fetchinjected). The Go AC fixture stack composes the realPlaneConvener,roundtableDispatch,spawn.SpawnerandmeetingTriggerover transport fixtures: exactly ONE MCPCreatewith templateld-roundtable, second sweep convenes nothing, a gapped record opens one decision-queue item and spawns nothing.3942217…a7e2) matches this machine'sggml-large-v3-turbo-q5_0.binbyte-for-byte; the whisper.cpp pin is asserted between workflow and Dockerfile; push is scoped to non-PR events with theHARBOR_PUSH_*robot plus a size gate.app.gorefuses to start the arm without a live convener;config/load.gorequires a state when the arm is on;prod#876 writesarms.meetings: falsewith both reasons and the unlock order.Findings
HIGH (design, inherited from the brief — must be resolved before
arms.meetingsis turned on; not a blocker for this merge). One meeting-record item has two convening paths. The item is created inTodo, which by construction is both the arm's state (MeetingsState()derives fromwatch.dispatchableStates[0]) and the watch's dispatchable state; the watch predicate has no marker exclusion andlaneForIssueroutes the marker item to the roundtable lane, so the ordinary loop claims it and spawns. The trigger arm also convenes from the same item and, by its own documented contract, never transitions it out ofTodo, so it does not retire the item against the watch. The fixtures prove each path in isolation — they never compose the watch predicate with the arm — so "exactly one spawn per meeting" is proven for the arm, not for the deployment. Live impact today: none (arms.meetings: false, no prod-side ingest). The brief created this tension (item inTodo+ §5.3's lane rule) while §5.2 rejected "enqueue via the watch", so it is an adjudication, not an implementer error. Options: exclude marker items in the watch predicate, or have the arm claim/transition the item before convening.MEDIUM. The mailbox sweep is a full-project scan bounded at
pageSize × maxPages= 2000 items and fails closed beyond that; theHOMELABproject is already at ≥1859 items, so ~140 items of headroom remain before every sweep refuses. The identical pattern pre-exists in LD4.1'sintake.PlaneReader.scan, so it is not introduced here, but this arm turns it on a second time.MEDIUM.
core/services/pi-meeting-ingestis not wired into.forgejo/workflows/ci.yaml(onlypi-review-receiverandpi-alert-receiverare), so the 21 node tests are not CI-enforced — the same census gap as HOMELAB-1800. The PR claims only a local run, which is accurate; the gap is that drift would go uncaught.LOW. §2's table calls the whisper-asr image "smoke-tested in CI"; it has never run, because the workflow gates on PRs to
livewhile this PR targetsepic/LD4. §4 states this plainly, so it is a wording tension, not dishonesty — and the Application has nosyncPolicy, so nothing canImagePullBackOff.NIT. The
DefaultTemplatescomment says the generic lane "keeps the plane surface the roundtable's Plane-read needs". Thelightrole registers theplane_*tools but has no Plane egress, so the tool surface is present while the network path is not — which is the verdict doc's own conclusion.Honest gaps restated (unchanged)
S4 (roundtable class) and S7 (voice notes → HOMELAB-1858) are not built, correctly per the verdict and the ruling. The AC's "roundtable meeting record with citations" is not demonstrable end to end: the roundtable's product has no way back — no class reaches Plane or Outline, and no result-submission path exists — so the record-with-citations half is unlocked by the egress + result-return work tracked as HOMELAB-1859. The AC's Outline-transcript half needs an operator-created
Meetingscollection andOUTLINE_COLLECTION_ID. Trigger dedupe is in-memory per process (a restart between convening and the item leavingTodowould convene again). The whisper-asr image has never been built; its first live build is the real test.Proceeding to merge into
epic/LD4once the head's commit status reads green (an orphaned atlantis "Plan in progress…" status from PR creation is being cleared by a re-plan). Companionprod#876 is E-adj (new CiliumNetworkPolicy + prod dispatcher config) and remains for operator review — this worker did not touch it.atlantis plan
Landing worker — merge BLOCKED by an orphaned atlantis status (not a code blocker)
Verdict stands: CLEAN, no blockers in this PR. But
forgejo_merge_prrefuses, because the head commit's combined status ispendingand the landing lock reads the whole status set. The non-green context is not CI:CI Review / dispatcher-testgo vet+go test -race)CI Review / alert-receiver-testCI Review / helm-validateCI Review / receiver-testatlantis)0/0 projects planned successfully.Diagnosis
http://10.10.10.226/healthz→{"status":"ok"}, v0.40.0.Plan in progress...entry — i.e. two orphaned statuses today.pull_request/issue_commentwebhook is delivered to atlantis athttp://10.10.10.226/events(hook id 2), and atlantis does receive comments: myatlantis plancomment above produced a newPlan in progress...status at 06:54:25. That plan then also never completes (still pending ~3 minutes later; atlantis' own jobs list shows a plan running forHomelab/infra-core #581, not #577, and there is no atlantis comment on this PR)..tf, and autoplan would plan 0 projects anyway.Unblock options (either one clears it)
f3e6b93, verdict posted after it, no blockers). There is precedent for this on this repo when the automated path is blocked by infrastructure.http://10.10.10.226/(it currently shows a leftover "Plan discarded and unlocked!" banner and a stale lock onHomelab/prod #868), or restarting the atlantis pod, should let a fresh plan post the0/0 …successes. Worth an infra ticket: the same stall hit #574, so it is not specific to this PR.Landing worker note (process, worth knowing): posting comments to this PR through the
aaron/infra-coreslug silently reported success while writing nothing — the three earlier attempts (verdict + twoatlantis plancomments) never landed, which is also why the first re-plan never happened. Re-posting throughHomelab/infra-coreworked; every comment above was read back from the API before I relied on it.f3e6b9325b38493bbe5ePost-rebase integration note (after head
38493bbe5e)The CLEAN landing verdict above was posted at the pre-rebase head
f3e6b93. This rebase ontoepic/LD4(which had just gained #576/R-24) rewrote history: 4 conflict regions in 2 files (config/config.go— Meetings-vs-Boundary structs, template-kind consts, default bindings;cmd/dispatcher/app.go— runner registration), resolved as a pure union by an integration worker — both sides' keys and runners kept, no semantic change to any reviewed code.go build/go vet/go test -racegreen on the rebased tree locally; CI run #1765 green on38493bbe5e. The CLEAN verdict carries to this head; the integration delta is config wiring only.