aaron/lab-director
1
0
Fork 0
Code Issues Pull requests 5 Projects Releases Packages Wiki Activity Actions

HOMELAB-1938: docs(ld5): B10 spike register — Plane actor probe result, S1/S15 prep, S12 dependency #13

Open
aaron wants to merge 0 commits from plane/HOMELAB-1938-spikes into main
pull from: plane/HOMELAB-1938-spikes
merge into: aaron:main
Conversation 2 Commits - Files changed -
aaron commented 2026-10-22 05:45:48 +00:00
Owner
Copy link

Adds design/ld5/ for LD5-B10's four spikes. Docs only, no code.

spikes.md — the register. Only one spike was answerable today.

Plane actor spike: ANSWERED (yes). The API does return who moved a ticket and
from/to which state, via an endpoint the public docs do not list:
GET .../projects/{pid}/work-items/{wid}/activities/ → per-transition entries with
verb, field, old_value, new_value, actor (member UUID, resolved against
.../projects/{pid}/members/). 5/5 transitions recorded for a scratch ticket
(HOMELAB-2204, created → Todo → In Progress → Blocked → Cancelled) with sub-second
freshness. work-items/{id}/ exposes only created_by/updated_by; comments and
plane_get_item expose no actor at all.

The 429s turned out not to be our own rate: 25-parallel and 50-sequential bursts
both returned 100% 200, while one early 20-request burst got 5×429 and the pi tools
got 2 at session start. The limiter is a token bucket keyed on the API key, and
every workstation session shares one key (claude-agent), so the 429 rate is a
function of fleet-wide concurrency on one identity. x-ratelimit-remaining (fresh
window: 40) is present on 200s but absent on 429s, which stall ~1 s rather than
fast-failing. Two consequences recorded: self-throttle on the header, and treat actor
attribution as an identity problem — B2's "signer account with no API token" ruling is
load-bearing, and the record DB (B5), not Plane, is the proof.

spike-s1-token.md — PREP ONLY. Rotation is the decisive leg: the operator ruling
on HOMELAB-2007 makes the quarterly cadence conditional on S1 proving old tokens die,
so the procedure polls the old token on a schedule and reports a kill window, not
"instant". Plus the environment-exposure checklist (/proc/*/environ by name, never
values). Blocked on B2 + the operator, because it rotates their token.

spike-s15-cost.md — PREP ONLY. Records why the requested model is not the
answer (combos resolve at request time; served connection sets the price). Verified
today that pi_agent_tokens_total exists with llm_model/sandbox_env/subagent_outcome
labels, but OmniRoute has no Prometheus job and /metrics is 401 — so served-model
and per-connection spend are not collected anywhere yet. Blocked on B2 + B9.

S12 — checklist entry only; the workflow validator it is scored against is built on
the E3 knob schema, so scoring it earlier measures a moving target.

Origin: HOMELAB-1938 / LD5-B10, epic HOMELAB-2194.

Adds `design/ld5/` for LD5-B10's four spikes. Docs only, no code. **`spikes.md`** — the register. Only one spike was answerable today. **Plane actor spike: ANSWERED (yes).** The API does return who moved a ticket and from/to which state, via an endpoint the public docs do not list: `GET .../projects/{pid}/work-items/{wid}/activities/` → per-transition entries with `verb`, `field`, `old_value`, `new_value`, `actor` (member UUID, resolved against `.../projects/{pid}/members/`). 5/5 transitions recorded for a scratch ticket (HOMELAB-2204, created → Todo → In Progress → Blocked → Cancelled) with sub-second freshness. `work-items/{id}/` exposes only `created_by`/`updated_by`; comments and `plane_get_item` expose no actor at all. The 429s turned out **not** to be our own rate: 25-parallel and 50-sequential bursts both returned 100% 200, while one early 20-request burst got 5×429 and the pi tools got 2 at session start. The limiter is a token bucket keyed on the **API key**, and every workstation session shares one key (`claude-agent`), so the 429 rate is a function of fleet-wide concurrency on one identity. `x-ratelimit-remaining` (fresh window: 40) is present on 200s but **absent on 429s**, which stall ~1 s rather than fast-failing. Two consequences recorded: self-throttle on the header, and treat actor attribution as an identity problem — B2's "signer account with no API token" ruling is load-bearing, and the record DB (B5), not Plane, is the proof. **`spike-s1-token.md`** — PREP ONLY. Rotation is the decisive leg: the operator ruling on HOMELAB-2007 makes the *quarterly* cadence conditional on S1 proving old tokens die, so the procedure polls the old token on a schedule and reports a **kill window**, not "instant". Plus the environment-exposure checklist (`/proc/*/environ` by name, never values). Blocked on B2 + the operator, because it rotates their token. **`spike-s15-cost.md`** — PREP ONLY. Records why the *requested* model is not the answer (combos resolve at request time; served connection sets the price). Verified today that `pi_agent_tokens_total` exists with `llm_model`/`sandbox_env`/`subagent_outcome` labels, but OmniRoute has **no Prometheus job and `/metrics` is 401** — so served-model and per-connection spend are not collected anywhere yet. Blocked on B2 + B9. **S12** — checklist entry only; the workflow validator it is scored against is built on the E3 knob schema, so scoring it earlier measures a moving target. Origin: HOMELAB-1938 / LD5-B10, epic HOMELAB-2194.
aaron commented 2026-10-22 05:53:01 +00:00
Author
Owner
Copy link

Landing review — APPROVE (no blockers)

Reviewer context: fresh, not the implementer. Sources: this diff only, HOMELAB-1938 (description + findings comment), the HOMELAB-2007 ruling comment, design/README.md + design/where-we-are/next.html on main, plus two live re-probes of the PR's own claims.

Verdict per dimension

  1. S1 answers the 2007 ruling — yes. spike-s1-token.md:70-105 polls the old token on a fixed ladder (t+0 … 24 h, then daily to 7 d), stops at the first hard rejection, and reports a kill window as an interval (last-good / first-bad), explicitly forbidding "instant" unless two adjacent probes prove it. The result template carries kill window: and cadence ruling input: fields. Distinguishing a revocation from a usage-limit refusal is called out as a failure mode (:100-104). That is exactly the measurement the ruling conditions the quarterly cadence on.
  2. S15 records $ per node + served model, with no invented sources — yes. The doc is correctly built on the fact that combos resolve at request time, so price comes from the serving connection, not the requested id (spike-s15-cost.md:17-30). I re-verified its two factual claims myself: pi_agent_tokens_total exists in Prometheus with llm_model / sandbox_env / plane_ticket_id / subagent_type / subagent_outcome (sandbox_env="unbound" today, as written), and OmniRoute has no Prometheus job (count by (job) (up{job=~".*omni.*"}) → 0 series) with /metrics → 401 (confirmed by unauthenticated curl). The "not collected anywhere yet" prerequisite is stated as a prerequisite, not papered over, and the doc forbids reporting $0.00 for invisible quota (:85-87).
  3. S12 deferred to the E3 knob schema — correct. spikes.md:124-134 records only the checklist and the consequence ("if it fails, the planner is switched off"). next.html on main corroborates the dependency verbatim: "the knob schema is frozen. If kimi fails it, the planner is switched off."
  4. Claim honesty — clean. S1 and S15 are PREP ONLY in frontmatter, table, and body; nothing is claimed as run. The rotation risk to the operator's own token is stated three times and unambiguously (spike-s1-token.md:27-41 "kills the operator's working token … cannot be run opportunistically or by an autonomous worker"; spikes.md:105-109; plus the scratch-slot requirement). The only executed spike (Plane actor) is corroborated: I re-ran GET …/work-items/580f998f-…/activities/ on scratch ticket HOMELAB-2204 and got 5 entries (1 created + Backlog→Todo→In Progress→Blocked→Cancelled), actor 401b58dc… on all five, and the quoted example entry (In Progress → Blocked, 05:31:20.072118Z) matches byte-for-byte. The B2 claim in spikes.md:96-99 matches next.html: "signer account: browser only, with MFA and no API token."
  5. Scope — exactly three new files (design/ld5/spikes.md, spike-s1-token.md, spike-s15-cost.md), 387 lines, no other paths, and design/ld5/ is a new subdirectory of an existing docs tree.
  6. Probe findings are recorded well enough for B2/B5 to rely on them. The activities endpoint with a real payload, the 404 list, the /work-items/{IDENT}/ PATCH-403 trap, and — the part that actually changes the build — the 429 analysis: token bucket keyed on the shared API key (x-ratelimit-remaining: 40, absent on 429, ~1 s stall rather than fast-fail), self-throttle with reserve ≥ 10, and "attribution is an identity problem, not a logging problem" with the record DB (B5) as proof and Plane as mirror (spikes.md:58-100). Two consequences land correctly for B2 and B5.

Concerns (non-blocking — fix in a follow-up, do not hold this PR)

  • spikes.md:52 — "5 of 5 state transitions" is one off: the feed holds 5 activity entries, of which one is verb: created and four are state transitions (verified above). Say "5 of 5 activity entries (1 creation + 4 transitions)" so B2 does not later treat created as a transition with an actor/signer.
  • spikes.md:108 — "step 4 of the procedure" rotates the token; in spike-s1-token.md the rotation is Leg 2 step 3 (Mint B); step 4 is "prove B works". One-word fix, but it is the leg whose sequencing matters.
  • spike-s1-token.md:79 — the middle outcome row ("keeps working for hours/days → … quarterly gains little → yearly + on exposure stands") pre-judges the call. On a strict reading of 2007, a multi-hour kill window does prove old tokens die, so the cadence is the operator's to rule. Suggest keeping the row to the measurement and its implication ("rotation is a delayed, not immediate, revocation control"), leaving the ruling to the cadence ruling input: field the template already has.
  • spike-s15-cost.md:47,82 — "$ per node … priced at the served connection's rate" never names where the rate (or a gateway-computed cost) comes from. Add one row: gateway-reported cost if the usage endpoint exposes it, otherwise a rate card committed with the findings (with its date). Without that, step 5 is not reproducible by a second worker.
  • Optional: design/README.md on main indexes the design tree by folder and does not mention ld5/. A one-line addition would keep the new folder discoverable — deliberately out of this PR's stated scope, so follow-up.

Merging per landing convention (docs-only, non-dangerous, no open blockers).

## Landing review — APPROVE (no blockers) Reviewer context: fresh, not the implementer. Sources: this diff only, HOMELAB-1938 (description + findings comment), the HOMELAB-2007 ruling comment, `design/README.md` + `design/where-we-are/next.html` on `main`, plus two live re-probes of the PR's own claims. ### Verdict per dimension 1. **S1 answers the 2007 ruling — yes.** `spike-s1-token.md:70-105` polls the *old* token on a fixed ladder (t+0 … 24 h, then daily to 7 d), stops at the first hard rejection, and reports a **kill window as an interval** (last-good / first-bad), explicitly forbidding "instant" unless two adjacent probes prove it. The result template carries `kill window:` and `cadence ruling input:` fields. Distinguishing a revocation from a usage-limit refusal is called out as a failure mode (`:100-104`). That is exactly the measurement the ruling conditions the quarterly cadence on. 2. **S15 records $ per node + served model, with no invented sources — yes.** The doc is correctly built on the fact that combos resolve at request time, so price comes from the *serving connection*, not the requested id (`spike-s15-cost.md:17-30`). I re-verified its two factual claims myself: `pi_agent_tokens_total` exists in Prometheus with `llm_model` / `sandbox_env` / `plane_ticket_id` / `subagent_type` / `subagent_outcome` (`sandbox_env="unbound"` today, as written), and OmniRoute has **no Prometheus job** (`count by (job) (up{job=~".*omni.*"})` → 0 series) with `/metrics` → **401** (confirmed by unauthenticated curl). The "not collected anywhere yet" prerequisite is stated as a prerequisite, not papered over, and the doc forbids reporting $0.00 for invisible quota (`:85-87`). 3. **S12 deferred to the E3 knob schema — correct.** `spikes.md:124-134` records only the checklist and the consequence ("if it fails, the planner is switched off"). `next.html` on `main` corroborates the dependency verbatim: *"the knob schema is frozen. If kimi fails it, the planner is switched off."* 4. **Claim honesty — clean.** S1 and S15 are `PREP ONLY` in frontmatter, table, and body; nothing is claimed as run. The rotation risk to the operator's own token is stated three times and unambiguously (`spike-s1-token.md:27-41` "**kills the operator's working token** … cannot be run opportunistically or by an autonomous worker"; `spikes.md:105-109`; plus the scratch-slot requirement). The only executed spike (Plane actor) is corroborated: I re-ran `GET …/work-items/580f998f-…/activities/` on scratch ticket HOMELAB-2204 and got 5 entries (1 `created` + `Backlog→Todo→In Progress→Blocked→Cancelled`), actor `401b58dc…` on all five, and the quoted example entry (`In Progress → Blocked`, `05:31:20.072118Z`) matches byte-for-byte. The B2 claim in `spikes.md:96-99` matches `next.html`: *"signer account: browser only, with MFA and no API token."* 5. **Scope — exactly three new files** (`design/ld5/spikes.md`, `spike-s1-token.md`, `spike-s15-cost.md`), 387 lines, no other paths, and `design/ld5/` is a new subdirectory of an existing docs tree. 6. **Probe findings are recorded well enough for B2/B5 to rely on them.** The activities endpoint with a real payload, the 404 list, the `/work-items/{IDENT}/` PATCH-403 trap, and — the part that actually changes the build — the 429 analysis: token bucket **keyed on the shared API key** (`x-ratelimit-remaining: 40`, absent on 429, ~1 s stall rather than fast-fail), self-throttle with reserve ≥ 10, and "attribution is an identity problem, not a logging problem" with the record DB (B5) as proof and Plane as mirror (`spikes.md:58-100`). Two consequences land correctly for B2 and B5. ### Concerns (non-blocking — fix in a follow-up, do not hold this PR) - `spikes.md:52` — "**5 of 5 state transitions**" is one off: the feed holds 5 *activity entries*, of which one is `verb: created` and four are state transitions (verified above). Say "5 of 5 activity entries (1 creation + 4 transitions)" so B2 does not later treat `created` as a transition with an actor/signer. - `spikes.md:108` — "step 4 of the procedure" rotates the token; in `spike-s1-token.md` the rotation is Leg 2 **step 3** (Mint B); step 4 is "prove B works". One-word fix, but it is the leg whose sequencing matters. - `spike-s1-token.md:79` — the middle outcome row ("keeps working for hours/days → … quarterly gains little → yearly + on exposure stands") pre-judges the call. On a strict reading of 2007, a multi-hour kill window *does* prove old tokens die, so the cadence is the operator's to rule. Suggest keeping the row to the measurement and its implication ("rotation is a delayed, not immediate, revocation control"), leaving the ruling to the `cadence ruling input:` field the template already has. - `spike-s15-cost.md:47,82` — "$ per node … priced at the served connection's rate" never names **where the rate (or a gateway-computed cost) comes from**. Add one row: gateway-reported cost if the usage endpoint exposes it, otherwise a rate card committed with the findings (with its date). Without that, step 5 is not reproducible by a second worker. - Optional: `design/README.md` on `main` indexes the design tree by folder and does not mention `ld5/`. A one-line addition would keep the new folder discoverable — deliberately out of this PR's stated scope, so follow-up. Merging per landing convention (docs-only, non-dangerous, no open blockers).
aaron commented 2026-10-22 05:53:37 +00:00
Author
Owner
Copy link

Merge outcome — blocked by the landing lock (needs a UI merge)

I attempted the merge and it was refused:

Refusing to merge aaron/lab-director#13 (head fa138b4): no CI status is recorded for the
head commit (commit-status source: combined).

Not a defect in this PR. .forgejo/workflows/ci.yaml triggers on pull_request: branches: [live], and this PR targets main, so no run is ever recorded and the lock fails closed on a docs-only change. There is no override parameter by design (HOMELAB-1679) — a human merges from the Forgejo UI.

Review verdict stands: APPROVE, no blockers, concerns in the comment above are follow-ups. Nothing in the diff needs to change before merge.

## Merge outcome — blocked by the landing lock (needs a UI merge) I attempted the merge and it was refused: ``` Refusing to merge aaron/lab-director#13 (head fa138b4): no CI status is recorded for the head commit (commit-status source: combined). ``` Not a defect in this PR. `.forgejo/workflows/ci.yaml` triggers on `pull_request: branches: [live]`, and this PR targets `main`, so no run is ever recorded and the lock fails closed on a docs-only change. There is no override parameter by design (HOMELAB-1679) — a human merges from the Forgejo UI. Review verdict stands: **APPROVE**, no blockers, concerns in the comment above are follow-ups. Nothing in the diff needs to change before merge.
This branch is already included in the target branch. There is nothing to merge.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin plane/HOMELAB-1938-spikes:plane/HOMELAB-1938-spikes
git switch plane/HOMELAB-1938-spikes

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff plane/HOMELAB-1938-spikes
git switch plane/HOMELAB-1938-spikes
git rebase main
git switch main
git merge --ff-only plane/HOMELAB-1938-spikes
git switch plane/HOMELAB-1938-spikes
git rebase main
git switch main
git merge --no-ff plane/HOMELAB-1938-spikes
git switch main
git merge --squash plane/HOMELAB-1938-spikes
git switch main
git merge --ff-only plane/HOMELAB-1938-spikes
git switch main
git merge plane/HOMELAB-1938-spikes
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
aaron/lab-director!13
No description provided.