aaron/pig-extensions
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Native Go extensions for PiG (LD6.5 Phase 0+: telemetry first). Goal: drop the Node sidecar for memory-bound fleet lanes.
5 commits 1 branch 0 tags 134 KiB
  • Go 98%
  • Shell 1.1%
  • Makefile 0.9%
Find a file
Exact
Aaron Reynoza c06bd2eefc HOMELAB-1767: docs, fused Piglet manifest, and a pig-build-safe go.mod 
* docs/span-contract.md updated with the two changes Phase 0 made (tool_result
  folded into tool_execution_end; the cell heap policy) and with the OTLP/HTTP
  JSON-vs-protobuf transport difference found by capturing raw request bodies.
* docs/fused-piglet-binary.md records the working fused build path, the measured
  3.96 MiB / one-process delta, and the three constraints that are not obvious
  from the PiG docs (PIG_SOURCE_ROOT, no local SDK replace in go.mod, origins must
  resolve inside the Piglet anchor).
* pig-otel-go.yaml + make piglet-validate/piglet-binary.
* go.mod no longer carries a local SDK replace: pig piglet build resolves local
  replacements at lock time, so the dev-only path broke the fused build (and a
  symlink is rejected outright). scripts/dev-sdk.sh now stages the SDK through a
  git-ignored go.work, which local go build/vet/test use and the fused build
  ignores.
* README carries the Phase 0 numbers.
2026-09-30 18:00:45 +00:00
docs HOMELAB-1767: docs, fused Piglet manifest, and a pig-build-safe go.mod 2026-09-30 18:00:45 +00:00
pi-otel-telemetry HOMELAB-1767: docs, fused Piglet manifest, and a pig-build-safe go.mod 2026-09-30 18:00:45 +00:00
scripts HOMELAB-1767: docs, fused Piglet manifest, and a pig-build-safe go.mod 2026-09-30 18:00:45 +00:00
.gitignore HOMELAB-1767: scaffold pig-extensions repo + frozen span/metric contract 2026-09-30 15:58:39 +00:00
Makefile HOMELAB-1767: docs, fused Piglet manifest, and a pig-build-safe go.mod 2026-09-30 18:00:45 +00:00
pig-otel-go.yaml HOMELAB-1767: docs, fused Piglet manifest, and a pig-build-safe go.mod 2026-09-30 18:00:45 +00:00
README.md HOMELAB-1767: docs, fused Piglet manifest, and a pig-build-safe go.mod 2026-09-30 18:00:45 +00:00

README.md

pig-extensions

Native Go extensions for PiG (Pi, rebuilt in Go), built by the homelab fleet. Each extension is a self-contained Go module against github.com/MichaelKinsy/PiG/extensions/sdk.

Extension Purpose Status
pi-otel-telemetry/ OpenTelemetry trace + metric pipeline for PiG/Pi agent sessions (Go re-authoring of the Node/TS pi-otel-telemetry). Spans to Tempo, metrics to Prometheus, plus the query_telemetry tool. Phase 0 prototype (HOMELAB-1767)

Why Go

Measured on the mgmt VM (HOMELAB-1699 / E3, E5, and Phase 0 of HOMELAB-1767), idle, whole-process-tree RSS, medians of 3 runs:

Cell Tree RSS Processes Startup→ready
pig bare 36.6 MiB 1 65 ms
pig + this Go extension (cell) 75.3 MiB 2 246 ms
pig + stock TS extension (Node sidecar) 224.3 MiB 2 1 261 ms
pi + stock TS extension (in-process) 187.0 MiB 1 1 121 ms
Fused Piglet Binary (this extension) 72.5 MiB 1 222 ms

The Node sidecar is the whole point: a lane that moves to PiG keeps the Node tax unless its extensions move to Go too. Against the pi in-process baseline this extension is −60 % per session; against the TS extension under PiG, −66 %.

node is never invoked — verified by running with a failing node first on PATH.

Layout

docs/                 cross-repo documentation (span contract, fused build)
pi-otel-telemetry/    the extension (Go module)
pig-otel-go.yaml      fused Piglet Binary manifest
scripts/dev-sdk.sh    stage the PiG Go SDK for local go build/vet/test

Local development

PiG builds extensions by generating a private go.mod that replaces the SDK module with the SDK staged by the installed pig binary. A bare go build in an extension directory therefore cannot resolve github.com/MichaelKinsy/PiG/extensions/sdk.

scripts/dev-sdk.sh writes a git-ignored go.work pointing at the staged SDK so go build, go vet, gofmt -l . and go test ./... all work:

make dev-sdk          # writes ./go.work (git-ignored)
make check            # gofmt + vet + test
make race             # tests under -race

The extension's committed go.mod deliberately carries no replace directive — pig piglet build resolves local replacements while locking build inputs, so a dev-only path would break the fused build on every machine that lacks it.

Loading the extension

pig -e "$PWD/pi-otel-telemetry" --no-extensions
pig install "$PWD/pi-otel-telemetry" --validate-only --json   # validate a factory

Configuration precedence is settings file → environment → default, with the same keys as the TypeScript extension (see docs/span-contract.md §2). Settings live in ~/.pig/pi-otel-telemetry.json (override the path with PI_OTEL_SETTINGS); the Go extension cannot register a settings panel because the Go SDK has no settings-node API.

export OTEL_EXPORTER_OTLP_ENDPOINT=http://10.10.10.230:4318
export OTEL_EXPORTER_OTLP_HEADERS='Authorization=Basic …'

Building the fused Piglet Binary

make piglet-validate            # schema + origin resolution
PIG_SOURCE_ROOT=~/src/PiG make piglet-binary

Feasible in stock PiG v0.3.0. It buys ~4 MiB and one process over the subprocess cell — see docs/fused-piglet-binary.md for the measured delta and the three build constraints that are not obvious from the PiG docs.

Phase 0 results (HOMELAB-1767)

Criterion Result
Span/metric attribute parity vs the TS extension PARITY — byte-identical attribute names and OTLP value kinds, verified in Tempo and by raw payload diff
30-turn long-session plateau 134.1 MiB (last-10 mean; slope −0.29 MiB/turn, R²=0.016 → flat). Above the pre-registered ~120 MiB bar by 14 MiB.
Flush-balloon bug class (E5 caveat 5) eliminated — bounded queue with drop-oldest; the TS path under PiG climbs +7.96 MiB/turn, this one plateaus
Fused vs subprocess Δ 3.96 MiB → fusion not worth the build weight; ship the subprocess cell
Per-worker one-shot saving −66 % vs the TS extension under PiG, −60 % vs pi in-process

Full report: ~/repos/homelab/.rpiv/artifacts/research/homelab-1699/go-stack/phase0/phase0-report.md.