HOMELAB-1120: Fix pi-otel-telemetry bugs and add custom metrics #1

Merged

aaron merged 2 commits from plane/HOMELAB-1120-otel-telemetry into main

2026-06-13 16:27:06 +00:00

aaron commented

2026-06-13 16:19:58 +00:00

Owner

What:

Fix 3 bugs from prior session handoff (2026-06-12)
Add custom agent-level metrics using OpenTelemetry Metrics API
Replace broken Paperclip trace-context API call with env-var fallback

Fixes:

Add missing ensureTunnels import (Fix 1)
Change Tempo tunnel port from 3100 to 3201 (Fix 2)
Add llm.model/llm.provider fallback from ctx.model (Fix 3)
Replace broken fetchPaperclipTraceContext API call with env-var fallback (Fix 4)
Add paperclip.company_id and paperclip.agent_name attributes

Features:

Add @opentelemetry/sdk-metrics and @opentelemetry/exporter-metrics-otlp-http deps
Reuse OpenLit's MeterProvider via metrics.getMeterProvider()
Add custom counters: pi_agent_sessions_total, pi_agent_tool_calls_total, pi_agent_tool_errors_total
Add custom histograms: pi_agent_session_duration_seconds, pi_agent_tool_duration_seconds
Instrument session_start, tool_execution_start, tool_execution_end, session_shutdown handlers
All metrics include llm.model, llm.provider, and tool.name labels

Verify:

query_telemetry returns traces without Tunnel error
Traces show llm.model, llm.provider, paperclip.company_id, paperclip.agent_name
Prometheus queries return pi_agent_* metrics
No port conflicts (Tempo on 3201, Paperclip on 3100)

**What:** - Fix 3 bugs from prior session handoff (2026-06-12) - Add custom agent-level metrics using OpenTelemetry Metrics API - Replace broken Paperclip trace-context API call with env-var fallback **Fixes:** - Add missing `ensureTunnels` import (Fix 1) - Change Tempo tunnel port from 3100 to 3201 (Fix 2) - Add `llm.model`/`llm.provider` fallback from `ctx.model` (Fix 3) - Replace broken `fetchPaperclipTraceContext` API call with env-var fallback (Fix 4) - Add `paperclip.company_id` and `paperclip.agent_name` attributes **Features:** - Add `@opentelemetry/sdk-metrics` and `@opentelemetry/exporter-metrics-otlp-http` deps - Reuse OpenLit's MeterProvider via `metrics.getMeterProvider()` - Add custom counters: `pi_agent_sessions_total`, `pi_agent_tool_calls_total`, `pi_agent_tool_errors_total` - Add custom histograms: `pi_agent_session_duration_seconds`, `pi_agent_tool_duration_seconds` - Instrument `session_start`, `tool_execution_start`, `tool_execution_end`, `session_shutdown` handlers - All metrics include `llm.model`, `llm.provider`, and `tool.name` labels **Verify:** - [ ] `query_telemetry` returns traces without `Tunnel error` - [ ] Traces show `llm.model`, `llm.provider`, `paperclip.company_id`, `paperclip.agent_name` - [ ] Prometheus queries return `pi_agent_*` metrics - [ ] No port conflicts (Tempo on 3201, Paperclip on 3100)

aaron added 2 commits

2026-06-13 16:19:58 +00:00

feat(otel): add telemetry read path with query_telemetry tool fcaf477772

Add traceId persistence to session JSONL, session-scoped kubectl
port-forward tunnels for Tempo/Prometheus (tunnel.ts), and an
LLM-callable query_telemetry tool supporting direct trace lookup,
TraceQL search, and PromQL metrics queries with adaptive truncation.

HOMELAB-1120: Fix pi-otel-telemetry bugs and add custom metrics fc7b6a32a1

Fixes:
- Add missing ensureTunnels import (Fix 1 from handoff)
- Change Tempo tunnel port from 3100 to 3201 to avoid Paperclip conflict (Fix 2)
- Add llm.model/llm.provider fallback from ctx.model (Fix 3)
- Replace broken fetchPaperclipTraceContext API call with env-var fallback (Fix 4)
- Add paperclip.company_id and paperclip.agent_name attributes from env vars

Features:
- Add @opentelemetry/sdk-metrics and @opentelemetry/exporter-metrics-otlp-http deps
- Reuse OpenLit's MeterProvider via metrics.getMeterProvider()
- Add custom metrics: pi_agent_sessions_total, pi_agent_tool_calls_total,
  pi_agent_tool_errors_total, pi_agent_session_duration_seconds,
  pi_agent_tool_duration_seconds
- Instrument session_start, tool_execution_start, tool_execution_end,
  and session_shutdown event handlers
- All metrics include llm.model, llm.provider, and tool.name labels