aaron/infra-core
1
0
Fork 2
Code Issues Pull requests 12 Projects Releases 1 Packages Wiki Activity Actions

HOMELAB-285: feat(monitoring): capacity alerting PrometheusRules #42

Merged
aaron merged 1 commit from plane/HOMELAB-285-capacity-alerts into live 2026-03-23 18:41:10 +00:00
Conversation 1 Commits 1 Files changed 1 +108
aaron commented 2026-03-23 18:40:51 +00:00
Owner
Copy link

Summary

  • 8 early-warning capacity alerts (9 rules total with headroom split)
  • NodeMemoryHigh (70%), NodeCPUHigh (80%), PodCountImbalance, request saturation (memory/CPU 80%), NodeMemoryOvercommit (200%), OOMKillRateHigh, ClusterMemoryHeadroom (4GiB/2GiB)
  • All PromQL expressions validated against live Prometheus

Acceptance Criteria

  • All 9 PrometheusRules deployed and visible in Prometheus UI
  • PromQL expressions return valid data
  • NodeMemoryHigh fires for prod-wk-02 (currently 72% > 70%)
  • No overlap/conflict with existing node-alerts.yaml rules

Ref: HOMELAB-285

## Summary - 8 early-warning capacity alerts (9 rules total with headroom split) - NodeMemoryHigh (70%), NodeCPUHigh (80%), PodCountImbalance, request saturation (memory/CPU 80%), NodeMemoryOvercommit (200%), OOMKillRateHigh, ClusterMemoryHeadroom (4GiB/2GiB) - All PromQL expressions validated against live Prometheus ## Acceptance Criteria - [ ] All 9 PrometheusRules deployed and visible in Prometheus UI - [ ] PromQL expressions return valid data - [ ] NodeMemoryHigh fires for prod-wk-02 (currently 72% > 70%) - [ ] No overlap/conflict with existing node-alerts.yaml rules Ref: HOMELAB-285
HOMELAB-285: feat(monitoring): add capacity alerting PrometheusRules
Some checks failed
CI Review / pr-title (pull_request) Successful in 1s
Details
CI Review / helm-validate (pull_request) Failing after 3s
Details
CI Review / ai-review (pull_request) Failing after 2s
Details
Lint & Validate / terraform-validate (pull_request) Failing after 1s
Details
Lint & Validate / yaml-lint (pull_request) Failing after 1s
Details
Lint & Validate / shellcheck (pull_request) Failing after 2s
Details
Plan failed.
0e23ca665f 
8 early-warning alerts for resource capacity: NodeMemoryHigh (70%),
NodeCPUHigh (80%), PodCountImbalance, request saturation (memory/CPU),
NodeMemoryOvercommit (200%), OOMKillRateHigh, ClusterMemoryHeadroom
(4GiB warning, 2GiB critical). All PromQL validated against live
Prometheus.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aaron merged commit 2264287cc1 into live 2026-03-23 18:41:10 +00:00
aaron deleted branch plane/HOMELAB-285-capacity-alerts 2026-03-23 18:41:11 +00:00
atlantis commented 2026-03-23 18:43:10 +00:00
Collaborator
Copy link

Plan Error

running git clone --depth=1 --branch plane/HOMELAB-285-capacity-alerts --single-branch https://atlantis:<redacted>@forgejo.aaron.reynoza.org/aaron/infra-core.git /atlantis-data/repos/aaron/infra-core/42/default: Cloning into '/atlantis-data/repos/aaron/infra-core/42/default'...
fatal: unable to access 'https://forgejo.aaron.reynoza.org/aaron/infra-core.git/': Failed to connect to forgejo.aaron.reynoza.org port 443 after 133612 ms: Could not connect to server
: exit status 128
**Plan Error** ``` running git clone --depth=1 --branch plane/HOMELAB-285-capacity-alerts --single-branch https://atlantis:<redacted>@forgejo.aaron.reynoza.org/aaron/infra-core.git /atlantis-data/repos/aaron/infra-core/42/default: Cloning into '/atlantis-data/repos/aaron/infra-core/42/default'... fatal: unable to access 'https://forgejo.aaron.reynoza.org/aaron/infra-core.git/': Failed to connect to forgejo.aaron.reynoza.org port 443 after 133612 ms: Could not connect to server : exit status 128 ```
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
aaron/infra-core!42
No description provided.