Engineering report: groot v1.1.1

groot incident response workflow

groot is a read-only Go CLI for Kubernetes on-call: one .tar.gz of nodes, events, pod logs, and selected API snapshots. It collects evidence; it does not mutate the cluster and does not invent root cause.

v1.1.1 (2026-08-11) is a patch on the 1.1.x line (kubeconfig path expansion, concurrent-safe sessionBase, S3 credential trim). Functional highlights below anchored on v1.1.0 (groot analyze) unless noted. Latest pin: see the v1.2.0 report.

1. Why a diagnostics archive

During an incident, copy-pasting kubectl into Slack loses context: which node, which restart, which event window. Live tools disappear when the API is sick. An offline archive survives the window and travels with the ticket.

groot answers three operator questions:

  1. Can I collect? — groot validate (config, API, RBAC, disk).
  2. What did the cluster look like? — groot collect → one .tar.gz.
  3. What is in this bundle? — groot inspect (inventory) and groot analyze (offline heuristic hints).

2. Stable contract (config_version: 1)

New YAML sets config_version: 1 at the top. Legacy configs without it still load. Archive layout is versioned in the manifest (archive_layout_version) so offline tools can degrade safely.

Command Role
groot validate Preflight: config, API reachability, RBAC, free disk
groot collect Parallel client-go capture → .tar.gz (+ optional notify / upload)
groot inspect Offline inventory of archive members (no cluster)
groot analyze Offline heuristic hints from a collected archive (1.1.0+)
kubectl groot … Same binary as a kubectl plugin (kubectl-groot)

Full behavior: SPECIFICATIONS.md.

3. Incident workflow

curl -fsSL https://get.groot.hermesrodriguez.com/install.sh | sh
groot -v

groot validate --config groot.yml
groot collect --config groot.yml --summary
groot inspect ./groot-*.tar.gz
groot analyze ./groot-*.tar.gz --output executive

Optional uploads after collect: S3 (incl. Contabo-compatible), GCS, SFTP. Use --no-upload to keep the archive local. Operator scheduling (cron, Helm CronJob, bastion): groot-selfhosted.

4. Offline analyze (v1.1.0)

groot analyze reads an existing .tar.gz — no kubeconfig. It emits evidence-cited hints (CrashLoopBackOff, OOMKilled, ImagePullBackOff, NotReady, Evicted), not authoritative RCA.

Output Flag / mode Use
Executive Markdown default / --output executive Ticket summary
Structured JSON --output json Automation
LLM-ready pack --output llm Budgeted paste pack (still evidence-first)

Golden fixtures under testing/fixtures/archives/ lock executive and LLM Markdown in CI. Caps: default 16 GiB decompressed total; override with --max-decompressed.

5. Exit codes (wrappers)

Code Meaning
0 Success
1 Config / validation
2 Kubernetes API error
3 Collect aborted / archive open failure
4 Notify delivery failed

Same 0–4 wrapper-friendly taxonomy as other Hermes operator CLIs.

6. v1.1.1 fixes

  • kubeconfig expands ~ and ${VAR} (YAML, KUBECONFIG, --kubeconfig); SFTP path fields too.
  • Unique sessionBase short suffix so concurrent collects in the same second do not collide.
  • S3 credentials trim whitespace (avoids Contabo / S3-compatible SignatureDoesNotMatch).

7. Ecosystem

Repo Role
hrodrig/groot CLI, SPEC, releases, ghcr.io/hrodrig/groot
groot-selfhosted Helm CronJob, bastion, Docker
groot-trigger In-cluster HTTP → Job groot collect
groot-share (gfs) VPS catalog for .tar.gz archives
groot-share-selfhosted Deploy gfs

kzero mutates (maintenance windows). groot captures evidence before/after — complementary, not a substitute. Landing: groot.hermesrodriguez.com.