k8s-aiops

# k8s AIops (preview)

> **Disclaimer**: This is a community-maintained open-source project and is **not affiliated with, endorsed by, or sponsored by the Cloud Native Computing Foundation, the Kubernetes project, or k3s/Rancher.** "Kubernetes" and "k3s" are trademarks of their respective owners. Source code is publicly auditable at [github.com/AIops-tools/K8s-AIops](https://github.com/AIops-tools/K8s-AIops) under the MIT license.

Governed Kubernetes operations — **15 MCP tools**, every one wrapped with the bundled `@governed_tool` harness: a local unified audit log under `~/.k8s-aiops/`, policy engine, token/runaway budget guard, undo-token recording, and graduated-autonomy risk tiers. Works with any kubeconfig-reachable cluster (standard Kubernetes, k3s, EKS, GKE, AKS).

> **Standalone**: the governance harness is bundled in the package (`k8s_aiops.governance`) — k8s-aiops has no external skill-family dependency. Preview: common operations, not yet exhaustive.

## What This Skill Does

| Category | Tools | Count | Read or Write |
|----------|-------|:-----:|:-------------:|
| **Pods** | list, get, logs, delete | 4 | 3 read / 1 write |
| **Deployments** | list, get, scale, rollout restart, delete | 5 | 2 read / 3 write |
| **Services** | list | 1 | 1 read |
| **Nodes** | list, cordon, uncordon | 3 | 1 read / 2 write |
| **Namespaces** | list | 1 | 1 read |
| **Events** | list | 1 | 1 read |

## Quick Install

```bash
uv tool install k8s-aiops
k8s-aiops doctor          # uses your current kube-context out of the box
```

## When to Use This Skill

- List/inspect pods, deployments, services, nodes, namespaces and recent events
- Read a pod's recent log lines to diagnose a crash loop
- Scale a deployment up/down, or trigger a rolling restart
- Delete a stuck pod (a controller recreates it) or a deployment
- Cordon a node before maintenance, then uncordon it after

**Do NOT use when** the target is not a Kubernetes cluster (hypervisor VM lifecycle, backup products, or cloud-provider consoles are out of scope for this skill).

## Related Skills — Skill Routing

| If the user wants… | Use |
|--------------------|-----|
| Kubernetes pods / deployments / nodes | **k8s-aiops** (this skill) |
| Hypervisor VM lifecycle (power, snapshot, migrate) | a hypervisor ops skill |
| Backup & restore | a backup ops skill |

## Common Workflows

### Diagnose a crash-looping pod and restart its deployment

1. `k8s-aiops pod list -n prod` → find the pod with high `restarts` / non-Running `phase`
2. `k8s-aiops pod logs <pod> -n prod --tail 200` → read the recent logs for the crash cause
3. `k8s-aiops events -n prod` → check for `FailedScheduling` / image-pull events
4. `k8s-aiops deployment restart <deploy> -n prod` → roll the deployment after fixing the cause
5. **Failure branch**: if logs/events show an RBAC `403`, the kube context lacks the verb — run `kubectl auth can-i get pods -n prod` and switch to a context with adequate RBAC; the skill never retries a denied auth.

### Drain a node for maintenance, safely reversible

1. `k8s-aiops node list` → identify the node and confirm it is `Ready`/schedulable
2. `k8s-aiops node cordon <node> --dry-run` → preview, then `k8s-aiops node cordon <node>` (double confirm) — records an inverse `uncordon_node` undo descriptor
3. After maintenance: `k8s-aiops node uncordon <node>` → re-enable scheduling
4. **Failure branch**: if `doctor` shows the cluster unreachable, fix the kubeconfig context (`kubectl config get-contexts`) before retrying — cordon is never issued against an unauthenticated session.

## Usage Mode

| Scenario | Recommended | Why |
|----------|:-----------:|-----|
| Local/small models (Ollama, Qwen) | **CLI** | fewer tokens than MCP |
| Cloud models (Claude, GPT) | Either | MCP gives structured JSON I/O |
| Automated pipelines | **MCP** | type-safe parameters, audited |

## MCP Tools (15 — 9 read, 6 write)

| Category | Tools | R/W |
|----------|-------|:---:|
| Pods | `pod_list`, `pod_get`, `pod_logs` | Read |
| | `delete_pod` | Write |
| Deployments | `deployment_list`, `deployment_get` | Read |
| | `scale_deployment`, `rollout_restart_deployment`, `delete_deployment` | Write |
| Services | `service_list` | Read |
| Nodes | `node_list` | Read |
| | `cordon_node`, `uncordon_node` | Write |
| Namespaces | `namespace_list` | Read |
| Events | `event_list` | Read |

**Harness features that light up**: write tools with a clean inverse pass an `undo=` lambda so the harness records an inverse descriptor (with `_undo_id`) to the undo store — `scale_deployment` records a scale-back to its returned `previous_replicas`, and `cordon_node` ↔ `uncordon_node` are mutual inverses. `delete_pod`, `delete_deployment`, and `rollout_restart_deployment` declare no undo; `delete_deployment` is tagged `risk_level=high`. All 15 tools are audit-logged under `~/.k8s-aiops/` and pass through the policy pre-check + budget/runaway guard + graduated risk-tier gate. Avoid tight poll loops (re-listing pods every second) — the runaway breaker backs this up.

## CLI Quick Reference

```bash
k8s-aiops pod list [-n <ns>] [-t <target>]
k8s-aiops pod get <name> [-n <ns>]
k8s-aiops pod logs <name> [-n <ns>] [--tail 200] [-c <container>]
k8s-aiops pod delete <name> [-n <ns>] [--dry-run]        # double confirm
k8s-aiops deployment list [-n <ns>]
k8s-aiops deployment get <name> [-n <ns>]
k8s-aiops deployment scale <name> <replicas> [-n <ns>]
k8s-aiops deployment restart <name> [-n <ns>]
k8s-aiops deployment delete <name> [-n <ns>] [--dry-run]  # double confirm
k8s-aiops service list [-n <ns>]
k8s-aiops node list
k8s-aiops node cordon <name> [--dry-run]                  # double confirm
k8s-aiops node uncordon <name>
k8s-aiops namespace list
k8s-aiops events [-n <ns>]
k8s-aiops doctor
k8s-aiops mcp                                             # start MCP server (stdio)
```

See `references/cli-reference.md` for the full command list.

## Troubleshooting

### "Could not load kubeconfig … context