analyze-cluster-postmortem

**PURPOSE: Find prompt weaknesses and FIX THEM so agents work out of the box for EVERY use case.**

This is NOT about debugging infrastructure. This is about making prompts BULLETPROOF.

## The Goal

Every cluster failure is a **prompt improvement opportunity**. Analyze what went wrong → Fix the prompt → Never see this failure pattern again.

## Input

`$ARGUMENTS` can be:

- **Cluster ID**: Analyze specific cluster
- **"recent"**: Find most recent clusters
- **"dump"**: User will paste logs directly

## Step 1: Get the Data

```bash
# List recent clusters
zeroshot list --json | jq '.[-5:]'

# Get cluster status and logs
zeroshot status $CLUSTER_ID --json
zeroshot logs $CLUSTER_ID 2>&1

# Query ledger for agent messages
sqlite3 ~/.zeroshot/clusters/$CLUSTER_ID/ledger.db "
  SELECT timestamp, topic, sender, content_text
  FROM messages
  ORDER BY timestamp ASC;
"
```

## Step 2: Identify What Agent Did Wrong

**READ THE LOGS. What SPECIFICALLY went wrong?**

| Failure Pattern         | Evidence                             | Prompt Gap                               |
| ----------------------- | ------------------------------------ | ---------------------------------------- |
| **Wrong files edited**  | Agent edited unrelated files         | Prompt doesn't scope file targets        |
| **Missed requirements** | Output missing key functionality     | Prompt doesn't emphasize requirements    |
| **Broke existing code** | Tests failed after changes           | Prompt doesn't enforce verification      |
| **Infinite loop**       | Worker/validator cycle >5 iterations | Validation criteria too strict or vague  |
| **Wrong approach**      | Used deprecated API, bad pattern     | Prompt missing technical constraints     |
| **Incomplete work**     | Partial implementation               | Prompt doesn't define "done" clearly     |
| **Hallucinated APIs**   | Called non-existent functions        | Prompt doesn't ground in actual codebase |
| **Ignored context**     | Didn't read provided files           | Context injection not working            |
| **Over-engineering**    | Added unnecessary complexity         | Prompt doesn't enforce simplicity        |
| **Under-testing**       | No tests written                     | Prompt doesn't require tests             |

## Step 3: Trace the Prompt Chain

**Which prompt caused this behavior?**

```
ISSUE_OPENED (user input)
    ↓
conductor-bootstrap.json → junior-conductor/senior-conductor prompts
    ↓
CLUSTER_OPERATIONS (classification)
    ↓
{config}.json → agent prompts (worker, validator, etc.)
    ↓
Agent behavior (good or bad)
```

**Find the weak link:**

1. **Conductor misclassified?** → Fix `cluster-templates/conductor-bootstrap.json`
2. **Wrong config loaded?** → Fix classification logic or config selection
3. **Worker did wrong thing?** → Fix worker prompt in config template
4. **Validator too strict/loose?** → Fix validation criteria
5. **Context missing?** → Fix context injection in agent config

## Step 4: Analyze Prompt Effectiveness

### For Each Agent That Failed:

**1. What was the prompt?**

```bash
# Find the config that was loaded
sqlite3 ledger.db "
  SELECT content_data FROM messages
  WHERE topic = 'CLUSTER_OPERATIONS' LIMIT 1;
" | jq -r '.operations[] | select(.action == "load_config") | .config'

# Read the actual prompt from that config
cat cluster-templates/base-templates/{config}.json | jq '.agents[] | select(.id == "worker") | .prompt'
```

**2. What did the agent actually do?**

- Read the agent's output from logs
- Compare intended behavior vs actual behavior

**3. Where did the prompt fail to guide?**

| Prompt Issue                 | Symptom                              | Fix                                 |
| ---------------------------- | ------------------------------------ | ----------------------------------- |
| **Too vague**                | Agent made random choices            | Add specific constraints            |
| **Too restrictive**          | Agent couldn't solve problem         | Loosen constraints, add flexibility |
| **Missing edge case**        | Agent broke on specific input        | Add explicit handling               |
| **Wrong emphasis**           | Agent focused on wrong thing         | Reorder priorities, use CAPS        |
| **No verification step**     | Agent declared done without checking | Add explicit verification           |
| **No examples**              | Agent misunderstood format           | Add concrete examples               |
| **Conflicting instructions** | Agent did inconsistent things        | Remove contradictions               |

## Step 5: Check Validation Criteria

**Validation failures are PROMPT BUGS, not agent bugs.**

If validator rejects good work:

- Validation criteria too strict
- Validation prompt doesn't understand the task

If validator approves bad work:

- Validation criteria too loose
- Validation prompt missing checks

```bash
# Find validation results
sqlite3 ledger.db "
  SELECT sender, content_text, content_data
  FROM messages
  WHERE topic = 'VALIDATION_RESULT';
"
```

**Questions to answer:**

1. Did validator check the RIGHT things?
2. Did validator understand the requirements?
3. Was rejection reason valid or false positive?
4. Was approval justified or false negative?

## Step 6: Generate Prompt Improvements

### Report Format

```markdown
# Prompt Analysis: [Cluster ID]

## Summary

- **Task**: [what user asked for]
- **Outcome**: [success/failure/partial]
- **Root Cause**: [which prompt failed and why]

---

## 🔴 FAILURE ANALYSIS

### What Went Wrong

> [Specific quote from logs showing the failure]

### Why It Went Wrong

[Analysis of which prompt instruction was missing/wrong/vague]

### The Prompt Gap
```

Current prompt says: "..."
But agent needed: "..."

```

---

## 📊 AGENT BEHAVIOR AUDIT

| Agent | Expected Behavior | Actual Behavior | Gap |
|-------|-------------------|-----------------|-----|
| conductor | Classify as ROUTINE/bugfix | Cla