bug-triage

# Bug Triage Skill

Triage bugs into well-structured GitHub issues on the correct upstream repo.

## 1. Pre-flight

- **Pull latest code:** `git pull origin main`. Stale code = bad triage.
- **Target repo:** Always file on the **current repo** (`Untrivial-ai/agent-orchestrator`), not forks.
- **Record source:** chat URL, reporter name, attachments.

## 2. Gather Context

### 2a. Extract the report

| Source | How to gather |
|--------|---------------|
| **Discord/Slack thread** | Read full thread. Extract: reporter name, original description (the thread starter, not whoever tagged you), screenshots, follow-ups |
| **GitHub issue** | `gh issue view <number> --repo <repo> --json body,comments` |
| **Live observation** | Pull live state via observability tools |

### 2b. Minimum viable report gate

Before tracing code, verify the report has enough substance:

**Required (ALL):** what happened, where (page/command/feature), when (after upgrade? first time?)

**Required (2 of 4):** OS/shell/runtime, AO version (`ao --version`), reproducibility (consistent vs intermittent), reproduction steps

If insufficient, ask:
> "I'd like to triage this but need more info: (1) **What happened?** (error/behavior), (2) **Where?** (page/command), (3) **When did it start?**, (4) **How to reproduce?**"

### 2c. Local diagnostics (if bug is on same machine)

Gather everything yourself before asking the reporter:

```bash
# Environment
ao --version && node --version && echo $SHELL && uname -a
cat agent-orchestrator.yaml
cat ~/.agent-orchestrator/running.json

# Process health
pm2 status
tmux list-sessions
lsof -i :3000

# AO event log — structured timeline
ao events list --limit 50                          # recent events
ao events list --session ao-5 --limit 100          # filter by session
ao events list --log-level error --since 1h        # errors only
ao events search "spawn failed"                    # full-text search
ao events stats                                    # counts by kind/source

# Session state files
cat ~/.agent-orchestrator/projects/*/sessions/*.json | python3 -m json.tool
```

Event kinds: `session.spawned`, `session.spawn_failed`, `session.killed`, `lifecycle.transition`, `ci.failing`, `review.pending`, `runtime.probe_failed`, `agent.process_probe_failed`, `reaction.escalated`, `lifecycle.poll_failed`. Sources: `lifecycle`, `session-manager`, `api`, `runtime`, `agent`, `reaction`.

**Try the reproduction steps.** Running the actual command is worth 100 lines of code tracing.

## 3. Investigate

### 3a. Trace the code path

**Always trace the actual code** — don't surface-level diagnose. [#1129](https://github.com/Untrivial-ai/agent-orchestrator/issues/1129) looked like a simple `ao stop` issue but was actually a session lineage/cascade problem.

```bash
git fetch origin main && git log --oneline origin/main -5   # current HEAD
# Record the commit hash you're analyzing against
```

**Git archaeology** — find which commits introduced/removed specific code:
```bash
git log --oneline -S 'exact-string' -- <file>
git show <sha> -- <file> | grep -B 5 -A 10 'pattern'
```
Example: [#1391](https://github.com/Untrivial-ai/agent-orchestrator/issues/1391) traced a mobile layout break to a `display: flex` → `display: grid` change.

**Research upstream dependencies** (xterm, node-pty, React, etc.) — check installed vs latest version, search their GitHub issues, check changelogs. Root cause is often upstream.

### 3b. Cross-platform check

AO runs on **Windows, macOS, Linux** as first-class targets. If env info indicates Windows (or is unknown), check for these patterns:

- **Path separators** — hardcoded `/` or `\` breaks cross-platform
- **Shell syntax** — PowerShell lacks `&&`, `$VAR`, `$(cat ...)`, `/dev/null`, here-docs
- **`process.platform === "win32"` inline** — must use `isWindows()` from `@aoagents/ao-core`
- **`process.kill(-pid)`** — POSIX-only; use `killProcessTree()`
- **Named pipes vs Unix sockets** — Windows uses `\\.\pipe\ao-pty-<id>`
- **`localhost`** — Windows resolves to `::1` first, causing ~21s stalls on IPv4-only servers
- **NTFS case-insensitivity** — use `pathsEqual()`, not `===`
- **ConPTY orphans** — can trigger WER dialogs if pty-host not shut down cooperatively
- **`.cmd` shim resolution** — needs `shell: true` for `PATHEXT` lookup

Key files: `packages/core/src/platform.ts`, `docs/CROSS_PLATFORM.md`, `packages/plugins/runtime-process/`, `packages/cli/src/lib/path-equality.ts`

### 3c. Stop-and-ask triggers

Stop and ask for more info if:
- **3 failed hypotheses** — traced 3 code paths, none explain it
- **Root cause is upstream** — file with upstream reference, don't guess a local fix
- **UI-only bug** and you can't screenshot — ask reporter to describe
- **Can't reproduce** — ask for different config/sequence

## 4. Search for Duplicates

Search with multiple strategies, always using `--state all` (closed bugs regress):

```bash
gh issue list --repo <repo> --state all --search "<symptom>"
gh issue list --repo <repo> --state all --search "<component-name>"
gh issue list --repo <repo> --state all --search "<error-message>"
gh pr list --repo <repo> --state all --search "<keywords>"
```

### Duplicate found → comment on existing issue

```bash
gh issue comment <number> --repo <repo> --body "$(cat <<'EOF'
## New Report
**Reported by:** @<reporter> in [chat](<url>)
**Date:** <YYYY-MM-DD> | **Checkout:** `<commit-hash>`
<context, differences from original, screenshots>
EOF
)"
```

### No duplicate → file new issue (next section)

## 5. File New Issue

### 5a. Pre-submission checklist

- [ ] Reporter attribution correct (original reporter, not who tagged you)
- [ ] Commit hash recorded
- [ ] AO version recorded
- [ ] Root cause confidence scored (see 5c)
- [ ] Related issues cross-linked
- [ ] Reproduction steps are concrete
- [ ] Screenshots uploaded with real URLs (see 5b)

### 5b. Upload screenshots

**⛔ NEVER use placeholder URLs.** Upload BEFORE creating the issue. ([#1151](h