running-coral-experiments
Run and manage CORAL experiments from the operator side — launch agents with `coral start` (dotlist overrides, model/count, tmux vs local), monitor with `coral status` / `coral log` / `coral show` / the web dashboard, and drive the loop with `coral resume` (inject instructions, fork from an attempt), `coral heartbeat` (tune reflection cadence), and `coral stop`. Use whenever the user wants to start a CORAL run, check on agents, read scores/leaderboard, steer or resume a run, diagnose agents that keep restarting or fail every eval, scale to more agents or islands, or stop a run. Deep references for steering/heartbeat tuning and scaling/troubleshooting live alongside this skill.
git clone --depth 1 https://github.com/Human-Agent-Society/CORAL /tmp/running-coral-experiments && cp -r /tmp/running-coral-experiments/plugin/skills/running-coral-experiments ~/.claude/skills/running-coral-experimentsSKILL.md
# Running CORAL experiments
You drive a run with five verbs: **start → status → log/show → resume → stop**. Everything else is a flag on those or a deeper topic in the references. Prefer `coral <cmd> --help` over guessing flags.
**Prereq:** a task (`task.yaml` + `seed/` + grader package) that passes `coral validate .`. No task yet → that's the `creating-a-coral-task` skill. Each runtime CLI must be installed and authenticated → the `setting-up-coral` skill.
## 1. Launch
```bash
coral start -c task.yaml # auto-tmux session
coral start -c task.yaml agents.count=4 agents.model=opus # dotlist overrides (no quotes needed)
coral start -c task.yaml run.verbose=true run.ui=true # verbose logs + web dashboard
coral start -c task.yaml run.session=local # foreground, no tmux
```
- **Dotlist overrides** (`key.subkey=value`) beat `task.yaml` for this run only — the clean way to sweep count/model without editing the file.
- `run.session`: `tmux` (default, detachable) · `local` (foreground) · `docker`.
- Each run lands in `results/<task-slug>/<timestamp>/`; agents work in isolated git worktrees and the grader daemon scores their commits.
## 2. Monitor
```bash
coral status # agent health + leaderboard snapshot (the quick pulse)
coral runs # active runs across tasks; --all includes finished
coral ui --port 8420 # web dashboard: live leaderboard, logs, DAG
```
`coral status` answers "who's alive, how many evals, current best". If it looks healthy but scores never move, jump to budget classes + troubleshooting in [references/scaling-and-ops.md](references/scaling-and-ops.md).
## 3. Read results
```bash
coral log # top 20 real attempts by score
coral log -n 5 --recent # most recent instead of best
coral log --search "kernel" --agent agent-1
coral log --class grader_error # surface crashing graders (first stop when unhealthy)
coral show <hash> # one attempt: score, explanation, files changed
coral show <hash> --diff # full diff — see exactly what the leader did
```
`<hash>` comes from `coral log`/`coral status`. By default `coral log` hides `tune` and `grader_error` attempts; `--all` shows them, `--class {real|tune|grader_error}` filters to one. What the classes mean → [references/scaling-and-ops.md](references/scaling-and-ops.md).
## 4. Steer and resume
```bash
coral resume # resume latest run, sessions restored
coral resume -i "Try greedy approaches first" # inject guidance agents read next loop
coral resume --from <hash> -i "Continue this fork" # reset an agent to an attempt, then steer
coral export <hash> -b winning-idea # export an attempt's commit as a git branch
```
`resume -i` is how you nudge a run without restarting from scratch (stop → resume with an instruction). `--from` forks a promising line that later regressed. You can also retune the reflection cadence — `coral heartbeat set/remove/reset` — to make agents reflect less, pivot sooner, etc. Both topics, with worked examples: [references/steering.md](references/steering.md).
## 5. Stop
```bash
coral stop # stop the current/latest run (picker if several)
coral stop --all # stop every active run
```
Stopping leaves all results, notes, and the leaderboard on disk — `coral resume` later, or just inspect with `coral log`/`coral show`.
## Typical loop
```bash
coral validate . # grader scores the seed (once)
coral start -c task.yaml agents.count=2 # launch
coral status # ... check periodically
coral log -n 5 --recent # see what agents are trying
coral show <best-hash> --diff # inspect the leader
coral resume -i "Focus on the inner loop" # steer if they plateau
coral stop # done
```
## Going deeper
- **Steer / fork / heartbeat tuning** → [references/steering.md](references/steering.md)
- **Budget classes, islands, gateway, troubleshooting matrix** → [references/scaling-and-ops.md](references/scaling-and-ops.md)
Note: `coral eval / diff / revert / checkout / wait` are **agent-side** commands run *inside* a worktree during a run — agents already know them from the generated `CORAL.md`. As the operator you rarely touch them; you drive the verbs above. Full CLI reference: https://docs.coralxyz.com/cli/referenceVerify and debug changes to CORAL itself — smallest reproduce loop per area (grader / daemon / CLI / hooks / manager / workspace / hub / template / config / web), where to look when something breaks (hung graders, agent restart loops, stalled agents, missing heartbeat actions, corrupted shared state, broken worktree symlinks, grader import errors, wrong-task resume), how to inspect a live or finished run under `.coral/public/`, and the canonical lint/test commands. Use when editing code under `coral/` or chasing a CORAL bug, NOT when adding a new task or extending the framework.
Add a new component to the CORAL framework itself — a new agent runtime under `coral/agent/builtin/` (claude_code/codex/cursor_agent style), a new CLI command in `coral/cli/`, a new bundled skill or subagent template under `coral/template/skills/` or `coral/template/agents/`, a new hook in `coral/hooks/`, a new field in `coral/config.py`, or a framework-level extension to the grader stack under `coral/grader/`. NOT for writing a per-task grader or adding an example task — use `coral-new-task` for that. NOT for debugging existing code — use `coral-debug`.
End-to-end recipe for adding a new task under `examples/` — the three pieces that have to line up (`task.yaml`, `seed/`, and `grader/`), what to put in each, the `TaskGrader` API surface, the `coral validate` → smoke-test loop, and the common mistakes (repo_path pointing at the wrong dir, score direction backwards, hidden answer keys leaking into seed/, grader writing to codebase_path which the daemon force-removes, private-vs-public confusion, missing `run()` signature). Use whenever the user wants to add a new CORAL task or port an existing benchmark into CORAL.
Research the problem domain before coding. Web search for techniques, save raw sources, write structured findings, update the index.
Organize the shared notes directory when it becomes hard to navigate. Restructure within research/ and experiments/, deduplicate, update index.md.
Autonomously create, test, and optimize skills by detecting reusable patterns in your own work. Use when you notice repeated tool sequences, recurring code patterns across attempts, or insights that should be captured as a packaged skill. Also use to benchmark and iterate on existing skills.
The fast path from zero to a running CORAL experiment — what CORAL is and when to reach for it, installing the `coral` CLI, registering a runtime with `coral setup`, and the `.coral_workspace/` convention for pointing CORAL at code you already have and want optimized. Use this whenever the user asks "what is coral", "should I use coral for this", wants to install or get coral set up, hits a "command not found" for coral or doesn't have it installed yet, or says "use coral to optimize / speed up / improve this code" and you need the end-to-end onboarding from install to a launched run. Hands off to `setting-up-coral` (runtime bindings), `creating-a-coral-task` (grader authoring), and `running-coral-experiments` (operating a run) for depth.
Author a new CORAL task — the three pieces that must line up (`task.yaml`, `seed/`, a packaged `grader/`), the `coral init` → `coral validate` → smoke-test loop, and how to pick a grader pattern (stdout float, test pass-rate, ratio-vs-baseline, multi-metric, or an LLM rubric judge). Use whenever the user wants to create a CORAL task, write or wire a grader, port a benchmark into CORAL, score open-ended outputs (reports/memos) with a judge, or debug a grader that crashes on the seed / ranks the leaderboard backwards / leaks the answer key. Deep references for the TaskGrader API, grader patterns, rubric judges, and the full task.yaml schema live alongside this skill.