Skip to main content
ClaudeWave
Skill543 repo starsupdated 3d ago

skill-iter-tune

Iterative skill tuning via execute-evaluate-improve feedback loop.

Install in Claude Code
Copy
git clone --depth 1 https://github.com/catlog22/maestro-flow /tmp/skill-iter-tune && cp -r /tmp/skill-iter-tune/.codex/skills/skill-iter-tune ~/.claude/skills/skill-iter-tune
Then start a new Claude Code session; the skill loads automatically.

SKILL.md

> **Plan tracking**: codex 无 TaskCreate/TaskUpdate/TodoWrite 任务板。进度清单用 `update_plan({ explanation?, plan: [{ step, status }] })` 维护(整体提交步骤数组,status: `pending` | `in_progress` | `completed`),权威状态始终在 session 工件中;依赖/认领(addBlockedBy/owner)是工件字段,不是工具参数。

<required_reading>
@~/.maestro/workflows/run-mode.md
@~/.maestro/workflows/codex-run-mode.md
</required_reading>

# Skill Iter Tune

Iterative skill refinement through execute-evaluate-improve feedback loops. Each iteration runs the skill via Claude, evaluates output via Agy, and applies improvements via Agent.

## Architecture Overview

```
┌──────────────────────────────────────────────────────────────────────────┐
│  Skill Iter Tune Orchestrator (SKILL.md)                                 │
│  → Parse input → Setup workspace → Iteration Loop → Final Report         │
└────────────────────────────┬─────────────────────────────────────────────┘
                             │
         ┌───────────────────┼───────────────────────────────────┐
         ↓                   ↓                                   ↓
    ┌──────────┐      ┌─────────────────────────────┐     ┌──────────┐
    │ Phase 1  │      │  Iteration Loop (2→3→4)     │     │ Phase 5  │
    │ Setup    │      │  ┌─────┐  ┌─────┐  ┌─────┐ │     │ Report   │
    │          │─────→│  │ P2  │→ │ P3  │→ │ P4  │ │────→│          │
    │ Backup + │      │  │Exec │  │Eval │  │Impr │ │     │ History  │
    │ Init     │      │  └─────┘  └─────┘  └─────┘ │     │ Summary  │
    └──────────┘      │       ↑               │     │     └──────────┘
                      │       └───────────────┘     │
                      │    (if score < threshold    │
                      │     AND iter < max)         │
                      └─────────────────────────────┘
```

### Chain Mode Extension

```
Chain Mode (execution_mode === "chain"):

Phase 2 runs per-skill in chain_order:
  Skill A → maestro delegate → artifacts/skill-A/
       ↓ (artifacts as input)
  Skill B → maestro delegate → artifacts/skill-B/
       ↓ (artifacts as input)
  Skill C → maestro delegate → artifacts/skill-C/

Phase 3 evaluates entire chain output + per-skill scores
Phase 4 improves weakest skill(s) in chain
```

## Key Design Principles

1. **Iteration Loop**: Phases 2-3-4 repeat until quality threshold, max iterations, or convergence
2. **Two-Tool Pipeline**: Claude (write/execute) + Agy (analyze/evaluate) = complementary perspectives
3. **Pure Orchestrator**: SKILL.md coordinates only — execution detail lives in phase files
4. **Progressive Phase Loading**: Phase docs read only when that phase executes
5. **Skill Versioning**: Each iteration snapshots skill state before execution
6. **Convergence Detection**: Stop early if score stalls (no improvement in 2 consecutive iterations)

## Interactive Preference Collection

```javascript
// ★ Auto mode detection
const autoYes = /\b(-y|--yes)\b/.test($ARGUMENTS)

if (autoYes) {
  workflowPreferences = {
    autoYes: true,
    maxIterations: 5,
    qualityThreshold: 80,
    executionMode: 'single'
  }
} else {
  const prefResponse = request_user_input({
    questions: [
      {
        question: "选择迭代调优配置:",
        header: "Tune Config",
        multiSelect: false,
        options: [
          { label: "Quick (3 iter, 70)", description: "快速迭代,适合小幅改进" },
          { label: "Standard (5 iter, 80) (Recommended)", description: "平衡方案,适合多数场景" },
          { label: "Thorough (8 iter, 90)", description: "深度优化,适合生产级 skill" }
        ]
      }
    ]
  })

  const configMap = {
    "Quick": { maxIterations: 3, qualityThreshold: 70 },
    "Standard": { maxIterations: 5, qualityThreshold: 80 },
    "Thorough": { maxIterations: 8, qualityThreshold: 90 }
  }
  const selected = Object.keys(configMap).find(k =>
    prefResponse["Tune Config"].startsWith(k)
  ) || "Standard"
  workflowPreferences = { autoYes: false, ...configMap[selected] }

  // ★ Mode selection: chain vs single
  const modeResponse = request_user_input({
    questions: [{
      question: "选择调优模式:",
      header: "Tune Mode",
      multiSelect: false,
      options: [
        { label: "Single Skill (Recommended)", description: "独立调优每个 skill,适合单一 skill 优化" },
        { label: "Skill Chain", description: "按链序执行,前一个 skill 的产出作为后一个的输入" }
      ]
    }]
  });
  workflowPreferences.executionMode = modeResponse["Tune Mode"].startsWith("Skill Chain")
    ? "chain" : "single";
}
```

## Input Processing

```
$ARGUMENTS → Parse:
  ├─ Skill path(s): first arg, comma-separated for multiple
  │   e.g., ".claude/skills/my-skill" or "my-skill" (auto-prefixed)
  │   Chain mode: order preserved as chain_order
  ├─ Test scenario: --scenario "description" or remaining text
  └─ Flags: --max-iterations=N, --threshold=N, -y/--yes
```

## Execution Flow

> **⚠️ COMPACT DIRECTIVE**: Context compression MUST check update_plan phase status.
> The phase currently marked `in_progress` is the active execution phase — preserve its FULL content.
> Only compress phases marked `completed` or `pending`.

### Phase 1: Setup (one-time)

Read and execute: `Ref: phases/01-setup.md`

- Parse skill paths, validate existence
- Create workspace at `{run_dir}/outputs/skill-iter-tune-{ts}/`
- Backup original skill files
- Initialize iteration-state.json

Output: `workDir`, `targetSkills[]`, `testScenario`, initialized state

### Iteration Loop

```javascript
// Orchestrator iteration loop
while (true) {
  // Increment iteration
  state.current_iteration++;
  state.iterations.push({
    round: state.current_iteration,
    status: 'pending',
    execution: null,
    evaluation: null,
    improvement: null
  });

  // Update update_plan
  update_plan(iterationTask, {
    subject: `Iteration ${state.current_iteration}/${state.max_iterations}`,
    status: 'in_progress',
    activeForm: `Running iteration ${state.current_iteration}`
  });

  // === Phase 2: Execute ===
  // Read: phases/02-execute.md
  // Single mode: one maestro delegate call for all skills
  // Chain mode: sequen