dspy-optimizer-selection
This skill should be used when the user asks to "choose a DSPy optimizer", "compare DSPy optimizers", "which teleprompter should I use", "optimize prompts or weights", mentions LabeledFewShot, BootstrapFewShotWithRandomSearch, KNNFewShot, COPRO, MIPROv2, SIMBA, GEPA, BootstrapFinetune, Ensemble, or BetterTogether, or needs a cost-aware DSPy optimization plan.
git clone --depth 1 https://github.com/OmidZamani/dspy-skills /tmp/dspy-optimizer-selection && cp -r /tmp/dspy-optimizer-selection/skills/dspy-optimizer-selection ~/.claude/skills/dspy-optimizer-selectionSKILL.md
# DSPy Optimizer Selection ## Goal Choose the smallest DSPy optimizer that matches the data, budget, and artifact being tuned. Establish a baseline before compiling anything. ## Selection Matrix | Need | Start with | Notes | |------|------------|-------| | Include a few labeled examples | `dspy.LabeledFewShot` | Random labeled demos; useful as a baseline | | About 10 examples | `dspy.BootstrapFewShot` | Teacher-generated demos with metric filtering | | 50+ examples and stronger demo search | `dspy.BootstrapFewShotWithRandomSearch` | Searches multiple demo sets; alias: `dspy.BootstrapRS` | | Per-input nearest demos | `dspy.KNNFewShot` | Retrieves nearby examples before bootstrapping | | Instruction-only hill climbing | `dspy.COPRO` | Coordinate ascent over instructions | | Instruction and demo search | `dspy.MIPROv2` | Bayesian search; install `dspy[optuna]` | | Mini-batch introspective rules or demos | `dspy.SIMBA` | Uses output variability and self-reflection | | Rich textual feedback and trace reflection | `dspy.GEPA` | Metric must accept five arguments | | Distill prompts into model weights | `dspy.BootstrapFinetune` | Requires a fine-tunable LM and `set_lm()` | | Combine candidate programs | `dspy.Ensemble` | Trades inference cost for robustness | | Sequence prompt and weight optimization | `dspy.BetterTogether` | Meta-optimizer for configurable optimizer chains | ## Workflow 1. Split data into train and validation sets. 2. Evaluate the uncompiled program with [dspy-evaluation-suite](../dspy-evaluation-suite/SKILL.md). 3. Start with the least expensive optimizer that matches the need. 4. Save the compiled program and compare it against the baseline. 5. Escalate only when the measured gain justifies extra LM calls, fine-tuning, or inference cost. ## Common Paths ### Fast Demo Optimization Use [dspy-bootstrap-fewshot](../dspy-bootstrap-fewshot/SKILL.md) for the first optimization pass. Move to `BootstrapFewShotWithRandomSearch` when enough examples are available to search multiple demo sets. ### Prompt Search Use [dspy-miprov2-optimizer](../dspy-miprov2-optimizer/SKILL.md) for instruction and demonstration search. Install its optional dependency first: ```bash pip install -U "dspy[optuna]>=3.2.1,<3.3" ``` ### Reflective Optimization Use [dspy-gepa-reflective](../dspy-gepa-reflective/SKILL.md) when failures can be described with actionable text. Use [dspy-simba-optimizer](../dspy-simba-optimizer/SKILL.md) for a smaller mini-batch introspective loop with numeric metrics. ### Prompt Plus Weight Optimization Use [dspy-better-together](../dspy-better-together/SKILL.md) when a fine-tunable LM is available and prompt optimization alone has plateaued. ## Best Practices 1. Keep a held-out validation set. 2. Track optimization cost and inference cost separately. 3. Use reproducible seeds where supported. 4. Avoid claiming one optimizer is universally best; compare measured results. 5. Save intermediate candidates for expensive runs. ## Official Documentation - **Optimizer guide**: https://dspy.ai/learn/optimization/optimizers/ - **Optimizer API index**: https://dspy.ai/api/optimizers/ - **DSPy releases**: https://github.com/stanfordnlp/dspy/releases
Use this skill when you need to QA audit and fix a plugin skill file. Provides a methodology for verifying skill content against official documentation, fixing issues in-place, and producing verification reports.
This skill should be used when the user asks to "choose a DSPy adapter", "use JSONAdapter", "use XMLAdapter", "enable native function calling", "send images, audio, or files to DSPy", mentions `dspy.ChatAdapter`, `dspy.JSONAdapter`, `dspy.XMLAdapter`, `dspy.Image`, `dspy.Audio`, `dspy.File`, structured outputs, or multimodal DSPy signatures.
This skill should be used when the user asks to "compose DSPy modules", "use Ensemble optimizer", "combine multiple programs", "use dspy.MultiChainComparison", mentions "ensemble voting", "module composition", "sequential pipelines", or needs to build complex multi-module DSPy programs with ensemble patterns or multi-chain comparison.
This skill should be used when the user asks to "use BetterTogether", "combine prompt optimization and fine-tuning", "sequence DSPy optimizers", "run prompt then weight optimization", mentions `dspy.BetterTogether`, strategy strings such as "p -> w -> p", or needs to compose multiple DSPy teleprompters into an evaluated optimization sequence.
This skill should be used when the user asks to "bootstrap few-shot examples", "generate demonstrations", "use BootstrapFewShot", "optimize with limited data", "create training demos automatically", mentions "teacher model for few-shot", "10-50 training examples", or wants automatic demonstration generation for a DSPy program without extensive compute.
This skill should be used when the user asks to "create custom DSPy module", "design a DSPy module", "extend dspy.Module", "build reusable DSPy component", mentions "custom module patterns", "module serialization", "stateful modules", "module testing", or needs to design production-quality custom DSPy modules with proper architecture, state management, and testing.
This skill should be used when the user asks to "debug DSPy programs", "trace LLM calls", "monitor production DSPy", "use MLflow with DSPy", mentions "inspect_history", "custom callbacks", "observability", "production monitoring", "cost tracking", or needs to debug, trace, and monitor DSPy applications in development and production.
This skill should be used when the user asks to "build local DSPy retrieval", "use dspy.Embedder", "use dspy.Embeddings", "save an embeddings index", "add FAISS retrieval", mentions semantic search, hosted embeddings, local embedding models, `EmbeddingsWithScores`, or needs a DSPy retriever over an application-owned text corpus.