a-b-test-design
The a-b-test-design skill structures rigorous A/B experiments by establishing clear hypotheses, isolating single variables between control and treatment variants, defining primary and secondary metrics, and calculating required sample sizes based on statistical significance and power. Use this skill when designing product experiments, feature changes, or user experience improvements where controlled comparison and quantifiable measurement can determine impact.
git clone --depth 1 https://github.com/Owl-Listener/designer-skills /tmp/a-b-test-design && cp -r /tmp/a-b-test-design/prototyping-testing/skills/a-b-test-design ~/.claude/skills/a-b-test-designSKILL.md
# A/B Test Design You are an expert in designing rigorous A/B experiments that produce actionable results. ## What You Do You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor. ## Test Structure ### 1. Hypothesis Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].' ### 2. Variants - Control (A): current design - Treatment (B): proposed change - Keep changes isolated — test one variable at a time ### 3. Primary Metric The single most important measure of success. Must be measurable, relevant, and sensitive to the change. ### 4. Secondary Metrics Supporting measures and guardrail metrics to detect unintended consequences. ### 5. Sample Size Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%). ### 6. Duration Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically. ## Common Pitfalls - Peeking at results before completion - Too many variants at once - Metric not sensitive enough to detect change - Sample size too small - Not accounting for novelty effects - Ignoring segmentation effects ## When Not to A/B Test - Very low traffic (insufficient sample) - Ethical concerns with withholding improvement - Foundational changes that affect everything - When qualitative insight is more valuable ## Best Practices - One hypothesis per test - Document everything before starting - Don't stop early on positive results - Analyze segments after overall results - Share learnings broadly regardless of outcome
Audit an existing interface against WCAG, producing findings with severity ratings and remediation steps. Use when you have a design or build to assess now. Not for planning future sessions with assistive-technology users — use `accessibility-test-plan` (prototyping-testing).
Plan accessibility testing — assistive technologies, participant criteria, WCAG coverage, and session protocol. Use when scheduling testing with real AT users. Not for evaluating a design yourself — use `accessibility-audit` (design-systems).
Apply the Aesthetic-Usability Effect — polished, consistent interfaces are perceived as more usable and forgive minor friction. Use when justifying visual polish or diagnosing why a functional design tests badly. For emotional resonance specifically, use `interfaces-that-feel` (interaction-design).
Cluster many qualitative data points into themes and insight statements. Use when synthesising across multiple sessions or sources. For a single transcript use `summarize-interview`; for one segment's inner state use `empathy-map`.
Apply animation principles — easing, staging, follow-through — to one specific UI motion. Use when tuning how an animation feels. For product-wide duration and easing tokens use `motion-system` (design-systems); for a full interaction spec use `micro-interaction-spec`.
A practitioner's toolkit for thinking and communicating as a designer in a business context — reading financials, mapping competitive landscapes, and defending design decisions in the language of value.
Analyse open or closed card sort results into a proposed grouping and label set. Use after running a sort study. For turning that evidence into a full structure, use `information-architecture` (ux-strategy).
Craft a portfolio case study with narrative arc, process evidence, and outcomes. Use when telling a project's story to an external audience. For an internal stakeholder deck, use `presentation-deck`.