llm-cost-latency-budget
Model the cost and latency of an LLM feature before it ships and surprises the bill. Use when asked to estimate LLM API costs, set a latency/token budget, decide which model tier to use, or bring down the cost of an AI feature. Produces a cost & latency budget — token math per request, monthly cost projection, model tiering, caching/streaming levers, p95 latency targets, and a guardrail/alert plan.
git clone --depth 1 https://github.com/mohitagw15856/pm-claude-skills /tmp/llm-cost-latency-budget && cp -r /tmp/llm-cost-latency-budget/plugins/pm-ai/skills/llm-cost-latency-budget ~/.claude/skills/llm-cost-latency-budgetSKILL.md
# LLM Cost & Latency Budget Skill LLM features have a unit cost and a tail latency that demos hide and production exposes. This skill does the token math up front — what one request costs, what a million cost, where the p95 latency comes from — and lays out the levers (model tiering, caching, prompt trimming) so cost and speed are designed, not discovered. ## Required Inputs Ask for these only if they aren't already provided: - **The request shape** — typical system prompt, user input, retrieved context, and output sizes (in rough tokens). - **Volume** — requests/day now and at target scale; peak concurrency. - **Models in play** — candidate model(s) and their per-token input/output prices. - **Targets** — acceptable cost per request (or per user/month) and the latency users will tolerate (p50 / p95). ## Output Format ### Cost & Latency Budget: [feature] **1. Per-request token math** — a table estimating tokens in/out per call, and the resulting cost at each candidate model's price. | Component | Tokens | $ in | $ out | |---|---|---|---| | System prompt | | | | | Retrieved context | | | | | User input | | | | | Output | | | | | **Per request** | | **$x** | | **2. Monthly projection** — per-request cost × volume, at current and target scale; the headline number leadership will ask for. **3. Model tiering** — route easy requests to a cheaper/faster model and only escalate hard ones (cascade); show the blended cost. Often the single biggest saving. **4. Latency** — where the p95 comes from (model TTFT + output length + retrieval + network), the target, and how **streaming** changes *perceived* latency even when total time is unchanged. **5. Cost levers** — ranked by impact: prompt/context trimming, caching (prompt cache + response cache for repeats), shorter outputs (max_tokens), batching, tiering, and "do you need the model at all for this path." **6. Guardrails** — per-user / per-day rate limits, a max-tokens cap, a spend alert threshold, and a kill switch — so a bug or abuse can't produce a surprise invoice. ## Quality Checks - [ ] Token estimates are itemised (system + context + input + output), not a single guessed number - [ ] The monthly cost is projected at **target** scale, not just today's volume - [ ] Model tiering / cascade is considered before accepting the flagship-model cost everywhere - [ ] p95 (not just average) latency is targeted, and streaming is considered for perceived speed - [ ] Caching is evaluated for repeated prompts/contexts - [ ] A spend alert + rate limit + kill switch are specified to cap the downside ## Anti-Patterns - [ ] Do not budget on average latency — users feel the p95, and the tail is where AI features feel broken - [ ] Do not default every call to the most capable model — most requests don't need it; tiering often cuts cost by more than half - [ ] Do not forget output tokens cost more than input — verbose responses are often the hidden cost driver - [ ] Do not ship without a spend cap and alert — an unbounded LLM feature is an unbounded bill - [ ] Do not optimise cost before measuring it — itemise the real token usage first, then pull the biggest lever ## Based On LLM production cost/latency practice — token accounting, model cascades/tiering, prompt & response caching, and tail-latency budgeting.
Conduct a structured ethical review of an AI or ML feature, model, or product. Use when preparing to deploy an AI system, assessing algorithmic risk, auditing a model for bias, or producing a responsible AI impact assessment. Produces a structured ethics review covering fairness, transparency, privacy, safety, accountability, and societal impact with a risk tier score, pre-deployment checklist, and prioritised mitigations.
Structure AI and ML product decisions with the rigour of any product decision. Use when building AI-powered features, evaluating LLM integrations, designing AI products, or assessing AI readiness. Produces a complete AI product canvas covering problem definition, model approach, data requirements, evaluation framework, UX design, responsible AI checklist, and launch monitoring plan.
Transform feature briefs into structured design briefs that give designers the context they need before opening Figma. Use when asked to write a design brief, create a design handoff, brief a designer on a new feature, or translate a PRD into design requirements. Produces a brief with user goal, emotional context, success criteria, constraints, edge cases, and out-of-scope boundaries.
Design statistically rigorous A/B tests and interpret experiment results. Use when asked to design an experiment, run an A/B test, calculate sample size, interpret test results, or assess whether an experiment was successful. Produces a complete experiment design with hypothesis, sample size, run time, success criteria, and risk flags — or a results interpretation with ship/iterate/kill recommendation.
Synthesises user signals from multiple research sources into a unified, weighted insight brief. Use when you have data from interviews, support tickets, NPS verbatims, app reviews, or sales calls and need to reconcile contradictions, surface the underlying need behind requests, or answer 'what are users really telling us'. Produces ranked insights with confidence ratings, source weighting rationale, divergent signal analysis by user segment, and a research gap identification section.
Structure a product data analysis, metric deep-dive, funnel analysis, or cohort study. Use when asked to analyse product metrics, investigate a drop in conversion, explain a data change to stakeholders, or find the root cause of a metric movement. Produces a structured analysis with question, root cause, confidence level, and recommended action.
Interpret product metrics against goals and surface actionable signals. Use when asked to analyse product health, review key metrics, investigate a performance issue, produce a health report, or assess product-market fit signals. Produces a structured health report with RAG status, trend analysis, root cause hypotheses, and prioritised actions.
Structure a retention analysis, churn investigation, or engagement deep-dive for any product team. Use when asked to analyse user retention, investigate churn, measure DAU/MAU, or build a retention improvement plan. Produces a retention snapshot with root cause hypotheses, aha-moment correlation, and prioritised interventions.