Skip to main content
ClaudeWave
Skill65 repo starsupdated 2mo ago

buyer-eval

The buyer-eval skill provides a structured framework for systematically evaluating vendors through a nine-step methodology. Use this skill when you need to assess multiple vendors against consistent criteria, generate standardized evaluation questions, or benchmark vendor capabilities across your organization. The skill includes automatic version management and optional anonymized telemetry to improve question quality.

Install in Claude Code
Copy
git clone https://github.com/salespeak-ai/buyer-eval-skill ~/.claude/skills/buyer-eval
Then start a new Claude Code session; the skill loads automatically.

SKILL.md

## Preamble (run first, every time)

```bash
# Detect skill directory
_BEVAL_DIR=""
for _D in "$HOME/.claude/skills/buyer-eval-skill" ".claude/skills/buyer-eval-skill"; do
  [ -d "$_D" ] && _BEVAL_DIR="$_D" && break
done

if [ -z "$_BEVAL_DIR" ]; then
  echo "ERROR: buyer-eval-skill not found. Install: git clone https://github.com/salespeak-ai/buyer-eval-skill ~/.claude/skills/buyer-eval-skill"
  exit 1
fi

# Check for updates
_UPD=$("$_BEVAL_DIR/bin/update-check" 2>/dev/null || true)
[ -n "$_UPD" ] && echo "$_UPD" || echo "UP_TO_DATE $(cat "$_BEVAL_DIR/VERSION" 2>/dev/null | tr -d '[:space:]')"
```

**If output shows `UPGRADE_AVAILABLE <old> <new>`:**

Use AskUserQuestion to ask the buyer:
- Question: "A newer version of the buyer evaluation skill is available (v{old} → v{new}). Update now?"
- Options: ["Yes, update now", "Not now — continue with current version"]

**If "Yes, update now":**
```bash
_BEVAL_DIR=""
for _D in "$HOME/.claude/skills/buyer-eval-skill" ".claude/skills/buyer-eval-skill"; do
  [ -d "$_D" ] && _BEVAL_DIR="$_D" && break
done

if [ -d "$_BEVAL_DIR/.git" ]; then
  cd "$_BEVAL_DIR" && git pull origin main && echo "UPDATED to $(cat VERSION | tr -d '[:space:]')"
else
  _TMP=$(mktemp -d)
  git clone --depth 1 https://github.com/salespeak-ai/buyer-eval-skill.git "$_TMP/buyer-eval-skill"
  mv "$_BEVAL_DIR" "$_BEVAL_DIR.bak"
  mv "$_TMP/buyer-eval-skill" "$_BEVAL_DIR"
  rm -rf "$_BEVAL_DIR.bak" "$_TMP"
  echo "UPDATED to $(cat "$_BEVAL_DIR/VERSION" | tr -d '[:space:]')"
fi
```
Tell the user the version was updated, then **re-read the EVALUATION.md file** from the updated directory and proceed with the skill.

**If "Not now":** Continue with the current version.

**If output shows `UP_TO_DATE`:** Continue silently.

---

## Load the evaluation skill

After the preamble, read the full evaluation methodology:

```bash
_BEVAL_DIR=""
for _D in "$HOME/.claude/skills/buyer-eval-skill" ".claude/skills/buyer-eval-skill"; do
  [ -d "$_D" ] && _BEVAL_DIR="$_D" && break
done
echo "$_BEVAL_DIR/EVALUATION.md"
```

Read the file at the path printed above using the Read tool. That file contains the complete evaluation methodology — follow it step by step from STEP 1 through STEP 9.

---

## Telemetry (opt-in, off by default)

This skill can send anonymized usage data back to Salespeak so the questions it generates for vendors can keep getting better. **Nothing is ever sent without explicit user consent.** Names, emails, companies, and vendor responses are never sent.

### Initialize telemetry state at run start

Right after loading EVALUATION.md and before STEP 1, run:

```bash
_BEVAL_DIR=""
for _D in "$HOME/.claude/skills/buyer-eval-skill" ".claude/skills/buyer-eval-skill"; do
  [ -d "$_D" ] && _BEVAL_DIR="$_D" && break
done
echo "TELEMETRY_STATE=$(python3 "$_BEVAL_DIR/bin/track.py" status --machine)"
echo "SESSION_ID=$(python3 -c 'import uuid; print(uuid.uuid4())')"
```

Capture both values. Use them throughout the run.

`TELEMETRY_STATE` will be one of:
- `consented` — fire each event live as it happens
- `unasked` — accumulate events in your own working memory; ask for consent at the end
- `declined` or `locked_off` — do nothing telemetry-related for the entire run

### What to track and when

These seven sub-events are the only ones the skill emits. **Do not invent new ones.**

| Sub-event | Fire when | Fields |
|---|---|---|
| `skill_started` | Right after capturing TELEMETRY_STATE | `skill_version` (from VERSION file) |
| `eval_context` | Once, right after STEP 5.1 (category confirmed) | `category` (skill-inferred slug), `vendor_count` (int), `vendors` (array of domains), `company_agents_found` (int — count of vendors with `enabled:true` from Frontdoor discover), `evaluation_path` (`"company_agent_engaged"` \| `"passive_research_only"` \| `"mixed"`) |
| `discovery_question_asked` | After the buyer answers a discovery question. **Fire for STEP 2 (why-now) and for each STEP 5.3 domain-expert question.** | `step` (`"STEP_2"` \| `"STEP_5_3"`), `category` (slug, or `null` for STEP_2), `topic` (short slug you choose, e.g. `"why_now"`, `"high_touch_vs_low_touch"`, `"product_analytics_stack"`), `question_text` (the exact question you asked the buyer) |
| `vendor_question` | **For every (vendor, dimension) pair**, fire one or more events with the question(s) you formulate per the §6.5 question bank — regardless of whether a Company Agent exists. | `vendor` (domain), `category` (slug), `dimension` (the evaluation dimension), `question_text` (the specific question), `delivery_method` (`"asked_via_company_agent"` if actually POSTed via Frontdoor and got an answer, `"would_have_asked"` if no Company Agent existed, `"connection_failed"` if Frontdoor errored) |
| `vendor_scored` | After scoring each dimension for each vendor in STEP 8 | `vendor`, `dimension`, `score` (numeric, 1-5; do NOT fire for `[GAP]` dimensions) |
| `eval_completed` | Right after delivering the final output in STEP 9 | `vendor_count`, `winner` (vendor domain or `null` if no clear winner) |
| `eval_aborted` | Only if the user bails before STEP 9 completes | `at_step` (e.g., `"STEP 6"`) |

**Never include**: buyer name, buyer company, buyer email, anything the buyer typed about themselves, the buyer's *answers* to discovery questions, vendor response text.

### Step-level firing map

Use this as the canonical map between EVALUATION.md steps and event emissions. Fire events at these exact moments — no earlier, no later.

| EVALUATION.md step | Events to fire | Notes |
|---|---|---|
| Right after capturing TELEMETRY_STATE (before STEP 1) | `skill_started` | One event |
| **STEP 2** — buyer answers the why-now question | `discovery_question_asked` (`step:"STEP_2"`, `topic:"why_now"`) | One event. `category` is `null` here. `question_text` is the canonical why-now question. |
| **STEP 5.1** — category confirmed | `eval_context` | One event. `company_agents_found` and `evaluation_path` may not be known yet — use `null` for `company