geo-probe

# geo-probe

Measure whether AI answer engines surface a brand. Split because a skill can't reach the model providers: this skill generates the queries and scores the answers; **the agent calls the model / `web_search` for each query** and feeds the answers back.

## When to use

- A GEO/visibility pass: "do AI engines mention or cite us, and how do we compare to competitors?"
- After GEO fixes, re-probe to see if mention/citation rates moved.

## When NOT to use

- On-page GEO readiness (citability/structure/entity) — that is `geo-score`, which needs no model calls.
- When you can't make model calls — without answers, `score` has nothing to measure.

## Preconditions

- `queries`: a `seo-crawl` JSON. `score`: an answers payload (below). Python 3.9+ stdlib only. The agent provides model answers between the two ops.

## How to call

1) Generate queries:
```
"$ORKAS_NODE" "$ORKAS_PC_DIR/bin/run-skill.cjs" geo-probe geo_probe -- --op queries --input <crawl.json> [--brand X] [--domain x.com] [--competitors "A,B"]
```
→ `{ ok, data: { brand, domain, competitors, context_terms, queries:[{query, kind, intent}] } }`

`kind` is `unbranded` (the measurement set) or `branded` (control). **A branded
query cannot measure visibility** — asked "What is <brand>?" a model names the
brand by construction, so counting those rows reports a share of voice the probe
never tested. Branded rows are kept only to separate "nobody recommends us" from
"the model does not know we exist"; keep their `kind` when you feed answers back.

1b) Validate agent- or user-supplied candidates before probing with them:
```
echo '{"brand":"Orkas","domain":"orkas.ai","candidates":["best ai agent tools for teams","..."]}' | "$ORKAS_NODE" "$ORKAS_PC_DIR/bin/run-skill.cjs" geo-probe geo_probe -- --op filter
```
→ `{ ok, data: { kept:[...], rejected:[{query, reason}] } }`. Rejects a query that
carries the brand or domain core, duplicates, is under 3 or over 12 words, uses a
bare ambiguous acronym (`GEO`, `AEO`, `CRM`… without its expansion — an answer
engine will answer for the wrong industry), or compares AI answer engines rather
than vendors. Every drop names its reason; never discard one silently.

2) The agent asks each query to one or more models / `web_search`, recording `{query, model, mode:"param"|"retrieval", text}` (mode = whether the model retrieved sources or answered from memory).

The `queries` op also returns `context_terms` (distinctive page-vocabulary words). **Pass them through into the score payload** so the brand can be disambiguated from a homonym.

3) Score the answers (pass the payload on stdin or `--input`):
```
echo '{"brand":"Orkas","domain":"orkas.ai","competitors":["Cursor"],"context_terms":["ai","agent","desktop"],"answers":[{"query":"...","model":"...","mode":"retrieval","text":"..."}]}' | "$ORKAS_NODE" "$ORKAS_PC_DIR/bin/run-skill.cjs" geo-probe geo_probe -- --op score
```
→ `{ ok, data: { share_of_voice, share_of_voice_basis, unbranded, branded_control, share_of_voice_all_answers, citation_rate, brand_mentions, domain_citations, ambiguous_mentions, competitor_share, context_terms, per_answer:[...], data_tier, note } }`

## Honesty

- **`share_of_voice` is measured over `unbranded` rows only.** Report it against
  `branded_control`: unbranded 0 with control 1.0 means the brand is recognised
  but never recommended; both near 0 means the answer engines do not know the
  entity at all — a different problem with a different fix. `share_of_voice_all_answers`
  is the un-split number, kept for comparison only; do not headline it. When a
  probe set carried no unbranded row, `share_of_voice_basis` says so instead of
  presenting the branded average as a clean score.
- `share_of_voice` counts only **corroborated product mentions**: the answer cites the domain, OR the brand token appears together with a page-context term. A brand-token hit with no context term and no domain is **`ambiguous`** (likely a homonym, e.g. "Orkas" → orcas/whales) and is excluded from share_of_voice (surfaced as `ambiguous_mentions`). Without `context_terms`, it falls back to counting any brand-token hit.
- `citation_rate` counts sourced domain citations and is the most reliable signal.
- `data_tier` is `Measured` only when every answer came from a retrieval-capable model, otherwise `Estimated`. Always report which it is — never present a parametric-memory mention as a real citation.