qdrant-search-quality-diagnosis

# How to Diagnose Bad Search Quality

Before tuning, establish baselines. Use exact KNN as ground truth, compare against approximate HNSW. Target >95% recall@K for production.

## Don't Know What's Wrong Yet

Use when: results are irrelevant or missing expected matches and you need to isolate the cause.

- For a no-code quick check, use the Web UI's ANN Recall tab to compare approximate vs exact `recall@k` [Web UI ANN Recall](https://skills.qdrant.tech/md/documentation/tutorials-search-engineering/ann-recall/?s=measure-ann-recall-with-the-web-ui)
- For the same comparison in code (CI gating, regression tests), run each query twice — once approximate, once with `exact=true` — and compute `recall@k` from the overlap [ANN recall in CI](https://skills.qdrant.tech/md/documentation/tutorials-search-engineering/ann-recall/?s=automate-in-ci-with-python)
- Exact search bad = model or search pipeline problem. Exact good, approximate bad = tune HNSW.
- Check if quantization degrades quality (compare with and without)
- Check if filters are too restrictive (then you might need to use ACORN)
- If duplicate results from chunked documents, use Grouping API to deduplicate [Grouping](https://skills.qdrant.tech/md/documentation/search/search/?s=grouping-api)

Payload filtering and sparse vector search are different things. Metadata (dates, categories, tags) goes in payload for filtering. Text content goes in sparse vectors for search.

## Approximate Search Worse Than Exact

Use when: exact search returns good results but HNSW approximation misses them.

- Increase `hnsw_ef` at query time [Search params](https://skills.qdrant.tech/md/documentation/ops-optimization/optimize/?s=fine-tuning-search-parameters)
- Increase `ef_construct` (200+ for high quality) [HNSW config](https://skills.qdrant.tech/md/documentation/manage-data/indexing/?s=vector-index)
- Increase `m` (16 default, 32 for high recall) [HNSW config](https://skills.qdrant.tech/md/documentation/manage-data/indexing/?s=vector-index)
- Enable oversampling + rescore with quantization [Search with quantization](https://skills.qdrant.tech/md/documentation/manage-data/quantization/?s=searching-with-quantization)
- ACORN for filtered queries (v1.16+) [ACORN](https://skills.qdrant.tech/md/documentation/search/search/?s=acorn-search-algorithm)

Binary quantization requires rescore. Without it, quality loss is severe. Use oversampling (3-5x minimum for binary) to recover recall. Always test quantization impact on your data before production. [Quantization](https://skills.qdrant.tech/md/documentation/manage-data/quantization/)

## Wrong Embedding Model

Use when: exact search also returns bad results.

Check [Qdrant team recommendations on how to choose an embedding model](https://skills.qdrant.tech/md/articles/how-to-choose-an-embedding-model/).

Test top 3 MTEB models on 100-1000 sample queries [Hosted Qdrant inference](https://skills.qdrant.tech/md/documentation/inference/). Score them against a labeled set to compare apples to apples [Measuring Retrieval Relevance](https://skills.qdrant.tech/md/documentation/improve-search/retrieval-relevance/).

## Unoptimized Search Pipeline

Use when: exact search also returns bad results and model choice is confirmed by user.

Optimize search according to advanced search-strategies skill.

## Need a Labeled Baseline to Score Recall, MRR, or NDCG

Use when: user has no golden set, asks "how do I know if my search is good?", or needs to gate releases on a retrieval metric.

- Build a labeled query set — human, log-based, or LLM-synthetic — and score retrieval with `ranx` [Measuring Retrieval Relevance](https://skills.qdrant.tech/md/documentation/improve-search/retrieval-relevance/)
- Pick the metric by usage: `Recall@k` for RAG, `MRR`/`Hits@1` for single-answer, `NDCG@k` for re-ranking [Choosing the metric](https://skills.qdrant.tech/md/documentation/improve-search/retrieval-relevance/?s=choosing-the-right-metric)
- For full RAG pipelines, also score generation with Ragas and use the retrieval-vs-generation 2x2 to isolate regressions [Pipeline Output Quality](https://skills.qdrant.tech/md/documentation/improve-search/pipeline-output-quality/)
- Gate CI on a per-metric threshold to catch regressions from embedding-model swaps, prompt changes, or index config changes

## What NOT to Do

- Tune Qdrant before verifying the model is right for the task (most quality issues are model issues)
- Use binary quantization without rescore (severe quality loss)
- Set `hnsw_ef` lower than results requested (guaranteed bad recall)
- Skip payload indexes on filtered fields then blame quality (HNSW can't traverse filtered-out nodes, and filterable HNSW is built only if payload indexes were set up prior)
- Deploy without baseline recall or other search relevance metrics (no way to measure regressions)
- Confuse payload filtering with sparse vector search (different things, different config)