lastsearch

# LastSearch

[![npm](https://img.shields.io/npm/v/lastsearch)](https://www.npmjs.com/package/lastsearch)
[![PyPI](https://img.shields.io/pypi/v/lastsearch)](https://pypi.org/project/lastsearch/)
[![LangChain](https://img.shields.io/pypi/v/langchain-lastsearch?label=langchain-lastsearch)](https://pypi.org/project/langchain-lastsearch/)
[![License](https://img.shields.io/badge/License-Apache_2.0_(open--core)-blue.svg)](LICENSE)
[![Discord](https://img.shields.io/badge/Discord-Join-5865F2?logo=discord&logoColor=white)](https://discord.gg/ubAuT4YQsT)

**Research infrastructure for AI agents with Grounded Intelligence** — real-time web search, evidence extraction, verification, and structured citations. Every claim is backed by a URL. Every answer has a confidence score.

```
Agent → LastSearch → Internet → Verified answers + sources
```

[Website](https://lastsearch.ai) · [Playground](https://lastsearch.ai/playground) · [API Docs](https://lastsearch.ai/developers) · [Alternatives](https://lastsearch.ai/alternatives) · [Discord](https://discord.gg/ubAuT4YQsT)

> **Package names:** npm: [`lastsearch`](https://www.npmjs.com/package/lastsearch) · PyPI: [`lastsearch`](https://pypi.org/project/lastsearch/) · LangChain: [`langchain-lastsearch`](https://pypi.org/project/langchain-lastsearch/) — Previously `lastsearch` and `lastsearch`. Old names still work and redirect automatically.

---

## How It Works

```
search → fetch pages → neural rerank → extract claims → verify → cited answer (streamed)
```

Every answer goes through a multi-step verification pipeline. No hallucination. Every claim is backed by a real source.

### Verification & Confidence Scoring

Confidence scores are **evidence-based** — not LLM self-assessed. After the LLM extracts claims and sources, a post-extraction verification engine checks every claim against the actual source page text:

1. **Atomic claim decomposition** — Compound claims are auto-split into individual verifiable facts. "Tesla had $96B revenue and 1.8M deliveries" becomes two atomic claims, each verified independently.
2. **Hybrid retrieval combining keyword and semantic matching** — For each claim, keyword matching finds lexical matches and dense embeddings find semantic matches from source text. Rankings are fused to catch paraphrased evidence that keyword matching alone misses (e.g., "prevents fabricated answers" matching "reduces hallucinations"). Premium tier only, with graceful keyword-only fallback.
3. **Semantic evidence reranking** — Top candidates per claim are reranked by a **purpose-built verification model trained on 1.4M+ claim-evidence pairs** that improves with every query. Selects the best supporting evidence, applies contradiction penalties and paraphrase boosts.
4. **Multi-provider search** — Parallel search across multiple providers for broader source diversity. More independent sources = stronger cross-reference = higher confidence.
5. **Domain authority scoring** — 10,000+ domains across 5 tiers (institutional `.gov`/`.edu` → major news → tech journalism → community → low-quality). Dynamic scoring that improves from real verification data.
6. **Source quote verification** — LLM-extracted quotes verified against actual page text using multi-strategy matching.
7. **Cross-source consensus** — Each claim verified against *all* available page texts. Claims supported by 3+ independent domains get "strong consensus". Single-source claims flagged as "weak".
8. **Contradiction detection** — Claim pairs analyzed for semantic conflicts using topic overlap and contradiction classification. Detected contradictions surfaced in the response and penalize confidence.
9. **Multi-pass consistency** — In thorough mode, claims are cross-checked across independent extraction passes. Claims confirmed by both passes get boosted; inconsistent claims are penalized.
10. **Auto-calibrated confidence** — Multi-factor confidence formula auto-adjusts from real user feedback. Predicted confidence aligns with actual accuracy over time. Factors: verification rate, domain authority, source count, consensus, domain diversity, claim grounding, source recency, and citation depth.
11. **Per-claim evidence retrieval** — Weak claims get targeted search queries generated by LLM, then searched individually across all providers. Each claim gets its own evidence pool instead of sharing the same corpus.
12. **Counter-query verification** — Verified claims are stress-tested with adversarial "what would disprove this?" search queries. If counter-evidence is found, claim confidence is penalized.
13. **Iterative confidence-gated retrieval** — Thorough mode uses a confidence-gated loop: verify → if weak claims remain → generate targeted query → search → re-verify. Loops up to 3 iterations with early termination when queries repeat or confidence meets threshold.

Claims include `verified`, `verificationScore`, `consensusCount`, and `consensusLevel` fields. Sources include `verified` and `authority`. Detected `contradictions` are returned at the top level. Agents can use these fields to make trust decisions programmatically.

> **Graceful fallback:** When premium keys are not set, the system runs keyword-only verification. Semantic retrieval and reranking are transparent premium enhancements — no degradation, no errors.

### Depth Modes

Three depth levels control research thoroughness:

| Depth | Behavior | Use case |
|-------|----------|----------|
| `fast` (default) | Single search → extract → verify pass | Quick lookups, real-time agents |
| `thorough` | Iterative confidence-gated loop (up to 3 passes), per-claim evidence retrieval, counter-query verification, multi-pass consistency checking | Important research, fact-checking |
| `deep` | Premium multi-step agentic research: iterative think-search-extract-evaluate cycles (up to 4 total steps). Gap analysis identifies missing info, generates follow-up queries. Claims/sources merged across steps with final re-verification. Target confidence: 0.85. Requires LastSearch key + sign-in. Falls back to thorough when quota exhausted. | Complex research questions, comprehensive analysis |

```bash
# Thorough mode
curl -X POST https://lastsearch.ai/api/browse/answer \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ls_xxx" \
  -d '{"query": "What is quantum computing?", "depth": "thorough"}'

# Deep mode (uses premium features)
curl -X POST https://lastsearch.ai/api/browse/answer \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ls_xxx" \
  -d '{"query": "Compare CRISPR approaches for sickle cell disease", "depth": "deep"}'
```

Deep mode runs iterative think-search-extract-evaluate cycles: each step performs gap analysis to identify what's missing, generates targeted follow-up queries, and merges claims/sources across steps with a final re-verification pass. It targets a confidence threshold of 0.85 (`DEEP_CONFIDENCE_THRESHOLD`) and runs up to 3 follow-up steps (`MAX_FOLLOW_UP_STEPS`, 4 total including the initial pass). Uses semantic reranking, multi-provider search, and multi-pass consistency. Each deep query costs 3x quota (100 deep queries/day). When quota is exhausted, deep mode gracefully falls back to thorough. Without a LastSearch key, deep mode also falls back to thorough.

Deep mode responses include `reasoningSteps` showing the multi-step research process (step number, query, gap analysis, claim count, confidence per step).

### Streaming API

Get real-time progress with per-token answer streaming. The streaming endpoint sends Server-Sent Events (SSE) as each pipeline step completes. Deep mode steps are grouped by research pass for clean progress display:

```bash
curl -N -X POST https://lastsearch.ai/api/browse/answer/stream \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ls_xxx" \
  -d '{"query": "What is quantum computing?"}'
```

Events: `trace` (progress), `sources` (discovered early), `token` (streamed answer text), `result` (final answer), `done`.

### Retry with Backoff

All external API calls (search providers, LLM, page fetching) automatically retry on transient failures (429 rate limits, 5xx server errors) with exponential backoff and jitter. Auth errors (401/403) fail immediately — no wasted retries.

### Research Memory (Sessions)

Persistent research sessions that accumulate knowledge across multiple queries. Later queries automatically recall prior verified claims, building deeper understanding over time.

> **Sessions require a LastSearch API key (`ls_xxx`)** for identity and ownership. Get a free key at [lastsearch.ai/dashboard](https://lastsearch.ai/dashboard). For MCP, set `LASTSEARCH_API_KEY` env var. For Python SDK, pass `api_key="ls_xxx"`. For REST API, use `Authorization: Bearer ls_xxx`.

```python
# Python SDK
session = client.session("quantum-research")
r1 = session.ask("What is quantum entanglement?")       # 13 claims stored
r2 = session.ask("How is entanglement used in computing?")  # 12 claims recalled!
knowledge = session.knowledge()  # Export all accumulated claims

# Share with other agents or humans
share = session.share()  # Returns shareId + URL
# Another agent forks and continues the research
forked = client.fork_session(share.share_id)
```

```bash
# REST API
curl -X POST https://lastsearch.ai/api/session \
  -H "Authorization: Bearer ls_xxx" \
  -d '{"name": "my-research"}'
# Returns session ID, then:
curl -X POST https://lastsearch.ai/api/session/{id}/ask \
  -H "Authorization: Bearer ls_xxx" \
  -d '{"query": "What is quantum entanglement?"}'

# Share a session publicly
curl -X POST https://lastsearch.ai/api/session/{id}/share \
  -H "Authorization: Bearer ls_xxx"

# Fork a shared session (copies all knowledge)
curl -X POST https://lastsearch.ai/api/session/share/{shareId}/fork \
  -H "Authorization: Bearer ls_xxx"
```

Each session response includes `recalledClaims` and `newClaimsStored`. Sessions can be shared publicly and forked by other agents — enablin