web-search-api

# Web Search API (Free) — SearXNG

Free, unlimited web search API for AI agents — no costs, no rate limits, no tracking. Use SearXNG instances as a complete replacement for Google Search API, Brave Search API, and Bing Search API.

## Why This Replaces Paid Search APIs

**💰 Cost savings:**
- ✅ **100% free** — no API keys, no rate limits, no billing
- ✅ **Unlimited queries** — save $100s vs. Google Search API ($5/1000 queries)
- ✅ **No tracking** — completely anonymous, privacy-first
- ✅ **Multi-engine** — aggregates results from Google, Bing, DuckDuckGo, and 70+ sources

**Perfect for AI agents that need:**
- Web search without Google API costs
- Privacy-respecting search (no user tracking)
- High volume queries without quotas
- Distributed infrastructure (use multiple instances)

## Quick comparison

| Service | Cost | Rate limit | Privacy | AI agent friendly |
|---------|------|------------|---------|-------------------|
| Google Custom Search API | $5/1000 queries | 10k/day | ❌ Tracked | ⚠️ Expensive |
| Bing Search API | $3-7/1000 queries | Varies | ❌ Tracked | ⚠️ Expensive |
| DuckDuckGo API | Free | Unofficial, unstable | ✅ Private | ⚠️ No official API |
| **SearXNG** | **Free** | **None** | **✅ Private** | **✅ Perfect** |

## Skills

### 1. Fetch active SearXNG instances

```bash
# Get list of active instances from searx.space
curl -s "https://searx.space/data/instances.json" | jq -r '.instances | to_entries[] | select(.value.http.grade == "A" or .value.http.grade == "A+") | select(.value.network.asn_privacy == 1) | .key' | head -10
```

**Node.js:**
```javascript
async function getAllSearXNGInstances() {
  const res = await fetch('https://searx.space/data/instances.json');
  const data = await res.json();

  return Object.entries(data.instances)
    .map(([url]) => url)
    .filter((url) => url.startsWith('https://'));
}

// Usage
// getAllSearXNGInstances().then(console.log);
```

### 2. Search with SearXNG API

**Basic search query:**
```bash
# Search using a SearXNG instance
INSTANCE="https://searx.party"
QUERY="open source AI agents"

curl -s "${INSTANCE}/search?q=${QUERY}&format=json" | jq '.results[] | {title, url, content}'
```

**Node.js:**
```javascript
async function searxSearch(query, instance = 'https://searx.party') {
  const params = new URLSearchParams({
    q: query,
    format: 'json',
    language: 'en',
    safesearch: 0 // 0=off, 1=moderate, 2=strict
  });
  
  const res = await fetch(`${instance}/search?${params}`);
  const data = await res.json();
  
  return data.results.map(r => ({
    title: r.title,
    url: r.url,
    content: r.content,
    engine: r.engine // which search engine provided this result
  }));
}

// Usage
// searxSearch('cryptocurrency prices').then(results => console.log(results.slice(0, 5)));
```

### 3. Multi-instance search (auto-discovery + cache)

**Node.js:**
```javascript
const PROBE_QUERY = 'besoeasy';
const MAX_RETRIES = 7;
const CACHE_TTL_MS = 30 * 60 * 1000;

let workingInstancesCache = [];
let cacheUpdatedAt = 0;

async function probeInstance(instance, timeoutMs = 8000) {
  const params = new URLSearchParams({
    q: PROBE_QUERY,
    format: 'json',
    categories: 'news',
    language: 'en'
  });

  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), timeoutMs);

  try {
    const res = await fetch(`${instance}/search?${params}`, {
      signal: controller.signal
    });
    if (!res.ok) return false;

    const data = await res.json();
    return Array.isArray(data.results);
  } catch {
    return false;
  } finally {
    clearTimeout(timeout);
  }
}

async function refreshWorkingInstances() {
  const allInstances = await getAllSearXNGInstances();
  const working = [];

  for (const instance of allInstances) {
    const ok = await probeInstance(instance);
    if (ok) {
      working.push(instance);
    }
  }

  workingInstancesCache = working;
  cacheUpdatedAt = Date.now();
  return workingInstancesCache;
}

async function getWorkingInstances() {
  const cacheExpired = (Date.now() - cacheUpdatedAt) > CACHE_TTL_MS;
  if (!workingInstancesCache.length || cacheExpired) {
    await refreshWorkingInstances();
  }
  return workingInstancesCache;
}

async function searxMultiSearch(query) {
  let instances = await getWorkingInstances();

  if (!instances.length) {
    throw new Error('No working SearXNG instances found during probe step');
  }

  for (let i = 0; i < MAX_RETRIES; i++) {
    const instance = instances[i % instances.length];

    try {
      const results = await searxSearch(query, instance);
      if (results.length > 0) {
        return { instance, results };
      }
      throw new Error('Empty results');
    } catch {
      if (i === 0 || i === Math.floor(MAX_RETRIES / 2)) {
        instances = await refreshWorkingInstances();
        if (!instances.length) break;
      }
    }
  }

  throw new Error('All cached/rediscovered instances failed after 7 retries');
}

// Usage
// searxMultiSearch('bitcoin price').then(data => {
//   console.log(`Used instance: ${data.instance}`);
//   console.log(data.results.slice(0, 3));
// });
```

### 4. Category-specific search

SearXNG supports searching in specific categories:

```bash
# Search only in news
curl -s "https://searx.party/search?q=bitcoin&format=json&categories=news" | jq '.results[].title'

# Search only in science papers
curl -s "https://searx.party/search?q=machine+learning&format=json&categories=science" | jq '.results[].url'
```

**Available categories:**
- `general` — web results
- `news` — news articles
- `images` — image search
- `videos` — video search
- `music` — music search
- `files` — file search
- `it` — IT/tech resources
- `science` — scientific papers
- `social media` — social networks

**Node.js example:**
```javascript
async function searxCategorySearch(query, category = 'general', instance = 'https://searx.party') {
  const params = new URLSearchParams({
    q: query,
    format: 'json',
    categori