1 repositories in the Claude AI ecosystem, indexed and ranked by ClaudeWave.
polite-crawler: a polite web crawler for research and AI agents. Fetch once, keep it, read it many times. robots.txt, one request per site, full-text search, MCP server.