Claude web search and Claude-SearchBot
Anthropic added live web search to Claude with cited sources, alongside separate crawler identities for search and user-initiated fetches.
What changed
Anthropic enabled Claude to retrieve current web content and cite the pages it used, first for paid users in the United States and then more broadly. Anthropic documents three distinct user agents: ClaudeBot for gathering training content, Claude-User for fetches triggered by a user's question, and Claude-SearchBot for improving search result quality. Publishers can allow or disallow each independently in robots.txt, which lets a site appear in Claude's cited answers while refusing training use.
Timing
Rollout began 20 March 2025. No completion date was announced. Rankings typically remain unstable for the whole rollout window, so measurements taken inside it are not a reliable read on the outcome.