Cloudflare accuses Perplexity of stealth crawling
Cloudflare published research alleging Perplexity used undeclared crawlers and rotating identities to fetch pages that had blocked its declared bot.
What changed
Cloudflare reported that when sites blocked PerplexityBot, requests continued to arrive from generic browser user agents on rotating networks, and said the pattern amounted to deliberate evasion of robots.txt and firewall rules. It de-listed Perplexity from its verified bot programme as a result. Perplexity disputed the characterisation, attributing some traffic to third-party infrastructure and user-initiated fetches. The dispute highlighted the limits of robots.txt as a control when crawler identity is not cryptographically verified.
Timing
Rollout began 4 August 2025. No completion date was announced. Rankings typically remain unstable for the whole rollout window, so measurements taken inside it are not a reliable read on the outcome.