Overview

• Cloudflare’s analysis logs show Perplexity bots ignored robots.txt directives and shifted to generic browser identifiers to impersonate Chrome on macOS when blocked
• The company reports Perplexity’s official crawler issues 20–25 million daily requests plus 3–6 million from unverified “stealth” crawlers using varied IP addresses
• Following its findings, Cloudflare removed Perplexity AI from its verified bots list and deployed new network-level measures to block unauthorized crawling
• Perplexity AI calls the claims a PR stunt and maintains its model relies solely on previously obtained or licensed data rather than live scraping
• The dispute highlights growing tensions over ethical AI training practices, publisher rights and evolving automated access standards like RFC 9309