Overview

• Cloudflare said it removed Perplexity from its verified bots program after detecting undeclared agents fetching content despite explicit no-crawl rules.
• The security firm implemented new managed-rule heuristics to catch bots rotating user-agent strings and IP addresses, including impersonating Chrome on macOS.
• Researchers reported that these stealth crawlers operated across tens of thousands of domains and generated millions of daily requests.
• Cloudflare has promoted its free anti-scraping tools and a pay-per-crawl marketplace to help publishers charge or block AI scrapers.
• Perplexity disputed the findings as misattribution, labeled the report a “sales pitch,” and denied that its bots accessed blocked content.