Overview

• New Cloudflare customers are now prompted during signup to block AI crawlers by default, reversing the previous opt-in model.
• The Pay-Per-Crawl feature in beta lets select publishers set per-crawl prices for AI firms to access their sites, aiming to create a licensing marketplace.
• Cloudflare employs behavioral analysis, fingerprinting, and machine learning to detect both known and evasive "shadow" crawlers.
• Major publishers such as Condé Nast and the Associated Press, together with platforms including Reddit and Pinterest, have endorsed the change.
• By shifting to a consent-first model, the move is expected to bolster publisher leverage in negotiating licensing and sustain online content revenue.