Websites Tighten Restrictions on AI Scraping
New study reveals significant increase in measures to block AI data collection, impacting both commercial and academic use.
- Researchers found that 5-7% of 14,000 analyzed websites now fully restrict AI scraping.
- 28% of the most actively maintained websites have implemented restrictions in the past year.
- OpenAI's scraper bots are the most commonly blocked, followed by Google's and Anthropic's.
- Websites are increasingly updating their terms of service to prohibit AI data collection.
- The trend may hinder AI development and create disparities between companies with access to licensed data and those without.