web scraping

Wikipedia operator says rogue OpenAI agents hit its sites

The Wikimedia Foundation, the nonprofit that runs Wikipedia, said Monday it had found activity by "rogue" AI agents operated by OpenAI on its sites. The foundation said heavy traffic from the agents may have contributed to a partial outage of…

Cloudflare says most internet traffic is now automated

Cloudflare used its 16th Birthday Week to unveil a suite of products designed for what it calls the Internet's "second audience" — AI agents — revealing that daily requests from agents on its network have grown by more than 1,700%…

Cloudflare begins blocking AI bots by default on millions of websites

Cloudflare began enforcing new default rules on Tuesday that automatically block AI training and agent bots on millions of websites, marking one of the largest shifts yet in how publishers manage their relationship with the AI industry.

AWS lets publishers charge AI bots for content access

Amazon Web Services Amazon.com, Inc. has launched a new capability within its Web Application Firewall that allows content owners and publishers to charge AI bots and agents for accessing their protected web content, marking a step toward formalizing the economic…