Cloudflare has introduced a one-click solution to block AI bots from scraping website content. This move is in response to the growing concern among customers about AI bots using their data without permission to train machine learning models.
The new feature allows web hosting customers to block all AI bots with a single click. The feature is available to all customers, including those on the free tier.
The issue of AI bots scraping content from websites has been a contentious one. While some AI companies clearly identify their web scraping bots, not all are transparent about their activities. This has led to instances where content is used without suitable credit or permission.
Cloudflare’s new feature aims to provide a more robust barrier to bot entry. The company has observed that the internet is now flooded with these AI bots, which visit about 39 percent of the top one million web properties served by Cloudflare.
The introduction of this feature follows Cloudflare’s announcement last year of a method to block rule-respecting AI bots. Despite these bots following the rules, Cloudflare found that customers overwhelmingly opted to block them.
Read more: www.theregister.com