OpenAI, Meta, Google, and others are positioning AI agents as personal assistants that can perform a range of tasks online. But the web is becoming increasingly unfriendly toward agents, undermining their usefulness.
OpenAI, Meta, Google, and others are positioning AI agents as personal assistants that can perform a range of tasks online. But the web is becoming increasingly unfriendly toward agents, undermining their usefulness.
The ones that are difficult to block use fake useragents, however there are WAFs that keep track of bots that do aggressive crawling and block those. Usually, that, combined with rate limiting rules, blocks most of them.
You are correct though, it cannot be completely prevented unless you restrict it to users and disallow user creation (like private torrent trackers do).