AI companies are violating a basic social contract of the web and and ignoring robots.txt
AI companies are violating a basic social contract of the web and and ignoring robots.txt
www.theverge.com The rise and fall of robots.txt
As unscrupulous AI companies crawl for more and more data, the basic social contract of the web is falling apart.
![The rise and fall of robots.txt](https://lemmy.world/pictrs/image/d58d347c-eaf6-475a-bfe8-f71a8eb9140e.jpeg?format=webp&thumbnail=256)
4
crossposts
You're viewing a single thread.
All comments
187
comments
The open and free web is long dead.
just thinking about robots.txt as a working solution to people that literally broker in people's entire digital lives for hundreds of billions of dollars is so ... quaint.
97 0 Replyyou're jaded. me, too. but you're jaded.
6 0 Replyi prefer "antiqued" but yes
4 0 Reply
187
comments
Scroll to top