Cloudflare Is Blocking AI Crawlers by Default
Cloudflare Is Blocking AI Crawlers by Default
www.wired.com
Cloudflare Is Blocking AI Crawlers by Default
5 comments
-
-
-
Let me introduce you to https://commoncrawl.org/
-
Very cool, thank you for sharing
-
-
That would present quite the attack vector. Any crawler could then subtly alter information to create disinformation
-
-
To sell access afterwards?
Good