Ai scraping is an effective DDoS on the entire interent
Ai scraping is an effective DDoS on the entire interent
Excerpt from a message I just posted in a #diaspora team internal forum category. The context here is that I recently get pinged by slowness/load spikes on the diaspora* project web infrastructure (Discourse, Wiki, the project website, ...), and looking at the traffic logs makes me impressively angr...
You're viewing a single thread.
Count them as ad visits, to make big tech pay for better hardware or line?
That opens you up to getting accused of click fraud, as AdNauseam found out the hard way but its worth it if you can squeeze some cash out of them before that happens.
I mean, scraping bots would obviously obey robots.txt so those scraping - bots, i mean users can't be bots
Right? We have standards for this and the reasonable assumption is that if it doesn't respect robots.txt and otherwise looks like a user then it's a user. It can't be the responsibility of every single server admin to perfectly recognize what's a user and what's a bot run by a billion-dollar company doing a decent job pretending to be a user.