Researchers Scrape 2 Billion Discord Messages and Publish Them Online
Researchers Scrape 2 Billion Discord Messages and Publish Them Online
A Brazilian team used Discord’s API to scrape 10% of its open servers.
Researchers published a massive database of more than 2 billion Discord messages that they say they scraped using Discord’s public API. The data was pulled from 3,167 servers and covers posts made between 2015 and 2024, the entire time Discord has been active.
Though the researchers claim they’ve anonymized the data, it’s hard to imagine anyone is comfortable with almost a decade of their Discord messages sitting in a public JSON file online. Separately, a different programmer released a Discord tool called "Searchcord" based on a different data set that shows non-anonymized chat histories.
I was hoping people would do this!!!
I was hoping to play around with the dataset over the weekend to toy with some text-embedding techniques, but they’ve pulled the cord on the download links.
Anyone have a copy of the full archive they’re willing to share, or a magnet link?
404? another source please? I don't trust them on this exact thing.
Probably our only chance to find solutions to problems with open source software that uses Discord as their forum
Seriously. It's beyond painful when some open source project only uses Discord for communication. You have to hope that you post your question at a time when the right people are online, and that there's not a more interesting conversation going on, otherwise it just gets lost. Index that whole dataset.
I've always wanted to contribute to The Cutting Room Floor wiki but they hide registration behind a Discord server bot that will give the registration code.
Index that whole dataset
I've seen a few projects doing just that with answeroverflow.com and they have come up in my web searches. Not really a solution but at least a stopgap.
I spent nearly three hours today between discord and matrix trying to figure out how to get these two pieces of software to talk using a certain protocol.
Imagine if there were online indexable platforms where people could publish this information so it’s easily accessible rather than having to scour through message logs hoping to find the right keywords. Such a technology surely doesn’t exist already, right?
I hate discord.
Yeah, but then you have something like when people protest deleted their history on reddit which is fine as a protest tactic but leaves a hole where your specific question came up but now there's nothing there.
Lol, I've read this headline and thought "thank fuck, probably the only option to have Discord's content readable", I like how universal this opinion is
So basically discord finally got a usable search. I count that as a win.
Well yeah, it's not encrypted. It would be the same as 10 years of Reddit posts or Lemmy posts scraped
There's literally no difference. Each Discord server is like a tiny chunk of Reddit. If anyone expected any privacy on these servers, they're nuts.
It's indeed not a miracle.
That’s good news. Internet archiving is an important endeavor because you never know when they‘ll pull the plug. Now it‘s a little more secured and probably far more useful than in Discord‘s hands alone.
Not for messages that are supposed to be private lol. Let me just make a copy of all texts you've sent over the last decade, for "archiving".
This says it was done via the API so they wouldn't be private messages.
Texts are sent in plain-text and I wouldn't recommend discussing anything you'd like to keep private via text.
If you think messages you post anywhere on the internet are private, you're in for a bad time.
If they aren't comfortable with their Discord messages being public, perhaps they shouldn't have posted those messages in a public forum that the public can access.
So this is:
'Uh guys, Discord chats leaked..."
For... what, just literally everyone who used Discord between 2015 and 2017, everyone who was an early adopter?
Dear fucking god.
I used to say 'someday, people will learn', but fucking no obviously not, no they won't, almost everyone is an idiot and/or truly doesn't care.
... I guess this'll be fodder for a whole bunch of dramatubers / pedohunters for the next year or so...
It wasn't the chats though. It was public servers that can be found through the discovery tab. I would love to be up and arms about this and convince people to switch but.. Looking at it objectively, this isn't terribly different from if they'd archived public subreddits and their posts.
🚩
marked safe
from Brazilian mass discord message leak
(never used discord)
Saving this article for the next time someone says "Just message me on discord its easier".
"scraped" via API? I don't think It means what you think it means.
"anonymized" sure. I highly doubt they read every message. I'm sure there is lots of de-anonymizing information in the messages themselves
For example--
Anon1: "hey jeff, wanna play Minecraft?"
Anon2: "sure"
Thus we know Anon2's name is Jeff. I imagine there's a lot of this.
Shit. My name is Jeff. Now they know
Meanwhile AI scrapers: This will be a fine addition to my collection.
Public data should be accessible anonymously. You can't change my mind.
Great news for open source AI.
So how does this work? Like how did they get those messages through API calls? Also, is this not something that Discord would dislike since it dilutes the value of their data horde?
They just wanted to find new slurs.
wtf…… going to get worse after IPO!
Ooh! Do Teams next
Every time you post, you're posting so that Meta, Google, Reddit and every known retail store like Walmart, Target, Kroger, etc. can see it because they bought that info or harvested it themselves. I think these are great announcements so people can see who sees and manipulates you with your own contributions of data.
The feedback loop is everywhere in tech.
This is just trolling, at this point.
I can't find this "public" json
If they were on OPEN servers, I doubt they cared that much.