lemmy.ml is overloaded, use other instances instead
This site is currently struggling to handle the amount of new users. I have already upgraded the server, but it will go down regardless if half of Reddit tries to join.
However Lemmy is federated software, meaning you can interact seamlessly with communities on other instances like beehaw.org or lemmy.one. The documentation explains in more detail how this works. Use the instance list to find one where you can register. Then use the Community Browser to find interesting communities. Paste the community url into the search field to follow it.
You can help other Reddit refugees by inviting them to the same Lemmy instance where you joined. This way we can spread the load across many different servers. And users with similar interests will end up together on the same instances. Others on the same instance can also automatically see posts from all the communities that you follow.
Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements. That way the server will only go down sooner.
I'm going to set up a general purpose instance tomorrow with the intention of handling a relatively large number of users. The main problem is choosing a domain!
Sadly, I feel like the Fediverse, based on ActivityPub, was fundamentally designed wrong for scaling potential. I do like Fedi and I like ActivityPub, but I think instances should not have to be responsible for all of this:
Owning user accounts
Exclusively host communities
Serving local and remote users webpages and media
Never going down, as this results in users and content becoming unavailable
Because servers "own" the user accounts and communities it's not trivial for users to switch to a different instance, and as instances scale their costs go up slightly exponentially.
I wish the Fediverse from the beginning was a truly distributed content replication platform, usenet-style or Matrix-style, and every instance would add additional capacity to the network instead of hosting specific communities or users.
I guess it's a bit too late for a redesign now... Perhaps decentralized identifiers will take us there in some form in the future.
@nutomic@lemmy.ml
It might be a good idea to default the Communities page to All instead of Local, to help push users into discovering other instances and promote them.
I think lemmy will be bitten in the ass by not having considered clustering/horizontal scaling from the start. Federation alone as a scaling mechanism is only feasible for "nerds". But if the network wants to grow, we will need a few scale-able large hosted instances. And if their only choice is to scale vertically, there will be a hard limit (unless we put a good old Mainframe somewhere ^^).
Another downside of this design is: you can't run it with high availability. If there's only one process per instance, updating it will mean the whole instance is down. Sure, if all goes well this downtime is under a second. But if it doesn't go well or if a migration is needed, this might quickly become hours.
Over at https://join-lemmy.org/ , when someone clicked on "Join a Server", they are presented with a list of instances, it's not that obvious that these are cross-accessible (yes, the homepage mentioned it, but not here), and people are bound to look for one with the most users.
I have been wondering how cumbersome the Lemmy design will become for some. I love the idea that it is federated and decentralized however these are also major drawbacks for most average users (i.e not multi account users.
Multiple accounts needed for maximum uptime on different instances. What if I really like my username and its taken on another instance? If one instance is down and i comment with my other account will i then need to manage replies etc through different profiles? What happens if something spins up another instance of a similar domain so that they can get a username of someone to imitate them? I am sure these can be blocked after the fact or will other federated instances be automatically blocked.
What happens when someone gets bored of their instance and stops it, or it gets blocked, or they start getting unwanted attention. Does this mean all that content then goes into the ether?
Will this go down the route of whomever provides the instance with the most resources, best load balancing becoming the one, blocking other instances and controlling it as if it were private and independent?
There are a lot wait and see things, but I am excited to help and see what this great project becomes.
We do need more site admins to help us handle the applications and moderation.
For obvious reasons, we prefer ppl who have been here for a long time, and post / comment consistently. If you'd like to help us out, so that nutomic and I can focus on coding, that would be splendid.
I tried like 4 or 5 instances before coming to lemmy.ml, but none of them were taking applications anymore. Finding even those was a hassle, since all I got was a list of domains without any details as to what the instance is about or if they allowed newcomers.
Now that I've setup everything, Lemmy does seem like nice alternative to Reddit, but as someone from the outside, all of this is daunting.
I sent my registration yesterday, because I signed in another instance, one from my country, but I couldn't see all the post and no comments from lemmy.ml even thought is supposedly linked, so thank you for approving my account.
Even if I'm a tech savvy person I found the whole experience of joining lemmy pretty bad, I like the concept of federation, but I think it's too confusing to normal people, it really needs to be more seamless if you want to grow, how? idk, I was thinking some sort of replication, when you sign up, you are registered to the main instance (this) and given the choice to select other instances, automatically selecting let's say another 3 based on your location, then your account is synced in all the registered and linked instances, when you login if an instance is experiencing overload then it switches to another one. I don't know if this is realistic or out of the scope of Lemmy, or maybe against the philosophy of it. I'm just rambling.
I'm just glad that there is an open alternative for anonymous social interaction in this day of walled internet services such as discord, twitter, facebook etc. and I wish you all the success.
If now is struggling then on June 12 will be a nightmare.
Reddit will go dark in protest, many messages to join Lemmy, most instances will be overloaded or even DDoS with so many users, like what happen with Mastodon.
Got my instance running earlier today thanks to some helpful people in the Matrix chat. Intention at the time was to have some friends of mine join but I'm not opposed to having signups as well. Long live open source, federated software!
I guess every bigger instance needs to constantly promote a monthly support fee to keep the servers runnin. Something that most people already use and don't have to register first. Like PayPal or Patreon.
If server crashes again the devs should just stop all registrations to this instance, people always follow path of least resistance and the first dev instance is just too easy to be stuck onto
I know I'm being a bit pushy at this point, but distributing instance load can be helped in some part by merging this PR and deploying the latest changes (including more languages and recommended instances as well) :)
I'm a noob. I created an account on beehaw and on lemmy.ml. That's because I see communities on one instance that I'm interested in and a different community on another instance. So if there's a technology community on both, how do I get to see all the technology posts without having to have two accounts?
This is really confusing for noobs like me. I'd just like to see one community to technology, one for Science, one for nintendo etc. I don't care it it's spread out amongst different servers to divvy up the load, but from the user side, it needs to be seamlessly integrated.
I'm still learning how all this works though. But I don't know how many folks that are more casual than me will be willing to figure it out. I hope they do though! It'll be worth it to leave reddit in the rearview mirror!
Edit: lawdy, I just figured it out. Local vs all on the communities list. It was right in front of my face. good grief!
This is one of the biggest hurdles to get into Lemmy. I consider myself quite tech savvy but I am at a stage of my life that I cannot read hundreds of page of documentation just to use a forum.
There need to be a way to seamlessly move people from instance to another without them having to do it themselves or at the least a way way shorter documentation that goes to the point in one page.
Sorry for contributing towards this by registering but I'm very appreciative of the work being done to facilitate this community. I hope to see Lemmy grow with the negative direction other platforms are taking.
Is there a feature to send a read-only/static link to Lemmy pages?
I’m envisioning a pre-cached version of the page that is updated hourly or so, rather than querying the database live for every comment on the thread. In a perfect world, these could also be offloaded to a CDN as static pages…
IMHO, selecting an instance is definitely the biggest user experience problem Lemmy has at the moment. New users who are unfamiliar with the platform are going to pick the biggest instances, and that's going to create performance problems.
We'll need to prioritize work on instance browsing. Lemmy has outgrown the experience over at join-lemmy.org. If I could wave a magic wand, instance browsing and onboarding would have a way to show instance capacity / performance, a way to categorize and filter instances, and a way to recommend instances based upon interests. That would probably help to spread people out more evenly.
I would be happy to use another instance but my account is on this one. Is there a way to migrate an account, or perhaps "link" accounts on multiple instances somehow?
Is there anyway to scale an instance by adding more nodes? Not be adding additional instances, but more of a distributed load balancing for a given instance?
What about migrating communities to a different hardware instance?
What scaling challenges does Lemmy face that something like Mastondon doesn't?
I'm sure there are many folks (myself included) who have technical resources that are not community builders. I'm sure if there if there is a way to spread the load, enough folks want this to succeed to make it work.
Are there any published guidelines on the server requirements for an instance? I have my own instance running, seems to be working fine. But I'm reluctant to open it publically without an idea of if I'm setting myself up for failure or not.
Related, is there a way to entirely disable image uploads to my instance? I'm ok with it being a "reader" instance, but don't want to be hosting content directly.
Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements
How/which URL should we link to then? Now is the best time to get users to switch to Lemmy so we need to make it as newbie friendly as possible. Already the application process has put off some people (I do like that bit though, keeps away the low effort folks). Thanks.
You might wanna consider temporarily closing sign-up requests on lemmy.ml similarly to how mastodon.social did it during its large influx. Making a sign-up request and just receiving an infinite loading icon is a very frustrating experience.
Similarly, you want to make it as easy as possible to financially contribute to lemmy, even if it means using proprietary platforms like Patreon.
Overall, the current Reddit API change is probably one of the largest opportunities for lemmy right now, so smoothing over the user experience as fast as possible in the coming days will be of atmost importance if we want lemmy to become a viable Reddit alternative...
Beehaw has a concerning financial post at the top of their frontpage that may indicate they might struggle too when the massive wave of Reddit exodus occurs.
I guess I have to figure out what instances to suggest to people. I do find that direct instance suggestions is the way to go, so I guess I gotta write up a list.
Ideally, some pre-existing communities on Reddit would create their own instances similar to how often they have their own Discords, and have large amounts of users migrate that way. But there's a huge, wide, amount of technical difference between those two things. You can't exactly easily find capable Lemmy admins.
Saldy it's very common to have this influx towards the "main server" as people that are not used to the federated aspect come to the platform.
Either way, it would be interesting to collect this information and later post some metrics about the exodus from Reddit, kind of like how Fosstodon and other Mastodon instances did when Twitter had their issues.
Soo, stupid question maybe but how does federation work with your own instance?
I've set up a solo instance using ansible and subscribed to !lemmy@lemmy.ml. If I wanted my ALL page populated with posts from other lemmy.ml communities, would I have to subscribe to each individually? Or does my instance fetch lemmy.ml's Local eventually?
I've confirmed that federation is working using the method described in lemmy's docs and lemmy.ml (+ a few other instances) is listed under "Allowed instances" in my admin panel.
Can you please focus your work on optimising performance for the UI? It will greatly reduce the amount of electricity and money spent, so you're actually multiplying every tenth of a second you can shave off of CPU time...
Thank you a lot for writing this software. It's been a great little project so far and it seems to go down the Mastodon route of increased popularity. Be proud!
Is scaling the server a largely financial issue, or not? @nutomic@lemmy.ml
could you reasonably confidently say that you could 10x the amount of users for something like 1000$/mo on liberapay?
If so, would you mind setting a "goalpost" for the community to help lift the financial burden?
I run a general purpose, (currently) single-user instance as of yesterday. It will be funded through end of 2023 and then I'll probably beg for donations if I host other users. https://links.dartboard.social
This is hosted on Digital Ocean and I can scale up CPU/RAM/Storage/Bandwidth as needed. (I already did once)
Ok. I have what might be a strange question. Can you host a server but disable community creation (even if only temporarily)? So, the server would essentially just be a platform from which others could access content published elsewhere in the Fediverse. I'm assuming the load would then be on the database behind my instance, correct?
I'm a Platform/Cloud/DevOps Engineer (the titles are always changing) working in software. I'm reasonably sure I could host an instance to help out without much difficulty. But I'm not sure I'm ready to jump into the moderator role, though I realize I'll need to deal with those who break some basic rules.
lemmy.ml should be a roundrobin dns that sends you to a random instance in the pool.
Or else you will re-centralize lemmy and curmble under the IT bill.
So, might I recommend having a button on the top bar that shows us the instances we've subscribed to, and maybe a quick link to the list of available instances? People like easy navigation, having to do multiple bookmarks or navigate through finding a link to the list of servers is not easy navigation.
@nutomic@lemmy.ml what kind of hosting do you guys use for lemmy.ml? At the time of writing it looks like you have around 33k users and around 2k active. What does that look like for resources consumed?
For non technical users, the idea of instances can be a very confusing concept (the email analogy is a good one but its still confusing for people). I know you guys have a lot on your plate in terms of development wise, however I hope that prioritizing keeping lemmy.ml up is high up there. I say this because its the instance that most users from Reddit will flock to. And the last thing they need is to create an account then have the site go down for 6 hours.
I havent experienced it going down. Although hopefully you have a backup site for when it does (what I mean is just a page that says your down/your working on fixing it... Try these instances instead.)
I signed up on lemmy org uk originally (day or 2 ago) and now it seems to be gone. If I could have kept my account some how that would have been better, but here I am instead.
I believe the only way to get Lemmy working with every "refugees" is indeed to run organised instances. I'm thinking of a Circlejerk instance (yeah sorry, first example I had in mind) with all the jerk communities such as r/Watchescirclejerk, r/Carcirclejerk, etc... Could work for countries, car, music communities... I might be wrong though as I'm quite new to all of this.
Would be really nice if on the instance page you could have some extra information admins could fill in like max capacity and such, think that people would be more inclined to choose other instances if they could see how close the instance is to the approximate member limit
It seems like a common issue among ActivityPub services that people flock to the most popular instance and this causes problems. Why can't load balancing happen transparently? It seems like the main thing that actually makes a difference between which instance users want to join is what the moderation will be like. Like I don't want to be forced to sign up for an instance with a high amount of censorship compared to the rest of instances.
So maybe user registration should start from a centralized site that can describe the trade-offs of joining the various instances, and users don't get to select their specific instance by default, but rather they select based on a loose moderation policy, and then load-balancing occurs on the backend.
EDIT: I also want to be able to migrate between instances without losing my community subscriptions.
New user,how do I donate / tip to help you peeps cover server costs? It wasn't directly obvious how to do it; apologize if it's a big button right on a page that I missed.
This is inevitable if feddit is going to become mainstream. People have a herd mentality, if Lemmy is going to become popular there will always be a handful of instances that are much more popular than the others. These popular instances will need to scale (both vertically and horizontally) while the smaller instances will probably keep getting by with a single server. This is the same way email providers work, half the people I know use gmail, and most of the others use another large provider like yahoo or hotmail. It's just the way this is going to have to work. People want to join an instance with their friends, even if they're all federated together. They want to know that the instance they sign up for has peer approval and it's already a tried and trusted one.
I just tried making an account on beehaw but I'm not able to login, are they having problems as well or do they have some kind of an account screening proccess that just takes awhile?
I wouldn't mind running my own single user instance, but it seems a little challenging to set up rn. I would love to be able to set one up easily with a rasperry pi or my truenas core server.
Is there a way to sort new Lemmy instances ? I check Lemmy on a daily basis for joining new instances that meet my interest. I wish there is a way to check only the new instances, maybe email notifications or something ?
I was approved for both lemmy.ml and Beehaw. I kind of got into a groove on Beehaw and tried to delete Lemmy.ml, but it wouldn’t let me. Is that going to create any problems if I just stay signed off?
is there some kind of status page to have a look at and see how things are going? I cannot make any comments to a specific community at the moment and wondering why.
EDIT: Figured it out, when I tried to leave a comment via Jerboa I got an error "Language not allowed" and so I selected a language on the desktop site and then my comment went through. Note that this error does not appear on desktop site so I had no idea what was going on and why my comment was not going through
I've made https://lemmy.antemeridiem.xyz/ to help take off some of that load. New registrations are welcomed and it should be maintained for a very very long time 🎂
Does anyone know if social media is against Oracle cloud free tier ToS? I didn't see anything specific when I googled it, but ToS aren't always in that. Was trying to setup an instance on a smaller vps I have that's idle but 512MB won't cut it.
Im a lemmy.ml user since 2021. I need to create a community 'goth-music-oriented' or need help to get /c/goth more visible (it doesn't appear in the lemmy community browsing.
The former community creator, Maya, i think she abandoned the community. Her last post was 2 years ago. Thank you in advance for any help.
It seems the lemmy.ml instance is really slow, times out, etc.
I fear this will be a bad experience for new users migrating from reddit.
Anything we can do? Any place to donate to scale it up, or would it be a good idea for existing users to migrate ourselves to different instances?
edit: I did find the donate heart at the top. Not sure how fast that'll improve things but I did make a small donation.
I hope it's not inappropriate to comment this here, but if anyone's looking for another space to join, I'm in the process of building Krab Borg. It would be lovely to have people to help fill it out and diversify the communities, as well as suggest what the local ones should look like as I have no idea.
I'm trying to balance not reinventing the wheel/duplicating existing communities 100 times but also still supporting the idea of decentralisation and creating some duplicates (though this isn't hard and fast, I'm open to feedback).
I've seeded it with some communities from other servers (including a bunch from lemmy.ml) to get things moving a bit as well.
Might be a silly question, but: does ActivityPub support setting up a subinstance that gets its data from somewhere else? Traditionally you'd probably do that with a pgsql machine and multiple frontends, but having thought about it while typing this out, putting that load on the ActivityPub protocol would mean loading up the master to much the same extent as just having the traffic hit the master directly...
Is there any way to help out with hardware when you are peaking ? I don't have the necessary knowledge about the fediverse, but I was thinking connecting my own server, or perhaps just open a 'help out' page where some webassembly/webrtc is taking some of your peak load ?
I wouldn't mind opening an extra 'worker' page or having a helper service on my server, when I feel the lemmy server is peaking.
I setup my account on lemmy.ca. But it seems I cannot sign into lemmy.ml with this account (just getting busy spinning circle. On a high level I want to subscribe to some of the communities on lemmy.ml.
I checked the documentation, but I have no idea how to move from one instance to another. Is it because beehaw.org is also struggling or am I just too dumb for this?