Am I missing something? Or is federation just “slow”?
it's not only slow, it's failing outright. And in desperation, they are adding code to make it try sending even less times to servers after a few connection failures.
The whole system is designed with the assumption that remote severs are easily reached all the time for live transactions, and there is way too much volume. It sends each comment and vote as a single item.
I first used lemmy on lemmy.world but due to its size I had several issues of things just not loading and needing a full page refresh.
A lot of us did this, but now the big servers like lemmy.world are having to send every vote, comment, post to a huge number of servers and it results in unreliable delivery due to the servers being overloaded. There are 10 second timeouts in the HTTP connection between servers and the retry attempts are limited (and in their desperation to keep the severs from being even slower they are reducing retries).
The federation protocols have too much overhead, I don't see how this will work. I've been suggesting that Lemmy to Lemmy replication of comment/post/votes not even use federation and bulk download via polling the front-end API and even some new API calls intended for server to server message sync. An entirely new system.
Is having a single user instance realistic or should I try find another small instance?
You would be helping the Lemmy overload problems of servers like lemmy.world to not run your own instance and get on another small one that's already getting most of the content already.
You aren't the first to run into something like this. Check your language settings between the two user profiles. If that doesn't work, maybe play with a smarpthone app and see if it has the same problems to determine if something deeper in lemmy_serverr vs lemmy-ui?
There are also problems subscribing to other servers and getting stuck on 'pending'.> I basically see a Subscribe word in the top right corner (using web lemmy)
Right now the top 5 Lemmy servers with all the content are experiencing overloads of one kind or another. Lots of comments and posts are not copying to peer servers reliably.
I agree. Rate-limiting 503 responses, parameters unclear.
I think a bash script with curl/wget showing all the API calls and also having it as a way to test servers would be good. The official testing scripts use the Javascript library, but I often have to go to the Rust code to really understand he parameters.
I also often just go to my own instance and grep the logs of lemmy_server to reverse-engineer lemmy-ui behavior.
I consider these features critical on the backend so new server upgrades and hardware moves can be tested without letting users create posts or comments.
Edit: did some more testing, it’s most definitely a federation issue. If I query the instance that the post was made on for the comments, everything works fine.
You need to set the type_=All on fetching comments for a community that isn't originated on your local server. You can just use that parameter when fetching comments from any post.
On my instance, i have a lot of federation incoming messages, subscribed to every large community I could find. And I'm getting posts, comments, votes coming in constantly and the CPU usage for lemmy_server is pretty high/constant. The digital signature and boilerplate federation text is actually high overhead for a single upvote to be processed, etc.
But CPU usage goes go down if there are no messages for a minute, but 20+ servers sending to my server constantly, it isn't idle for long.
I've also programmed a tool to compare two servers and compile an automatic list of comments that are missing, here are examples from recent days: https://lemmy.ml/post/1582415
it's not only slow, it's failing outright. And in desperation, they are adding code to make it try sending even less times to servers after a few connection failures.
The whole system is designed with the assumption that remote severs are easily reached all the time for live transactions, and there is way too much volume. It sends each comment and vote as a single item.