[Help] New self-hosted instance not crawling communities
[Help] New self-hosted instance not crawling communities
Just got my own instance up and running finally, but it doesn't seem to be federating as expected.
I can search for and subscribe to communities, but unless I specifically search for the URL (e.g. selfhost@lemmy.ml) nothing shows up. I'm also making this post from my instance as well (to see if anything happens...)
Is this something I just need to wait a few hours to happen or is something incorrect.
Thanks in advance
Edit: Oh, and when I do subscribe to a community, I can only see new posts. It doesn't retrieve older posts
I can only see new posts. It doesn’t retrieve older posts
This is normal. You'll only receive new posts going forward from ActivityPub. Think of it like joining a mailing list, you'll only receive new mails after subscribing. If you want to pull older post, you'll have to do it manually by pasting the post (or comment) url in the search field.
Yep you're right. And I've discovered that, in order to see posts from a community, at least one user in your instance needs to be subscribed to a community to view it.
Would be nice if someday the browse functionality provided by some sites was integrated into Lemmy but for now I had to go through and manually subscribe to a bunch of things.
I'd use a scraper but there's a lot of content I don't care to see right now, and I'm assuming subscribing comes with a small storage cost, so may as well avoid it for now
You don't get federated automatically. On your own instance you will only see posts in communities that you subscribe to. What I did was set up a "bot" user that I used to subscribe to a bunch of communities that I might find interesting when just browsing the All feed but I didn't want in my personal Subscribed feed. There's a few tools that will automatically subscribe a bot user on your instance to all the top communities but I found just manually going through https://lemmyverse.net/communities and clicking subscribe as that bot user worked for me.
I set up this: https://github.com/Fmstrat/lcs
It goes somewhat slowly but it's been pulling in communities for me.
What version are you using? There were issues federating with v 0.18.0. I updated my instance to 0.18.0-rc.10 and that resolved the issues.
Things don’t get backfilled, so until a new action happens on an old post/comment/etc they won’t show up on your instance. New things should make their way in eventually though.
Taking the link of a specific post/comment from the community instance and searching for it from your instance should populate it on your instance, just like you probably had to do to get this community to show up so you could subscribe/post at all.
There are backfill tools/scripts, but unless you really want old posts I wouldn't use them. It unnecessarily increases the load on already struggling popular/overloaded instances like lemmy.world.
For micro/personal server runners, I built a tool to automatically discover and add communities to your local instance :)
Do you know how much data is put onto the local device when subscribing to all these communities? Ballpark is it in the MBs? GBs?
Trying to determine whether discovering things I don't want is worth it
My personal instance of about:
communities | persons | posts | comments | post_likes | comment_likes -------------+---------+-------+----------+------------+--------------- 568 | 103616 | 72621 | 383006 | 1906668 | 2152368takes up about 17GB of storage thus far!