2y ago

Just found out that DuckDuckGo can be used to search Fediverse sites easily.

If you didn't know, on DuckDuckGo you can search for posts or magazines with site:kbin.social just like you would with Reddit. The same applies to any other Fediverse site like Kbin, Lemmy, Mastodon, etc.. I was really frustrated because it seems like Google was intentionally suppressing them.

url

21 comments

Hey thanks for the tip! I also felt like fediverse was being suppressed by google, but wasn't sure whether thats the case or does google just need time to properly index these sites.
- I think that's part of it, but Lemmy has been around for as long as Tildes so it doesn't really add up. Tbh this was my main issue so now I don't have a good reason to go back to Reddit ever.
It the other way around: kbin.social explicitly does not allow indexing by search engines. If DuckDuckGo ignores this default, the fault is basically not with Google.
- Why?
  
  Because it puts a considerable strain on Kbin’s servers, because there is a lot of content to crawl.
  I suspect (hope, even) that if Kbin runs on beefier hardware some time in the future, crawling can be enabled. However, since Kbin is completely free and devoid of ads, we would probably need to start a donation drive to make that happen.
So in order every single Fediverse instances, I'd have to include all of them? It'll be more fun if something that can exist to globally search every instances (and give us filter to opt out of some that the user do not enjoy) instead of making it a long long string like site:kbin.social + site:lemmy.world... Like kind of how greasyfork is doing with their search engine, I like it.
- Cool! I added this URL as a bookmark: https://www.google.com/search?q=(site%3Akbin.social+OR+site%3Alemmy.world+OR+site%3Ash.itjust.works+OR+site%3Abeehaw.org+OR+site%3Alemmy.ml+OR+site%3Alemmy.ca+OR+site%3Amidwest.social+OR+site%3Alemmy.blahaj.zone).
  Now I can click it and add anything in the front of the search to find things across the entire reddit-like fediverse. I tried the same with DuckDuckGo and it didn't really work? Might need different formatting.
@slicedcheesegremlin
Mastodon users have the option to opt-out of search engines indexing by adding tags to their post hat are readable by web crawlers.
So you might not find everything from mastodon, but for a reddit-like it makes sense to be indexed.
- Assuming crawlers actually respect it. I've had to IP ban some crawlers because they didn't.
  
  @Trebach @slicedcheesegremlin
  Yeah....
- That makes sense, thanks.
Since nobody has mentioned them in this thread, you should know that there are search sites for Lemmy communities.
https://lemmyverse.net/communities
https://browse.feddit.de/

21 comments