Skip Navigation

archival software for Lemmy needed

Most Lemmy users are vulnerable to data loss arising out of an admin spontaneously pulling the plug on their Lemmy instance. I have lost data several times by this cause (both on Lemmy and on Mastodon). Infosec includes availability (thus backup copies), but this has been neglected by developers of clients for fedi platforms.

Mastdon has /something/, at least

We have a crutch for Mastodon: mastodon-archive by Kensenada. It only works on some¹ Mastodon instances, but when it works it’s a quite useful tool. It uses the API to grab all posts you author as well as posts by others who mention you. It would be even more useful if it would grab whole threads for which you participate or bookmark regardless of mentions, but last time I checked there is no plan to implement that. You don’t even have a copy of the parent messages you reply to. And (IIRC) you also don’t get a copy of mentions in situations where fedi barriers prevent responses from reaching the instance you are on.

¹ Some instances are simply incompatible for unkown reasons

What Lemmy needs

A gnu linux tool to fetch whole threads that the user starts as well as whole threads for which they comment. Ideal features:

  • produce a searchable local SQL database.
  • optionally, grab threads or posts the user upvotes.
  • optionally, detect cross-posts and grab those threads too.
  • periodically revisit the thread to record new activity, including moderator actions. The period between re-visits should get increasingly longer as the thread ages.
  • when an author deletes their post, it should be marked as deleted in the local DB. And users should have the option to have those records purged automatically or selectively purged upon review.
  • (science fiction?) get the current host to digitally sign something certifying that the user’s profile/content is the genuine original artifict for the purpose of migrating to another host. The current Mastodon migration mechanism is dysfunctional for cases of a host going down before migrating, and I assume Lemmy might have the same issue.
  • fedi politics circumvention: give users the option to grab copies of the same thread from other instances so a browsing tool can compare the various thread versions, suppress dupes, and show the most complete aggregated version.
  • for extra credit: integrate the DB with @theblawsybogsy@lemmy.ml’s emacs “Lem” app as a front-end for offline browsing

It’s important for retention

When a user puts a lot of effort into producing content only to lose it all on the whim of an admin deciding out of the blue to kill the server, it’s demoralising. The user might opt to abandon the fedi or start over from a giant centralised walled-garden like LW. In both cases the decentralised free world shrinks.

It’s important for digital sovereignty and fedi-balance

There are already users who conciously decide to pile onto the biggest instances for the perception of stability. Nervous Bob might have a specific passion for a small mission-focused instance like lemmy.radio, lemmybefree.net, mander.xyz or linkage.ds8.zone, but is risk-averse. He cannot stomach the thought of losing all content and believes that if an instance is large, the admins will be more careful.

Having an archive settles the nerves of Nervous Bob enough to be able to follow his passion. It disables the cognitive dissonance of licking the boots of an oppressor (such as a Cloudflare instance).


Why this is (or will be) posted in !spreadfediverse@flamewar.social

Some would say information security is essential -- a precondition to transitioning into the fedi. Reguardless, such an app would serve to encourage people to contribute to the threadiverse and ultimately the proportionate growth and spread of it.

0 comments

No comments