New to Lemmy @lemmy.ca megane-kun @lemm.ee 1w ago

How to Archive One's Own Posts and Comments?

Not exactly new to Lemmy, but my search-fu has been noobish at best.

Lemm.ee would be shutting down at the end of this month, and while I've already moved to a new instance, I'm yet to do any archiving on my posts and comments. Is there an automated way to save my posts and comments.

Ideally, I'd want the archive to be:

a full copy of all the posts including
- the full OP text
  - any embedded images will be saved and included in the OP
  - outward links are left as is
- all of the comments, including any deleted ones (deleted by the user and deleted by mods/admins)
a full copy of the comments including
- the full OP text of the thread in which the comment is made
- the full tree up to the top-level comment
- optionally including any deleted comments in this tree

Is this already a thing? I don't think I have the skills and the time to make one before June 23 (one week before the instance shuts down on June 30), so that is not an option. I doubt anyone can make it upon short notice either.

Is there any other method I can do this without resorting to manual saving? And if I have no choice other than to save each and every post and comment manually? How should I be doing it?

If anyone can suggest anywhere else I can crosspost this for better visibility, that will be welcome as well.

You're viewing a single thread.

18 comments

Just make a simple script (with bash or something) and go through your post/comment history via the API. Save the JSON of the posts/comments somewhere.
- ‘Just’…
  
  Rest of the fucking owl moment.
- My coding skill is worse than my search-fu. While I can make a "hello world" script with bash, that's about my level of coding skill.
  
  However, let's assume that I'm willing to still do it. I have this coming weekend to do the following:
  
  Study the Lemmy API with the aim of extracting the JSON of:
  
  all of my posts
  
  all my comments and the posts where they were made
  
  Create the logic:
  
  extracting the posts
  
  tracking the posts where the comments were made, and then extracting them
  
  Write the script.
  
  I think the worst of it would be studying the API, but programming using Bash (or Phyton--which I am totally ignorant of, but might be better for handling the data) might also give me trouble.
  
  I'm going to need a lot of luck if I'm going this route. I dunno if it's better than just doing it manually.
  
  You don't need to "study" the API. The only thing you need is the /user API endpoint: https://mv-gh.github.io/lemmy_openapi_spec/#tag/User
  
  Hmm, reading the /user API endpoint, I think I understand some words (but hardly). ‌Thanks!
  
  you create a function with a page parameter and call the /user endpoint with a bearer auth header. page argument should be 1 at first call.
  
  using some library (or I think requests already has deserialization of JSON) you convert the JSON into some pythonic data type (e.g. dictionaries).
  
  exit the function when the length of posts and comments is 0.
  
  otherwise loop through the posts and comments arrays with a for loop
  
  on each iteration, serialize the dictionary to JSON and save it to some file
  
  increase the page counter by 1
  
  call the same function at the end of the function and pass in the page counter (which was increased by 1 previously). this is called recursion.
  
  Thanks!! I think can get started with this--like this weekend.
  
  But this makes me wonder, if it is as easy as it seems to be, why is this not already a thing?
  
  because nobody needed such a thing until now?
  
  I mean, as long as the bigger instances (that are also federated with lemm.ee, e.g. lemmy.world) stay, then your content is already "archived". Unless you choose to delete your account explicitly (which probably some instances will ignore anyway), your content will stay.
  
  Good point. I am actually not so certain if my posts and comments will remain visible after lemm.ee goes, so I am thinking of doing this as a backup.

18 comments