Skip Navigation

User banner
Posts
127
Comments
598
Joined
2 yr. ago

  • Also, my magazine name for this is now obsolete!

    I thought it was a great reference, but I doubt I would have ever found this community if it were not for Lemmy testing and looking in odd places ;

  • The episode was fine to me. Writing is hard, I don't expect everything to be 'best of' material. I thought the meta was fine, the show is about being a corporate business run by a mad scientist who doesn't care about his employees... yet always comes back to them. Fry getting out of the suit was decent, fit with the Calculon style melodrama. With such a legacy, it's like new seasons of The Simpsons, how would you write it?

    I thought the voices were fine. Marge SImpson, Julie Kavner, her voice has changed a lot since the start as voice actors I follow for a long time go... and I always notice the change in her voice... but it just seems real to me... I'm over age 50, and I know people whose voice has changed a lot from age 30 to 50.

  • I suggest trying to unsubscribe from each and every community that is delivering to you and figure out why you are still getting incoming data.

  • There are bugs in Lemmy right now about remote-home-remote instances, moderator actions seem to be one of the most common, as are deletes of comments (but edit of comments works fine). If the community is homed at lemmy.ml, and a change is made on an instance other than that, the code doesn't federate it to the other 1500 instances who might have subscribed to that community. There are some open issues on GitHub going back weeks about some of the issues. I gave a heads up on some of the issues on !mods@lemmy.world

    I am actively working on a series of tests to demonstrate these problems. Right now I'm waiting on more critical performance server-crash issues to get merged.

  • Speaking of language overhead, pg_stat_statements shows this particular SELECT statement returning almost 2000 rows per execution (if I'm interpreting the row count correctly).

    EDIT, NEVERMIND: if I actually divided in my head, it is 184 rows per query, which I think is the default number of languages installed on a lemmy instance. so 184 rows isn't the wild 2000 number I thoughtlessly estimated.

    The statement has been executed 1313 times, that number is far lower than the 3000 comment INSERT statements, so it isn't per comment. I have 803 + 703 INSERT into the post table, so that doesn't match.

    Oh, ok, 186 times a similar SELECT is being run that has a high number of rows returned per execution:

    1313 + 186 = 1499. 803 + 703 = 1506 post inserts (there may be more variations of each statement). It could be once per post INSERT?

    I suspect these two SQL statements are supposed to have a WHERE site_id clause on them. Can someone help identify in the Rust code why incoming federation of comments/posts is running this query over and over, but not per-comment or per-post, but something less?

  • FYI, some major PostgreSQL optimization has been found with the stored procedures: https://github.com/LemmyNet/lemmy/pull/3704

    This should really help with Beehaw and other sites that have local users creating posts and comments, as the runaway site_aggregates queries were creating unnecessary writes.

  • Update July 24, 2023

    Some major stored procedure SQL problems were overlooked until 2 days ago. I submitted revised statements to fix a massive write operation on every single post and comment creation on a local site. The site_aggregates table... every row modified instead of a single row. https://github.com/LemmyNet/lemmy/pull/3704

    I was curious why pg_stat_statements didn't draw more attention to the INSERT statements hitting so many rows, and I found out that by default it does not take into account stored procedure execution! https://stackoverflow.com/questions/56741860/pg-stat-activity-how-to-see-current-activity-inside-a-running-stored-procedure

     
        
    pg_stat_statements.track = all
    
    
      

    Now I'm seeing far more activity than I have been looking at for the past 2 months... the stored procedure statements are showing up!

    Install steps: https://gist.github.com/rcanepa/535163dc249539912c25

  • It's all a matter of how many you think 'a lot' is. The major Lemmy servers that most small instances are drawing comments and posts from have had a series of major performance problems for the past 60 days. A fix is sitting on GitHub right now for one such serious problem.

    https://lemmy.world/u/Ricey

    lemmy.world has had serious problems recently. The 10-second timeout in the http connect doesn't give much time for it to respond.

    I think it helps to get some idea of quantity of incoming comments and posts that you are successfully sending and receiving to other instances. Right now the federation protocol favors giving up quickly (10 seconds) and retries... and doesn't aim for guaranteed delivery.

  • In this one case, it was hand-written SQL inside a PostgreSQL FUNCTION that the ORM knows nothing about. But there is a approach in the entire application to have live-data from PostgreSQL for every little thing.

  • Major performance problems within the code, related to PostgreSQL, have been there for years behind the surface.... and developers are just starting to find major mistakes. We have a big change we are working on right now that is relatively easy and should have immediate improvement.

  • We have had DBA's, the problem is the Rust code uses ORM and an auto JSON framework that makes tracing the code time-consuming to learn.

  • It’s not on every comment,

    My testing with latest code is that it is indeed on every single comment INSERT, every new comment. I have the ability to view my live data while using Lemmy: https://lemmyadmin.bulletintree.com/query/raw_site_aggregates?output=table

    Every one of the 1486 rows on that table gets +1 on comment when I post a new comment on my instance.

    it’s mostly triggered on deletions and edits

    That is not correct. Edits do not change the count of comments column on site_aggregates - because the number isn't changing. Deletes (of a comment or post) in Lemmy are also not SQL DELETE statements, they are just a delete data column in the table. That DELETE PostgreSQL trigger only gets run when a end-user cancels their Lemmy account in their profile.

  • I imagine they have full offsite backups they can pull data from. That's why we have documentary shows like Mr. Robot, never forget 5/9!

  • If there isn't any post they left to act on, it's maybe an oversight in the lemmy-ui design. Should open a bug on lemmy-ui that there should be an option on the moderator listing on the community page. Perhaps some alternate to lemmy-ui has the option, there are lot of new apps out there. A small JavaScript API client could probably remove it with a dozen lines of code without having to resort to resort to going directly to PostgreSQL.

  • I linked your post on !lemmy_helper@lemmy.ml - as I started out building a list of direct PostgreSQL queries with that webapp last month. Thank you for sharing.

  • I now see that the stated intent is to retain anonymized comments as Reddit does

    You cite a conversation from November 4, 2022.

    A Febuary 6, 2023 comment from developers https://github.com/LemmyNet/lemmy/issues/2624#issuecomment-1419559651 says "The correct way privacy-wise, is to delete your account through your profile settings, which overwrites all your content to remove any info."

    The intention is to routinely purge all content of users who delete their account. In fact, there are open GitHub issues about serious performance problems in executing this code, even with a relatively small amount of content. Developers in the past 2 days have commented on this and made no mention of intention to retain content.

  • This is unfortunately only possible if you still own the original domain. Think about it this way: if you could migrate domains without proving you own the original, then what’s stopping a bad actor from migrating any domain they want?

    I'm suggesting a whitelist, that each peer has to put in a substitute list of vlemmy.ml==vlemmy.ml to re-federate.

    Much like Reddit, comments continue to exist even when the author deletes their account.

    That is NOT how the testing code of lemmy_server tests things, nor how the GitHub front page advertises Lemmy.