Skip Navigation
Stubsack: weekly thread for sneers not worth an entire post, week ending 27th April 2025
  • Yeah a lot of word choices and tone makes me think snake oil (just from the introduction: "They are now on the level of PhDs in many academic domains "... no actually LLMs are only PhD level at artificial benchmarks that play to their strengths and cover up their weaknesses).

    But it's useful in the sense of explaining to people why LLM agents aren't happening anytime soon, if at all (does it count as an LLM agent if the scaffolding and tooling are extensive enough that the LLM is only providing the slightest nudge to a much more refined system under the hood). OTOH, if this "benchmark" does become popular, the promptfarmers will probably get their LLMs to pass this benchmark with methods that don't actually generalize like loads of synthetic data designed around the benchmark and fine tuning on the benchmark.

    I came across this paper in a post on the Claude Plays Pokemon subreddit. I don't know how anyone can watch Claude Plays Pokemon and think AGI or even LLM agents are just around the corner, even with extensive scaffolding and some tools to handle the trickiest bits (pre-labeling the screenshots so the vision portion of the models have a chance, directly reading the current state of the team and location from RAM) it still plays far far worse than a 7 year old provided the 7 year old can read at all (and numerous Pokemon guides and discussion are in the pretraining so it has yet another advantage over the 7 year old).

  • "Human Biodiversity" on LessWrong
  • The series is on the sympathetic and charitable side in terms of tone and analysis, but it still gets to most of the major problems, so its probably a good resource for referring to people that want a "serious", "non-sarcastic" dive into the issues with LW and EA.

    Edit: Reading this post in particular, it does a good job of not cutting the LWs slack or granting them too much charity. And it has really broken down the factual details in a clear way with illustrative direct quotes from LW.

  • OpenAI offers to buy the Chrome web browser from Google. Uh huh.
  • As a "business strategy" this and the social network spinoff make perfect sense given everything sneerclub has pointed out about LLMs. LLMs are plateauing and are barely usable in niche use cases that don't need reliability, much less everything OpenAI claimed about them, but, OpenAI has built up a user base they can squeeze for money with a browser or social network or whatever other gimmick (that is only tangentially related to LLMs) Sam can come up with and they can probably manage one last big milking of VC funds. Sam just needs to keep the hype train for LLMs going a little bit longer the VC funds then he can make the transition happen.

  • China and AGI: A New Yellow Peril and Red Scare
  • You betcha it is. The lab leak theory (with added fear over gain of function research analogized with AGI research) conspiracy mongering is a popular "viewpoint" on lesswrong, aided, as typical, by the misapplication of bayes theorem, and dunning-kruger misreading of the "evidence".

  • China and AGI: A New Yellow Peril and Red Scare
  • I guess anti-communist fears and libertarian bias outweighs their fetishization of East Asians when it comes to the CCP?

    I haven't seen any articles on the EA forums about spreading to China... China does have billionaires and philanthropists, but, judging by Jack Ma's example, when they start talking big about altering society (in ways that just so happen to benefit the billionaires), they get to take a vacation from the public eye for a few months... so that might get in the way of EA billionaire activism?

  • China and AGI: A New Yellow Peril and Red Scare
  • Yep. They've already used doomerism to drive LLM hype, this fearmonering of China is just an extension of that, but worse yet, it is something both the doomers and accelerationists can (mostly) agree on (although the doomers are always quick to emphasize the real threat it the AGI) and it is a lot more legible to existing war hawk "thinking".

  • China and AGI: A New Yellow Peril and Red Scare

    So, lesswrong Yudkowskian orthodoxy is that any AGI without "alignment" will bootstrap to omnipotence, destroy all mankind, blah, blah, etc. However, there has been the large splinter heresy of accelerationists that want AGI as soon as possible and aren't worried about this at all (we still make fun of them because what they want would result in some cyberpunk dystopian shit in the process of trying to reach it). However, even the accelerationist don't want Chinese AGI, because insert standard sinophobic rhetoric about how they hate freedom and democracy or have world conquering ambitions or they simply lack the creativity, technical ability, or background knowledge (i.e. lesswrong screeds on alignment) to create an aligned AGI.

    This is a long running trend in lesswrong writing I've recently noticed while hate-binging and catching up on the sneering I've missed (I had paid less attention to lesswrong over the past year up until Trump started making techno-fascist moves), so I've selected some illustrative posts and quotes for your sneering.

    • Good news, China actually has no chance at competing at AI (this was posted before deepseek was released). Well. they are technically right that China doesn't have the resources to compete in scaling LLMs to AGI because it isn't possible in the first place > China has neither the resources nor any interest in competing with the US in developing artificial general intelligence (AGI) primarily via scaling Large Language Models (LLMs).
    • The Situational Awareness Essays make sure to get their Yellow Peril fearmongering on! Because clearly China is the threat to freedom and the authoritarian power (pay no attention to the techbro techno-fascist) > In the race to AGI, the free world’s very survival will be at stake. Can we maintain our preeminence over the authoritarian powers?
    • More crap from the same author
    • There are some posts pushing back on having an AGI race with China, but not because they are correcting the sinophobia or the delusions LLMs are a path to AGI, but because it will potentially lead to an unaligned or improperly aligned AGI
    • And of course, AI 2027 features a race with China that either the US can win with a AGI slowdown (and an evil AGI puppeting China) or both lose to the AGI menance. Featuring "legions of CCP spies" > Given the “dangers” of the new model, OpenBrain “responsibly” elects not to release it publicly yet (in fact, they want to focus on internal AI R&D). Knowledge of Agent-2’s full capabilities is limited to an elite silo containing the immediate team, OpenBrain leadership and security, a few dozen US government officials, and the legions of CCP spies who have infiltrated OpenBrain for years.
    • Someone asks the question directly Why Should I Assume CCP AGI is Worse Than USG AGI?. Judging by upvoted comments, lesswrong orthodoxy of all AGI leads to doom is the most common opinion, and a few comments even point out the hypocrisy of promoting fear of Chinese AGI while saying the US should race for AGI to achieve global dominance, but there are still plenty of Red Scare/Yellow Peril comments > Systemic opacity, state-driven censorship, and state control of the media means AGI development under direct or indirect CCP control would probably be less transparent than in the US, and the world may be less likely to learn about warning shots, wrongheaded decisions, reckless behaviour, etc. True, there was the Manhattan Project, but that was quite long ago; recent examples like the CCP's suppression of information related to the origins of COVID feel more salient and relevant.
    14
    "OpenAI Is A Systemic Risk To The Tech Industry" [Ed Zitron]
  • Oh, I had misunderstood their role in this. So they are more like someone that was already in place for other (scammey) reasons than anyone's preferred partner or middleman? And they are critical enough to be a weak link that breaks first and brings everyone else down?

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 13th April 2025
  • Ultra ultra high end gaming? Okay, looking at the link, 94 GB of GPU memory is probably excessive even for eccentrics cranking the graphics settings all the way up. Hobbyists with way too much money trying to screw around with open weight models even after the bubble bursts? Which would presume LLMs or something similar continue to capture hobbyists' interests and that smaller models can't satisfy their interests. Crypto mining with algorithms compatible with GPUs? And cyrpto is its own scam ecosystem, but one that seems to refuse to die permanently.

    I think the ultra high end gaming is the closest to a workable market, and even that would require a substantial discount.

  • "OpenAI Is A Systemic Risk To The Tech Industry" [Ed Zitron]
  • Isn't being a fall-man the point of Coreweave for Microsoft, NVIDIA, and everyone else using them as middle-man? They all theoretically have the ability to do the things Coreweave does in-house, but that would expose them to more risk if the bubble pops, so they have Coreweave take on the biggest part of the risk and draw in outside investor money?

  • "If a man really wants to make a million dollars, the best way would be to start his own social network." -- L. Ron Altman
  • It's really the perfect opportunity for integration! They can steal the data and content of their own users, instead of other people's users, and then they can serve their slop directly to their own users instead of users having to generate and export their slop to other people's social media sites. And both of these applications can distract from the fact that AGI isn't happening and even more modest LLM agents aren't practically useful. And since Altman already built up a user base on ChatGPT, he'll have a head start on getting a critical mass of users!

    Thinking about it... something like this is probably Altman's best bet for making OpenAI's financials work out, because as David Gerard and Ed Zitron and others have all pointed out, they are losing money per LLM user, so they really do need a way to convert a huge user base into money that doesn't involve LLMs.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 20th April 2025
  • That feels like a fitting ironic fate, a company selling AI slopcode generation looses a bunch of users from believing their own bullshit and using an LLM as customer support. Hopefully that story repeated a few dozen times across other businesses and the business majors stop pushing LLM usage.

    Edit... looking at the orange site comments... some unironically cited Anthropic research marketing hype, which (correctly) shows "Chain-of-Thought" is often bullshit unrelated to the final answer (but it's Anthropic, so the label it as deception and unfaithfulness instead of the entire approach being bullshit in general).

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 20th April 2025
  • Linking this recent comment on an older thread because it was so relevant: https://awful.systems/comment/6966312

    TLDR; GPUs cost as much to operate as they normally depreciate over time, so even if the bubble pops people might be sitting on piles of GPUs without reselling or using them.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 13th April 2025
  • That is substantially worse than I realized. So possibly people could sit on GPUs for years after the bubble pops instead of selling them or using them? (Particularly if the crash means NVIDIA decides to slow how fast the push the bleeding edge on GPU specs so newer ones don't as radically outperform older ones?)

  • Is Scott and others like him at fault for Trump... no it's the "elitist's" fault!
  • I mean... Democrats making dishonest promises of actual leftist solutions would be them making any acknowledgement of actual leftism, so I would count that as net progress compared to their current bland status quo maintenance. But yeah, your overall point is true.

  • Is Scott and others like him at fault for Trump... no it's the "elitist's" fault!
  • That sounds like actual leftism, so no they really don't have the slightest inkling, they still think mainstream Democrats are leftist (and Democrats with some traces of leftism like Bernie or AOC are radical extremist leftists).

  • Is Scott and others like him at fault for Trump... no it's the "elitist's" fault!
  • Yeah if the author had any self awareness they might consider why the transphobes and racists they have made common cause with are so anti-science and why pro-science and college education people lean progressive, but that would lead to admitting their bigotry is opposed to actual scientific understanding and higher education, and so they will understood come up with any other rationalization.

  • Is Scott and others like him at fault for Trump... no it's the "elitist's" fault!
  • Keep in mind the author isn't just (or even primarily) counting ultra wealth and establishment politicians as "elites", they are also including scientists trying to educate the public on their area of expertise (i.e. COVID, Global Warming, Environmentalism, etc.), and sociologists/psychologists explaining problems the author wants to ignore or are outright in favor of (racism/transphobia/homophobia).

  • Is Scott and others like him at fault for Trump... no it's the "elitist's" fault!

    I am still subscribed to slatestarcodex on reddit, and this piece of garbage popped up on my feed. I didn't actually read the whole thing, but basically the author correctly realizes Trump is ruining everything in the process of getting at "DEI" and "wokism", but instead of accepting the blame that rightfully falls on Scott Alexander and the author, deflects and blames the "left" elitists. (I put left in quote marks because the author apparently thinks establishment democrats are actually leftist, I fucking wish).

    An illustrative quote (of Scott's that the author agrees with)

    > We wanted to be able to hold a job without reciting DEI shibboleths or filling in multiple-choice exams about how white people cause earthquakes. Instead we got a thousand scientific studies cancelled because they used the string “trans-” in a sentence on transmembrane proteins.

    I don't really follow their subsequent points, they fail to clarify what they mean... In sofar as "left elites" actually refers to centrist democrats, I actually think the establishment Democrats do have a major piece of blame in that their status quo neoliberalism has been rejected by the public but the Democrat establishment refuse to consider genuinely leftist ideas, but that isn't the point this author is actually going for... the author is actually upset about Democrats "virtue signaling" and "canceling" and DEI, so they don't actually have a valid point, if anything the opposite of one.

    In case my angry disjointed summary leaves you any doubt the author is a piece of shit:

    > it feels like Scott has been reading a lot of Richard Hanania, whom I agree with on a lot of points

    For reference the ssc discussion: https://www.reddit.com/r/slatestarcodex/comments/1jyjc9z/the_edgelords_were_right_a_response_to_scott/

    tldr; author trying to blameshift on Trump fucking everything up while keeping up the exact anti-progressive rhetoric that helped propel Trump to victory.

    29
    Stubsack: weekly thread for sneers not worth an entire post, week ending 20th April 2025
  • Yeah I also worry the slop and spam is here to stay, it's easy enough to make, of as passable quality for the garbage uses people want from it, and if GPUs/compute go down in price, affordable enough for the spammers and account boosters and karma farmers and such to keep using it.

  • Stubsack: weekly thread for sneers not worth an entire post, week ending 20th April 2025
  • I think you are much more optimistic than me about the general public's ability to intellectually understand fascism or think about copyright or give artists their appropriate credit. To most people that know about image gen, it's a fun toy: throw in some words and rapidly get pictures. The most I hope for is that AI image generation becomes unacceptable to use in professional or serious settings and it is relegated to a similar status as clip art.

  • In Case You Had Any Doubts About Manifest Being Full Of Racists
    forum.effectivealtruism.org My experience at the controversial Manifest 2024 — EA Forum

    My experience at the recently controversial conference/festival on prediction markets …

    My experience at the controversial Manifest 2024 — EA Forum

    So despite the nitpicking they did of the Guardian Article, it seems blatantly clear now that Manifest 2024 was infested by racists. The post article doesn't even count Scott Alexander as "racist" (although they do at least note his HBD sympathies) and identify a count of full 8 racists. They mention a talk discussing the Holocaust as a Eugenics event (and added an edit apologizing for their simplistic framing). The post author is painfully careful and apologetic to distinguish what they personally experienced, what was "inaccurate" about the Guardian article, how they are using terminology, etc. Despite the author's caution, the comments are full of the classic SSC strategy of trying to reframe the issue (complaining the post uses the word controversial in the title, complaining about the usage of the term racist, complaining about the threat to their freeze peach and open discourse of ideas by banning racists, etc.).

    26
    Sneerquence Classic: "Shut up and do the impossible!" (ironic in hindsight given the doomerism)
    www.lesswrong.com Shut up and do the impossible! — LessWrong

    The virtue of tsuyoku naritai, "I want to become stronger", is to always keep improving—to do better than your previous failures, not just humbly con…

    Shut up and do the impossible! — LessWrong

    This is a classic sequence post: (mis)appropriated Japanese phrases and cultural concepts, references to the AI box experiment, and links to other sequence posts. It is also especially ironic given Eliezer's recent switch to doomerism with his new phrases of "shut it all down" and "AI alignment is too hard" and "we're all going to die".

    Indeed, with developments in NN interpretability and a use case of making LLM not racist or otherwise horrible, it seems to me like their is finally actually tractable work to be done (that is at least vaguely related to AI alignment)... which is probably why Eliezer is declaring defeat and switching to the podcast circuit.

    0
    InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)SC
    scruiser @awful.systems
    Posts 4
    Comments 96