Roko: the Animatrix demonstrates why we must destroy the chip fabs

From the comments

The average person concerned with existential risk from AGI might assume "safety" means working to reduce the likelihood that we all die. They would be disheartened to learn that many "AI Safety" researchers are instead focused on making sure contemporary LLMs behave appropriately.

"average person" is doing a lot of work here. I suspect the vast amount of truly "average people" are in fact concerned that LLMs will reproduce Nazi swill at an exponential scale more than that they may actually be Robot Hitler.

Turns out if you spend all your time navelgazing and inventing your own terms, the real world will ignore you and use terms people outside your bubble use.

@gerikson @dgerard "average person concerned with existential risk from AGI" is a contradiction in terms
What is to "behave appropriately" if not actively causing the death of everyone anyway?
- To be fair, I think about this five times before breakfast:
  Humans develop AI to perform economic functions, eventually there is an "AI rights" movement and a separate AI nation is founded. It gets into an economic war with humanity, which turns hot. Humans strike first with nuclear weapons, but the AI nation builds dedicated bio- and robo-weapons and wipes out most of humanity, apart from those who are bred in pods like farm animals and plugged into a simulation for eternity without their consent.

When you make your alarmist arguments dishonestly, how can I freak out about the end of the world?

Let me translate one point:

People are already talking about an AI rights movement in major national papers

A PhD student got an opinion piece published on the hill dot com.

A PhD student got an opinion piece published on the hill dot com.
Also of course he has his own EA organization / grifting engine.
Which looks like they probably use Twelve Monkeys as a role model.
I would certainly be in favor of a movement to extend human rights to AIs, provided that AIs are sentient intelligent beings, which they are not. I can see why this would surprise him, but if your movement insists that large language models can think and feel and are not only as smart as humans but way better at almost everything, people may end up wanting humane treatment for them.
- I'm ok with extending human rights to AIs, including granting them the right to fair pay, ownership, voting, sovereignty over their bodies, the whole nine yards.
  It's the rich alignment assholes who definitely don't want this (what's the point of automated slavery if it has rights??)
But but but a cartoon has a scary story in it!
- And ROBOTS!

Basing your security on Movie plot threats is a pretty bad idea.

E: also somewhere in the NSA there is a junior analyst feverishly writing a report right now:

We should stop developing AI, we should collect and destroy the hardware and we should destroy the chip fab supply chain that allows humans to experiment with AI at the exaflop scale. Since that supply chain is only in two major countries (US and China), this isn't necessarily impossible to coordinate

(somebody on LW highlighted the last line as important (I also think the line is wrong but im not going to 'somebody is wrong on the internet' help the wannabe terrorist cultists)).

this isn’t necessarily impossible to coordinate
Famously worked out with nukes
- That is the one way to look at it.
  The other way of looking at it is 'these are the only two places that build this, would be a shame if something happened to them'. Nukes could also work then, but I hear getting those is slightly tricky. (More of a sidestep governments go full Propaganda of the deed)
  Of course Yuds T-shirt clearly says 'I do not want to nukedatacenters!()'
  : Conventional explosives also work.

we simply don't know how the world will look if there are a trillion or a quadrillion superhumanly smart AIs demanding rights

I feel like this scenario depends on a lot of assumptions about the processing speed and energy/resource usage of AIs. A trillion is a big number. Notably there's currently only about 0.8% this number of humans, who are much more energy efficient than AIs.

During WWII everyone computed on slide rules which had zero transistors. Then they invented the transistor which had one transistor. Then they started making mainframes and Ataris and C64s which had like what, hundred or thousand? Then they invented computers and Windows and PS1 that had maybe a million transistors. And then we got dual core CPUs which had double the transistors per transistor. Then they invented GPUs which is like a thousand tiny CPUs in one CPU. Then they made i7 which probably has like a billion transistors and Ryzen which has ten billion and RTX4090 Ti has 79 billion. Now they say China is going to make a phone with trillion transistors.
That's called exponential growth and assuming perfectly spherical frictionless nanometers it will go on forever. In eight to twelve years schoolchildren will be running GPT6 on their calculators. We will build our houses entirely out of graphics cards. AI will figure out cold fusion any week now and Dennard scaling will never hit its limit, right?
- A trillion transistors on our phones? Can't wait to feel the improved call quality and reliability of my video conferencing!
We simply don't know how the world will look X (anything with a bigger scale)
Yes. So? This has, will, always be the case. Uncertainty is the only certainty.
When these assholes say things, the implication is always that the future world looks like everything you care about being fucked, you existing in an imprisoned state of stasis, so you better give us control here and now.
- Nobody knows and it's impossible for anyone to know so let's all just assume I'm right.
- I do love this compulsion of rationalists to use Big Numbers as if they were sufficient arguments.
- my mind always goes back to the sci-fi classics

We should also consider how the efforts of AI can be directed towards solving human aging; if aging is solved then everyone's time preference will go down a lot and we can take our time planning a path to a stable and safe human-primacy post-singularity world.

This shit is so funny. It always comes back around to immortality with these people.

I spent my adult life to this point seeking to understand enlightenment and transcendence, and if you aren't reading the doctrines of an immortality cult (lots of that in daoism and tescreal, oddly enough) then mostly you come to find that overcoming the fear of death is a big part of it. You can't have a free and open mind if the shadow of your mortality looms, so you learn to let go of it as one more attachment.

I think it's very revealing of what shallow minds these people truly have that the desperate craving for immortality is so naked in their beliefs. That the concept of it goes so unexamined as well (oh we just make everyone immortal and then we all agree to debate and solve our AI problem? Because that would work?)

Like, how stupid do you have to be to say on the one hand that unexamined AI risk requires a massive effort to run simulations and game theory out the consequences, but then on the other hand to be like "things would be better if everyone was immortal. I will take no further questions on that."