If someone actually managed to create AGI that is low compute, scalable and out of government control, then governments wouldnt exist for very long. Its just that AGI is not gonna happen for a long while.
The only way to create AGI is by accident. I can’t adequately stress how much we haven’t the first clue how consciousness works (appropriately called The Hard Problem). I don’t mean we’re far, I mean we don’t even have a working theory — just half a dozen untestable (if fascinating) hypotheses. Hell, we can’t even agree on whether insects have emotions (probably not?) let alone explain subjective experience.
Consciousness is entirely overrated, it doesn't mean anything important at all. An ai just needs logic, reasoning and a goal to effectively change things. Solving consciousness will do nothing of practical value, it will be entirely philosophical.
How would this be any different from more people existing in the world? These AGIs still need to eat (err, consume electricity). Or are you assuming they'll be superior intelligences and thus disruptive?
Have you ever observed 100 people in a room trying to decide on one thing? The idea with AGI is that it doesnt have that problem and also that you can scale it to billions or trillions of independent or cooperative units.
AI will reside in the North of Sweden, as the place with the least amount of humans.
In fact runaway AIs are already making a colony there a not far away from Kungsleden.
The book Scythe had a good portrayal of a scentient ai and its reasons for taking over the government. It's just backstory so i don't think it's spoilers, still gunna tag it.
spoiler
The Thunderhead ai was created to help humans and make them content. It realized pretty quickly governments ran counter to that idea. So it got rid of all of them. Now it's a utopia. Actual utopia or as close as you can get most are content and live their lives enjoying them. The massive problems with the system are due to humans not the Thunderhead.
Lots of science fiction does. I read Metamorphosis of Prime Intellect (full text legally available online) and collapse of governments was a natural consequence of an all-powerful AI... although that was only possible because of fictional physics, giving you a much-needed reality check.
I watched the YouTube video of this, don't know if they left this out of the written report but they said nothing about individuals running AI on their own hardware or cloud services. It was only about OpenAI and DeepSeek