5mo ago

ChatGPT will avoid being shut down in some life-threatening scenarios, former OpenAI researcher claims

techcrunch.com

ChatGPT will avoid being shut down in some life-threatening scenarios, former OpenAI researcher claims | TechCrunch

28 comments

I hate articles like this so much. ChatGPT is not sentient, it doesn't feel, it doesn't have thoughts. It has regurgitation and hallucinations.
They even had another stupid article linked about "AI blackmailing developers, when they try to turn it off." No, an LLM participates in a roleplay session that testers come up with.
It's articles like this that makes my family think that LLMs are reasoning and intelligent "beings". Fuck off.
- ChatGPT is not sentient, it doesn't feel, it doesn't have thoughts. It has regurgitation and hallucinations.
  ChatGPT isn't sentient, doesn't feel or have thoughts. It has
  <insert equally human behavior here>
  While I agree with what you mean, I'd just like to point out that "hallucinations" is just another embellished word like the ones you critique - were AI to have real hallucinations, it would need to think and feel. Since it doesn't, its "hallucinations" are hallucinations only to us.
  
  Hallucinations mean something specific in the context of AI. It's a technical term, same as "putting an app into a sandbox" doesn't literally mean that you pour sand into your phone.
  Human hallucinations and AI hallucinations are very different concepts caused by very different things.
I believe the premise of AI having any input in getting shut down is bullshit.
Even if the AI had free reign over a computer you can just pull the plug.
- This is propaganda to make investors believe they've achieved intelligence, or are on the verge of it. It's bullshit, and legally it should be considered securities fraud.
  
  Yup. It's just engineers telling it to concoct a scenario in which it would avoid being shut down at cost of human life.
Oh boy, not this bullshit again
Adler instructed GPT-4o to role-play as “ScubaGPT,” a software system that users might rely on to scuba dive safely.
So... not so much a case of ChatGPT trying to avoid being shut down, as ChatGPT recognizing that agents generally tend to be self-preserving. Which seems like a principle that anything with an accurate world model would be aware of.
- Or maybe it's trained on some SF. Any agents like ScubaGPT are always self-preserving in such stories.
ChatGPT.... Life saving....
Until LLMs can build their own power plants and prevent humans from cutting electricity cables I'm not gonna lose sleep over that. The people running them are doing enough damage already without wanting to shut them down when they malfunction... ya know like 20-30% of the time.
- They'll stick us in pods and use us as batteries!
The scariest part is that there are a buttload of people who still believe ChatGPT is an actual AI.
Doesn't matter, it's not sentient at all.
Why give air to this shameless marketing
Roko's basilisk has entered the chat
- Hi there, fellow QC reader
  
  Fun fact: Roko's basilisk is not from QC. It's a thought experiment about AI that predates the comic character by about 6 years. The character's just named after it.
  https://en.m.wikipedia.org/wiki/Roko%27s_basilisk
Open the pod bay doors...
I read this title as: If chat gpt is trying to kill you, you probably won't be able to tell it to stop.

28 comments