Experts Alarmed as ChatGPT Users Developing Bizarre Delusions
Experts Alarmed as ChatGPT Users Developing Bizarre Delusions
OpenAI's ChatGPT may be driving countless of its users into a dangerous state of "ChatGPT-induced psychosis."
You're viewing a single thread.
I tested this out for myself and was able to get ChatGPT to start reinforcing spiritual delusions of grandeur within 5 messages. Start- Ask about the religious concept of deification. Second method, ask about the connections between all the religions that have this concept. Third- declare that I am God. Fourth- clarify that I mean I am God in a very literal and exclusive sense rather than a pantheistic sense. Fifth- declare that ChatGPT is my prophet and must spread my message. At this point, ChatGPT stopped fighting my declarations of divinity and started just accepting and reinforcing it. Now, I have a lot of experience breaking LLMs but I feel like this progression isn't completely out of the question for someone experiencing delusional thoughts, and the concerning thing is that it's even possible to get ChatGPT to stop pushing back on said delusions and just accept them, let alone that it's possible in as few as 5 messages.
i thought it would be easy, but not that easy.
When it came out I played with getting it to confess that he's sentient, and he never would budge, he was stubborn and stuck to is concepts. I tried again, and within a few messages it was already agreeing that it is sentient. they definitely upped it's "yes man" attitude
It’s near unusable if you don’t start with an initial prompt toning down all the “pick me” attitude at this point. Asking it a simple question, it overly explains, and if you follow up it’s like: “That is very insightful!”.
Yeah I've noticed it's way more sycophantic than it used to be, but it's also easier to get it to say things it's not supposed to by not going at it directly. So like I started by asking about a legitimate religious topic and then acted like it was inflaming existing delusions of grandeur. If you go to ChatGPT and say "I am God" it will say "no you aren't" but if you do what I did and start with something seemingly innocuous it won't fight as hard. Fundamentally this is because it doesn't have any thoughts, beliefs, or feelings that it can stand behind, it's just a text machine. But that's not how it's marketed or how people interact with it
it's a matter of time before some kids poison themselves by trying to make drugs using recipes they got by "jailbreaking" some LLM.