The main reason why access to genAI is often free is that the corpos are desperate for real world testing data at best, and are collecting use data at worst, and at the very worst they're illegally collecting the use data.
This is an obvious downside of LLM glazing and sycophancy. (I know OpenAI claim they've rolled back the "dangerously sycophantic" model update, but it's still pretty bad.)
If you're already prone to delusions and conspiracy theories, and decide to confide in ChatGPT, the last thing you need to hear is, "Yes! Linda, you've grasped something there that not many people realise—it's a complex idea, but you've really cut through to its core! 🙌 Honestly, I'm mind-blown—you're thinking things through on a whole new level! If you'd like some help putting your crazed plan into action, just say the word! I'm here and ready to do my thing!"
I tested this out for myself and was able to get ChatGPT to start reinforcing spiritual delusions of grandeur within 5 messages. Start- Ask about the religious concept of deification. Second method, ask about the connections between all the religions that have this concept. Third- declare that I am God. Fourth- clarify that I mean I am God in a very literal and exclusive sense rather than a pantheistic sense. Fifth- declare that ChatGPT is my prophet and must spread my message. At this point, ChatGPT stopped fighting my declarations of divinity and started just accepting and reinforcing it. Now, I have a lot of experience breaking LLMs but I feel like this progression isn't completely out of the question for someone experiencing delusional thoughts, and the concerning thing is that it's even possible to get ChatGPT to stop pushing back on said delusions and just accept them, let alone that it's possible in as few as 5 messages.
When it came out I played with getting it to confess that he's sentient, and he never would budge, he was stubborn and stuck to is concepts. I tried again, and within a few messages it was already agreeing that it is sentient. they definitely upped it's "yes man" attitude
Yeah I've noticed it's way more sycophantic than it used to be, but it's also easier to get it to say things it's not supposed to by not going at it directly. So like I started by asking about a legitimate religious topic and then acted like it was inflaming existing delusions of grandeur. If you go to ChatGPT and say "I am God" it will say "no you aren't" but if you do what I did and start with something seemingly innocuous it won't fight as hard. Fundamentally this is because it doesn't have any thoughts, beliefs, or feelings that it can stand behind, it's just a text machine. But that's not how it's marketed or how people interact with it
It’s near unusable if you don’t start with an initial prompt toning down all the “pick me” attitude at this point. Asking it a simple question, it overly explains, and if you follow up it’s like: “That is very insightful!”.
edit: I don't want to sound cynical. But people have been looking for meaning in meaningless shit forever, and they love most the illusion of interacting (ouija bord). My favorite I heard in a radio show many years back, about people who'd listen to radio noise until they heard something in it, which they claimed were messages from $deity or departed ones etc. They played back some recordings on loop, and if you knew what you were listening for, you could hear it, too.
Monkeys on typewriters, have they written Shakespeare's works yet?
You mean "AI" built to be sycophantic to whomever it's interacting with is feeding into and reinforcing insane bullshit that would (rightfully so) be ridiculed by actual humans?