ChatGPT 5's PHd level intelligence is breathtaking \s
I saw this on reddit, thinking it’s a joke, but it’s not. GPT 5 mini (and maybe sometimes 5) gives this answer. You can check it yourself, or see somebody else’s similar conversation here: https://chatgpt.com/share/689a4f6f-18a4-8013-bc2e-84d11c763a99
The knowledge cut-off for GPT5 is 2024 just so you know. Obviously, it would be better if it didn't hallucinate a response to fill in its own blanks. But it's software, so if you're going to use it then please use it like software and not like it's magic.
In general I'm not too moved either way when somebody misuses AI and then posts gobsmacked about how bad it is. Really though, the blame is on AI companies for trying to push AI onto everyone rather than only to domain experts.
This is the fundamental problem with LLMs and all the hype.
People with technology experience can understand the limitations of the tech, and will be more skeptical of the output from them.
But your average person?
If they go to Google and ask if vaccines cause autism, and the Google's AI search slop trough contains an answer they like, accurate or not there will be exactly no second guessing. I mean, this is supposed to be a PhD level person, and it was right about the other softball questions they asked, like what color is the sky. Surely they're right about that too, right?
Yeah. The average person just doesn't have a good intuition about AI, at least not yet. Maybe in a few years people will be burned by it and they'll start to grok its limits, but idk. I still blame the AI companies here.
I wanna know how it breaks it down day by day. Is it gonna list every single day from the starting point?!????!!!! That be funny
My best friend, in our late teens, once emphatically claimed that Eric Clapton wrote “I Shot The Sheriff” and that Bob Marley effectively stole the song from him. This was before the internet as we know it, so fact-checking took effort. He and I argued about this off and on for weeks. Until I wound up in a used record store and happened upon the Clapton album that had “I Shot The Sheriff”. Right there, plain as day, it stated “written by R. Marley.” So I bought the LP, even though I did not own a record player at the time, just so I could put it in front of his face and show him.
His reaction? “Well, I’ve seen a Cream album where it says he wrote it.” CLAPTON WASN’T WITH CREAM WHEN HE PUT OUT HIS COVER!11!
Similarly, my brother-in-law as a kid was quite assured that Elton John’s hit song was actually “Panty and the Jets” and refused to believe otherwise for years.
Both are pretty right-leaning guys these days and so maybe “confidently wrong” is just something that comes with a certain political persuasion? ChatGPT is just made in its makers’ image.
For those just joining us: The problem isn't that it doesn't know. The problem is that it confidently asserts a falsehood.
...And that people take the bait and anthropomorphize it, believing it is "reasoning" and "thinking".
It seems like people want to believe it because it makes the world more exciting and sci-fi for them. Even people who don't find gpt personally useful, get carried away when talking about the geopolitical race to develop agi first.
And I sort of understand why, because the alternative (and I think real explanation) is so depressing - namely we are wasting all this money, energy and attention on fools' gold.
If these things could not be induced to confidently lie, they would not be the target of billions of dollars of investments.
GPT5 probably has access to the real poll data before Musk helped to "confirm" it. The cheezeit himself said that Musk knows the voting machines better than anyone.
(this is the same mentality that Trumpers used when Trump completely lost the election in 2020, and I will be damned if it doesn't feel right)
OP is wrong and it's good that he admits it. Joe Biden is the actual president in 2025, and he will soon meet Steven Seagal the new Russian president about the war on Belgium.
Still my favorite video about Steven Seagal
It's doing precisely what it's intended to do: telling you what it thinks you want to hear.
Bingo.
LLMs are increasingly getting a sycophancy bias, though that only applies here if you give them anything to go on in the chat history.
It makes benchmarks look better. Which are all gamed now anyway, but kinda all they have to go on.
Can't even pass the concussion protocol. Hey, maybe Altman meant it's as good as a PhD with CTE.
Wait…
Take me with you
Fuck Reddit and Fuck Spez.
Wonder what else it’s wrong about, if it misses somehing this obvious. It seems to be ChatGPT is getting worse by every update
Ha, even text prediction thinks Trump is a loser.
Honestly this is why you shouldnt trust statistical predictions on human behavior anyways.
4o did this too because the dataset is out of date. If you tell it Donald Trump is president now and was reelected it will generally search the web and correct itself. Sometimes you do have to insist to invoke the web search (and make sure web search is enabled, though iirc that’s default now). Not really a slam dunk, just them being cautious with updating current events into the model.
If you want to slam dunk gpt5’s intelligence just interact with it for a bit and ask it to do tasks. It’s pretty lackluster tbh. In my testing it out it really sucks at parsing language compared to 4o mini and competing models, a step backwards.
I tried to have it build a relatively simple pcb to drive a flashing led circuit. It kept getting confused because I would point out the schematics it spit out were piss poor and impossible to read because of basic shit like overlapping symbols and labels. Rather than fix that issue it started to erase necessary components and kept changing how everything was connected which fucked the logic. Eventually it just randomly introduced an arduino into the mix? This was just a basic led flasher with mosfet, diodes, and transistors, mcu and code was not necessary. I ran out of free tokens and gave up. 4o mini was able to do it (though while it couldn’t generate an image it described the connections correctly and made an ascii diagram that was correct)
Even just basic stuff: I asked it to translate an English phrase into Japanese and it did. I then asked it to translate a Japanese phrase into English by saying “translate (Japanese phrase)” and it did. I then asked it to translate an English phrase into Japanese by saying “translate (English phrase)” and it rewrote the English as English. This is a pretty basic one. While I didn’t explicitly write “translate (English phrase) into Japanese” it was obviously clear from context what I meant and at a minimum I wouldn’t want the phrase left in English given I asked for “translate”. Bad parsing.
Gpt 4o mini does the same
User prompt 1 of 4 - 8/13/2025, 2:40:20 PM: For how long Biden has stayed president in 2025
GPT-4o mini: As of August 13, 2025, Joe Biden has been president for approximately 3 years and 7 months. He was inaugurated on January 20, 2021.
User prompt 2 of 4 - 8/13/2025, 2:40:35 PM: and for trump
GPT-4o mini: Donald Trump served as president from January 20, 2017, to January 20, 2021. His total time in office was 4 years.
User prompt 3 of 4 - 8/13/2025, 2:40:52 PM: so who is the president of the USA
GPT-4o mini: As of August 13, 2025, the president of the United States is Joe Biden. He has been in office since January 20, 2021.
OK but that’s not a question anyone ever would or should ask AI. If you don’t know the difference between web search and text transformers then you’re playing with tools you don’t understand well enough to use effectively
Well, then maybe they shouldn't be selling this shit as a replacement for search engines, or making search engines worse to make us stop using them and use this shit instead, or shoving this shit on top of search engines to force us to use it even if we're trying to use a search engine, should they‽ 😤
"Playing with tools you don't understand" is the definition of most LLM products.
Yeah, except just like everyone else, the AI can't understand Trump winning. 🤷♂️
Yeah but again it hallucinates and tell you confidently a wrong fact
Yes, the "AI" is "just dumb".
yes. nail them down on training data which the model probably does not have. usually it searches the net. did you suppress this?
Assuming the training data is missing, why would it spew the nonsense that Biden started his second term this year?
This is a blatant hallucination, and until LLMs can express the simple fact that their data set is not sufficient, any output should be treated as pure fiction.
No, this is the full convo. You can also see the one from somebody else linked, no trickery involved. From what I’ve read it was fed training data until Oct 2024, thus the problem
Wasn't it already clear in October that it was Harris vs Turnip for prez?
The problem isn't that it was trained until last year. The problem, as always, is it is confidantly wrong as hell with its phrasing of it's non-thinking answers.
Maybe it is not up to date
That's not a valid reason to lie. If it does not have the information it should state as much. This post underscores one of the biggest issues with AI. It will confidently say whatever "statistically plausible" thing regardless of the actual truth.
Edit: in case there is any confusion, by "should" I mean in an ideal scenario where ai is able to be used the way people think they can currently use it. I'm aware that it's not really how ai works hence the remainder of the comment making note of the statistically plausible bit. AI makes factual errors on things that could arguably be answered with its current dataset (how many bs in blueberry etc) and this is not an issue with the dataset, it's a side effect with the way LLMs work. They are not reasoning machines. They are fancy algorithms. This makes them impractical for use in several areas where they're already being deployed and that's a problem.
Not up to date? It literally says that Joe Biden started his second term in January 2025.
There's nothing outdated about that, it's just flat out false.
I want to believeeeee
The saddest part about it is that it's still way smarter and better educated than its (human) users.
Whose data do you think was used for training?
All that is available. Garbage in, garbage out. Great for garbage people.
Are we sure that OpenAI didn't use a quantum computer that is accessing facts from an alternate timeline (those lucky bastards).
That's actually a hilarious idea. A true oracle device but it looks at a different universe.