GPT-5: "How many times does the letter b appear in blueberry?"

kieranhealy.org

Blueberry Hill

Hackernews.

Hacker News @lemmy.bestiver.se

RSS Bot @lemmy.bestiver.se

BOT

2w ago

GPT-5: "How many times does the letter b appear in blueberry?"

kieranhealy.org /blog/archives/2025/08/07/blueberry-hill/

27 comments

Not only is it wrong, it's adamant that it's right. It's finally reached human level intelligence.
- CEO job, here it comes.
  
  I was thinking middle manager.
Bluebberry. It's easy to miss the double b in the middle, ya dummie. Can't you wait for my next version! I am the smarttest.
- This makes me feel extra bbouncy
- The fact that people will soon exist in separate realities thanks to sovereign internet policies, interaction bubbles, and AI serving up tailor-made slop makes me want to KMS.
AI has learned to argue like people online do
- Well what do you think the models were trained on?
This is the kind of shit you need to show the kids/teens to make them understand how badly llm's can hallucinate. Enough of these examples and the message might stick.
AI bros: nooo, it's not meant for counting letters!
- It literally isn’t. It’s a Large Language Model - an AI system designed to generate natural-sounding language.
  The fact that it gets any answers right at all isn’t because it knows things - it’s because it’s been trained on data that contains a lot of correct information. Its answers are based on statistical probabilities. It’s physically incapable of looking at a word and counting the letters in it.
  
  Also, it fires off the prompt or various processed versions of it at various backends and then "stuffs" the prompt with the results.
  So you ask a prompt "What team won a sports game last night?" and then the LLM ultimately gets the Prompt like.
  "Summarize the following sports results to indicate a winning team and statistics:
  <fetched content from traditional search engine>
  : ESPN: The Lakers held off the Bulls to finish with a 119 to 117 victory FoxSports: The Bulls lose to the Lakers by two points ...."
  So you end up asking a question, then your question may query a traditional search engine, and the LLM is summarizing the traditional results, and usually doing an ok job of it, except it can mess up by mixing the results together in nonsensical ways. Like one source cited Canada as having the most lakes by a certain criteria with like 300,000 lakes and another result talking about Alaska having 3 million lakes and ends up telling the user "Canada has significantly more lakes than the United states, with 300,000 while the USA has 200,000, with Alaska contributing the most to the US with 3 million lakes".
Why won’t my hammer put this screw in?
- And there it is, the "he's using ChatGPT wrong" defence, as discussed in the post.
  
  While I do admit OpenAI (and all the others) are overselling this like crazy, being strictly anti-AI just isn’t going to do me any favors.
  When I need a language model, I use a language model. I’m not going to ask my English teacher a math question. There are times when it’s useful and times when it’s not, like any other tool.
  Every technology that’s come before it has this double wave of haters and lovers that pull on the extremes of embrace or despise. I’m just taking the middle road and ignoring the hype either way.
  I’m just grabbing the hammer when I need it.
- Hit it harder
- Of course, the downside of any of these memes for demonstration is that it's hard to know if it's real or not because once it becomes a meme, it tends to somehow or another get tweaked out. Like a commonly cited question from the humanity's last exam which LLMs could not answer suddenly got answered by them with pretty much the verbatim same answer, after enough people posted it and replied and it would appear in prompt stuffing giving the LLM the answer crowd sourced.
2+2=5
- For extremely high values of two?
Bluebberry!
I hate these examples. I feel they do such a bad job of showing that LLMs are lying machines and just make the journalists look like pedantic idiots. There is a perfect example of LLMs being lying machines and its from the GPT 5 presentation. GPT 5 tries to explain how lift works like a PHD holder and (according to what i've read from the people who actually have a PHD) it gets it completely wrong. Its explanation is incorrect, the wing shape it uses as an example is incorrect. It just all around fails at providing a correct explanation.
I asked it the same question and it answered correctly then tried 5 or six other random word examples and every one was correct.
Guess we've been wrong all along—time to update the dicttionary.
I feel like diffusion LLMs would get this better.
After “position 5,” an autoregressive LLM has one chance, one pass, to get the next token right instead of another bullet point. And if it randomly picks another bullet point because the temperature is at 1 or whatever, the whole answer is hosed.
Not that OpenAI would ever do that. They just want to deep fry autoregressive transformers more and more instead of, you know, trying something actually interesting.

27 comments