Can modern LLMs count the number of b's in "blueberry"?
zeropointone @ zeropointone @lemmy.world Posts 0Comments 83Joined 3 days ago
zeropointone @ zeropointone @lemmy.world
Posts
0
Comments
83
Joined
3 days ago
What makes you think that using single letters as tokens instead could teach a stochastic parrot to count or calculate? Both are abilities. You can't create an ability only from a set of data no matter how much data you have. You can only make a model seem to have that ability. Again: All you can ever get out of it is something that resembles human language. There is nothing beyond/behind that, by design. Not even hallucinations. Whenever a LLM gives you the advice to eat a rock per day it still works. Because it outputs a correct sounding sentence purely and entirely based on probability. But counting and calculating are not based on probability which is something everyone who ever had a math class knows very well. No math teacher will let students guess the result of an equation.