AI agents wrong ~70% of time: Carnegie Mellon study
AI agents wrong ~70% of time: Carnegie Mellon study
www.theregister.com
AI agents wrong ~70% of time: Carnegie Mellon study
3 comments
-
-
I'll start believing in AI when, and if, it's able to eliminate error. When will AI be able to work out whether the training material it used is true, fasle, myth, or other narrative?
-
We tried to build systems that perform a kind of basic, rudimentary, extremely power intensive and inefficient mimicry of how (we think maybe) brain cells work.
Then that system lies to us, makes epic bumbling mistakes, expresses itself with extreme, overconfidence, and constantly creatively misinterprets simple instructions. It recognizes patterns that aren't there, and regurgitates garbage information that it picks up on the internet.
Hmmm... Actually, maybe we're doing a pretty good job of making systems that work similarly to the way brain cells work...
-
This bit at the end, wow:
Agentic AI is wrong 70% of the time, but even assuming a human employee is barely correct most of the time and wrong 49% of the time, is it really still more efficient to replace them?