As a programmer: most people vastly overestimate the efficacy of large language models.
CEOs seem to overestimate them even more than everyone else.
A lot of AI researchers think LLMs are a dead end (See: Timnit Gebru) because by their structure they cannot understand truth.
The "hallucinations" are intrinsic to the structure and the best minds are saying there's no way around that.
We might be able to cludge together filters over it but at some point that's just hard coding the world anyways, which is what LLMs are supposed to avoid.
I've been using chatgpt a lot, and it's really clear to me it has many uses, but it's almost more like asking your buddy who knows a lot but is full of shit too -- sometimes he tells you exactly what you need, sometimes he sends you on a wild goose chase with all kinds of false leads.
In the end you need your own competence because the human needs to be able to make a final decision about whether to listen or not.
Have you seen the work where they use another instance to fact check the first? The MS Research podcast made it seem like a really viable way to find hallucinations without really needing to code more. I'm curious if other people find that works or if MS researchers are just too invested in gpt.
I'll check out that podcast but I'm deeply skeptical that one LLM can correct another since neither of them truly understands anything: it's all statistics. Very detailed stats but still stats.
And stats will be wrong.
Before chatgpt released most Google AI engineers were looking into alternatives to LLMs as the limitations of an LLM were increasingly clear.
They're convincing facsimiles of intelligence and a good tool for maybe 80% of basic uses.
But I agree with the consensus: they're a dead end in our search for intelligence and their output is vastly overestimated