OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further
OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further
Once you’ve trained your large language model on the entire written output of humanity, where do you go? Here’s Ilya Sutskever, ex-OpenAI, admitting to Reuters that they’ve plateaued: [Reuters] The…
I'm usually the one saying "AI is already as good as it's gonna get, for a long while."
This article, in contrast, is quotes from folks making the next AI generation - saying the same.
You're viewing a single thread.
OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further
Lol, no they didn't. The quotes this articles are using are talking about LLMs not chatbots. This is yet another stupid article from someone who doesn't understand the technology. There is a lot of legitimate criticism for the way this technology is being implemented but FFS get the basics right at least.
Are you asserting that chatbots are so fundamentally different from LLMs that "oh shit we can't just throw more CPU and data at this anymore" doesn't apply to roughly the same degree?
I feel like people are using those terms pretty well interchangeably lately anyway
People that don't understand those terms are using them interchangeably
LLM is the technology, Chatbot is an implementation of it. So yes a Chatbot as it's talked about here is an LLM. Although obviously chatbots don't have to be LLM, those that are not are irrelevant.
No, a chat bot as it's talked about here is not an LLM. This article is discussing limitations of LLM training data and inferring that chat bots can not scale as a result. There are many techniques that can be used to continue to improve chat bots.
The chatbot is a front end to an LLM, you are being needlessly pedantic. What the chatbot serves you, is the result of LLM queries.
That may have been true for the early LLM chatbots but not anymore. ChatGPT for instance, now writes code to answer logical questions. The o1 models have background token usage because each response is actually the result of multiple background LLM responses.
Yes of course I'm asserting that. While the performance of LLMs may be plateauing, the cost, context window, and efficiency is still getting much better. When you chat with a modern chat bot it's not just sending your input to an LLM like the first public version of ChatGPT. Nowadays a single chat bot response may require many LLM requests along with other techniques to mitigate the deficiencies of LLMs. Just ask the free version of ChatGPT a question that requires some calculation and you'll have a better understanding of what's going on and the direction of the industry.
I think you're agreeing, just in a rude and condescending way.
There's a lot of ways left to improve, but they're not as simple as just throwing more data and CPU at the problem, anymore.
I'm sorry if I'm coming across as condescending, that's not my intent. It's never been "as simple as just throwing more data and CPU at the problem". There were algorithmic challenges for every LLM evolution. There are still lots of potential improvements using the existing training data. But even if there wasn't, we'll still see loads of improvements in chat bots because of other techniques.
Edit: typo
Claiming that David Gerrard an Amy Castor "don't understand the technology" is uh.... Hoo boy... Well it sure is a take.
The title of the article is literally a lie which is easily fact checked. Follow the links to quotes in the article to see what the quoted individuals actually said about the topic.
Please learn the difference between "lying" and "presenting a conclusion."
I know the difference. Neither OpenAI, Google, or Anthropic have admitted they can't scale up their chat bots. That statement is not true.
So is your autism diagnosed or undiagnosed?
I ask this as an autistic person, because the only charitable way to read what's happening here is that you're clearly struggling with statements that aren't intended to be read completely literally.
The only other way to read it is that you're arguing in bad faith, but I'll assume thats not the case.
Also an autistic person here.
How are people supposed to tell this is an opinion?
And please dont say “by reading the article, maybe some (like me) do so but its well known that most people stop at the title.
Grammatically speaking it remains a direct statement. They admit == appear to hint == pure opinion (Title: “Ai cant be scaled further”)
While i am not disagreeing with the premise perse i have to perceive this as anti-ai propaganda at best, a attempt at misinformation at worst.
On a different note, do you believe things can only be an issue if neurotypical struggle with it? There is no good argument to not communicate more clearly in the context of sharing opinions with the world.
David and Amy are - openly - skeptics in the subject matters they write about. But it's important to understand that being a skeptic is not inherently the same thing as being unfairly biased against something.
They cite their sources. They backup what they have to say. But they refuse to be charitable about how they approach their subjects, because it is their position that those subjects have not acted in a way that is deserving of charity.
This is a problem with a lot of mainstream journalism. A grocery store CEO will say "It's not our fault, we have to raise prices," and mainstream news outlets will repeat this statement uncritically, with no interrogation, because they are so desperate to avoid any appearance of bias. Donald Trump will say "Immigrants are eating dogs" and news outlets will simply repeat this claim as something he said, with adding "This claim is obviously insane and only an idiot would have made it." Sometimes being overly fair to your subject is being unfair to objective truth.
Of course OpenAI et al are never going to openly admit that they can't substantially improve their models any further. They are professional bullshitters, they didn't suddenly come down with a case of honesty now. But their recent statements, when read with both a critical eye, and an understanding of the limitations of the technology, amount to a tacit admission that all the significant gains have already been made with this particular approach. That's the claim being made in this headline.