AI Expert Warns Crash Is Imminent As AI Improvements Hit Brick Wall

"LLMs such as they are, will become a commodity; price wars will keep revenue low. Given the cost of chips, profits will be elusive," Marcus predicts. "When everyone realizes this, the financial bubble may burst quickly."

Please let this happen

I wish just once we could have some kind of tech innovation without a bunch of douchebag techbros thinking it's going to solve all the world's problems with no side effects while they get super rich off it.

No shit. This was obvious from day one. This was never AGI, and was never going to be AGI.

Institutional investors saw an opportunity to make a shit ton of money and pumped it up as if it was world changing. They'll dump it like they always do, it will crash, and they'll make billions in the process with absolutely no negative repercussions.

largely based on the notion that LLMs will, with continued scaling, become artificial general intelligence

Who said that LLMs were going to become AGI? LLMs as part of an AGI system makes sense but not LLMs alone becoming AGI. Only articles and blog posts from people who didn't understand the technology were making those claims. Which helped feed the hype.

I 100% agree that we're going to see an AI market correction. It's going to take a lot of hard human work to achieve the real value of LLMs. The hype is distracting from the real valuable and interesting work.

It's so funny how all this is only a problem within a capitalist frame of reference.

Thank fuck. Can we have cheaper graphics cards again please?

I'm sure a RTX 4090 is very impressive, but it's not £1800 impressive.

Oh no!

Anyway...

is this where we get to explain again why its not really ai?

Huh?

The smartphone improvements hit a rubber wall a few years ago (disregarding folding screens, that compose a small market share, improvement rate slowed down drastically), and the industry is doing fine. It's not growing like it use to, but that just means people are keeping their smartphones for longer periods of time, not that people stopped using them.

Even if AI were to completely freeze right now, people will continue using it.

Why are people reacting like AI is going to get dropped?

The hype should go the other way. Instead of bigger and bigger models that do more and more - have smaller models that are just as effective. Get them onto personal computers; get them onto phones; get them onto Arduino minis that cost $20 - and then have those models be as good as the big LLMs and Image gen programs.

"The economics are likely to be grim," Marcus wrote on his Substack. "Sky high valuation of companies like OpenAI and Microsoft are largely based on the notion that LLMs will, with continued scaling, become artificial general intelligence."

"As I have always warned," he added, "that's just a fantasy."

Because nobody could have possibly saw that coming. /s

Marcus is right, incremental improvements in AIs like ChatGPT will not lead to AGI and were never on that course to begin with. What LLMs do is fundamentally not "intelligence", they just imitate human response based on existing human-generated content. This can produce usable results, but not because the LLM has any understanding of the question. Since the current AI surge is based almost entirely on LLMs, the delusion that the industry will soon achieve AGI is doomed to fall apart - but not until a lot of smart speculators have gotten in and out and made a pile of money.

I think I've heard about enough of experts predicting the future lately.

I am so tired of the ai hype and hate. Please give me my gen art interest back please just make it obscure again to program art I beg of you

This is why you're seeing news articles from Sam Altman saying that AGI will blow past us without any societal impact. He's trying to lessen the blow of the bubble bursting for AI/ML.

It's been 5 minutes since the new thing did a new thing. Is it the end?

As I use copilot to write software, I have a hard time seeing how it'll get better than it already is. The fundamental problem of all machine learning is that the training data has to be good enough to solve the problem. So the problems I run into make sense, like:

Copilot can't read my mind and figure out what I'm trying to do.
I'm working on an uncommon problem where the typical solutions don't work
Copilot is unable to tell when it doesn't "know" the answer, because of course it's just simulating communication and doesn't really know anything.

2 and 3 could be alleviated, but probably not solved completely with more and better data or engineering changes - but obviously AI developers started by training the models on the most useful data and strategies that they think work best. 1 seems fundamentally unsolvable.

I think there could be some more advances in finding more and better use cases, but I'm a pessimist when it comes to any serious advances in the underlying technology.

Welcome to the top of the sigmoid curve.

If you were wondering what 1999 felt like WRT to the internet, well, here we are. The Matrix was still fresh in everyone's mind and a lot of online tech innovation kinda plateaued, followed by some "market adjustments."

I work with people who work in this field. Everyone knows this, but there's also an increased effort in improvements all across the stack, not just the final LLM. I personally suspect the current generation of LLMs is at its peak, but with each breakthrough the technology will climb again.

Put differently, I still suspect LLMs will be at least twice as good in 10 years.

Of course it'll crash. Saying it's imminent though suggests someone needs to exercise their shorts.

I just want a portable self hosted LLM for specific tasks like programming or language learning.

Seems to me the rationale is flawed. Even if it isn't strong or general AI, LLM based AI has found a lot of uses. I also don't recognize the claimed ignorance among people working with it, about the limitations of current AI models.

I hope it all burns.

so long, see you all in the next hype. Any guesses?

Nice, looking forward to it! So much money and time wasted on pipe dreams and hype. We need to get back to some actually useful innovation.

Sigh I hope LLMs get dropped from the AI bandwagon because I do think they have some really cool use cases and love just running my little local models. Cut government spending like a madman, write the next great American novel, or eliminate actual jobs are not those use cases.

Short on the AI stocks before it crash!

Fingers crossed.

Oh nice, another Gary Marcus "AI hitting a wall post."

Like his "Deep Learning Is Hitting a Wall" post on March 10th, 2022.

Indeed, not much has changed in the world of deep learning between spring 2022 and now.

No new model releases.

No leaps beyond what was expected.

\s

Gary Marcus is like a reverse Cassandra.

Consistently wrong, and yet regularly listened to, amplified, and believed.

Yay

The tech priests of Mars were right; death to abominable intelligence.

Theres no bracing for this, OpenAI CEO said the same thing like a year ago and people are still shovelling money at this dumpster fire today.

Crash? Doesn't it have to be moving at all to crash?

It's gonna crash like a self driving tesla. It's gonna fall apart like a cybertrukkk.

yep Knew ai should die some day.

It's had all the signs of a bubble for the last few years.

Ya AI was never going to be it. But I wouldn’t understate its impact even in its current stage. I think it’ll be a tool that will be incredibly useful for just about every industry