Large Language Model Performance Doubles Every 7 Months
Large Language Model Performance Doubles Every 7 Months
By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.
By 2030, AI will greatly outperform humans in some complex intellectual tasks. Discover how LLMs are doubling their capabilities every seven months.
Classic pseudo-science for the modern grifter. Vague definitions, sloppy measurements, extremely biased, wild unsupported predictions, etc.
That graph is hilarious. Enormous error bars, totally arbitrary quantization of complexity, and it's title? "Task time for a human that an AI model completes with a 50 percent success rate". 50 percent success is useless, lmao.
On a more sober note, I'm very disappointed that IEEE is publishing this kind of trash.
in yes/no type questions, 50% success rate is the absolute worst one can do. Any worse and you're just giving an inverted correct answer more than half the time
This is such bullshit. Models have already consumed all available data and have nothing left to consume, whole needing exponentially more data for progressive advancements
This. It's the old "to the moon" mentality.
If my 2yo continues to grow at the current rate, we'll have to buy new doors soon becouse at age 10 the kid will be the tallest person on Earth.
Apparently, throwing more data at it will not help much from now on... But anyway what they're saying, I can't trust the snake oil seller, he is suspicious...
How is completely fucking up literally 50% of the time outperforming exactly???
You see, in 7 months, they'll fuck up literally 100% of the time! Progress.
It would be even better but unfortunately you can't exceed 100% wrong.
Wait, maybe you can, let me check the AI!
It's outperforming "messier" problems with a much lower success rate.
I very much like those huge generalizations in AI articles that makes you small and stupid. Those generalizations proves nothing but they sound like something big is coming. It's parody. How long we see them before people wake up ? Just wait 2 more years and AI will be better bro. You're not using AI properly, you need to learn how to use AI bro. You need to use different model for this task bro. Just pay for corporate products bro. Amount of junk of top of this pile of shit is amusing.
So only 10 years until it isn't a ressource hog anymore...
Only if people give up on the whole concept by then. Each new generation of AI model takes more energy than the last.
Then why do I feel like it's programming abilites are getting worse? I've stopped paying for it now because it causes more frustration than anything else. Works for simple "how can I simplyfi this code" queries when my head hurts, but that's about it.
Yawn...
they are improving at an exponential rate. It's just that the exponent is less than one.
I saw something once that explained how you can have an ai trained on a set of soccer games and have it generate soccer games as a use for it.
The idea is that the model has compressed all the soccer games into a smaller data size form than the total of having let's say 100+ games on video or whatever.
That's the real utility I see in generative ai that I know can keep going basically as long as we want to.
new moore law dropped
Hopefully they hurry up, let the AI takeover so it can just liquidate us all.
This is like measuring the increasing speeds of cars in the early years and extrapolating that they would be supersonic by now by ignoring the exponential impact that air resistance has.
My son has doubled in size every month for the last few months. At this rate he'll be fifty foot tall by the time he's seven years old.
Yeah, it's a stupid claim to make on the face of it. It also ignores practical realities. The first is those is training data, and the second is context windows. The idea that AI will successfully write a novel or code a large scale piece of software like a video game would require them to be able to hold that entire thing in their context window at once. Context windows are strongly tied to hardware usage, so scaling them to the point where they're big enough for an entire novel may not ever be feasible (at least from a cost/benefit perspective).
I think there's also the issue of how you define "success" for the purpose of a study like this. The article claims that AI may one day write a novel, but how do you define "successfully" writing a novel? Is the goal here that one day we'll have a machine that can produce algorithmically mediocre works of art? What's the value in that?
Air resistance has cubic not exponential impact
Or like looking at the early days of semiconductors and extrapolating that CPU speed will double every 18 months ..smh these people
Since CPU speeds are still doubling every 18 months you have a solid point!
Or maybe not since you are probably referring to the doubling of transistors that was an observation which was accurate over a lengthy period of time in the context of when the observation was made. Nobody said that would continue indefinitely either.
*with 50 percent reliability.
Heck of an asterisk on this claim.
That sounds like a coin flip, but 50% reliability can be really useful.
If a model has 50% chance of completing a task that would cost me an hour - and I can easily check it was completed correctly - on average, I'm saving half of the time it would take to complete this.
That said, exponentials don't exist in the real world, we're just seeing the middle of a sigmoid curve, which will soon yield diminishing returns.
and I can easily check it was completed correctly
Can you always though?
That said, exponentials don’t exist in the real world, we’re just seeing the middle of a sigmoid curve, which will soon yield diminishing returns.
Yes, but the tricky thing is we have no idea when the seemingly exponential growth will flip over into the plateuing phase. We could be there already or it could be another 30 years.
For comparison Moores law is almost certainly a sigmoid too, but weve been seeing exponential growth for 50 years now.
All that power used for a fucking coin flip.
I doubt it
2 X 0 = 0
Is the performance increase related to computing power? I suspect the undelying massive datacenters running the cloud based LLMs are expanding at a similar rate...
"performance"