Leaked Documents Show Nvidia Scraping ‘A Human Lifetime’ of Videos Per Day to Train AI
Leaked Documents Show Nvidia Scraping ‘A Human Lifetime’ of Videos Per Day to Train AI
Internal emails, Slack conversations and documents obtained by 404 Media show how Nvidia created a yet-to-be-released video foundational model.
I feel like the amount of training data required for these AIs serves as a pretty compelling argument as to why AI is clearly nowhere near human intelligence. It shouldn't take thousands of human lifetimes of data to train an AI if it's truly near human-level intelligence. In fact, I think it's an argument for them not being intelligent whatsoever. With that much training data, everything that could be asked of them should be in the training data. And yet they still fail at any task not in their data.
Put simply; a human needs less than 1 lifetime of training data to be more intelligent than AI. If it hasn't already solved it, I don't think throwing more training data/compute at the problem will solve this.
There is no "intelligence", ai is a pr word. Just a language model that feeds on a lot of data.
Oh yeah we're 100% agreed on that. I'm thinking of the AI evangelicals who will argue tooth and nail that LLMs have "emergent properties" of intelligence, and that it's simply an issue of training data/compute power before we'll get some digital god being. Unfortunately these people exist, and they're depressingly common. They've definitely reduced in numbers since AI hype has died down though.
You’ve had the entire history of evolution to get the instinct you have today.
Nature Vs Nurture is a huge ongoing debate.
Just because it takes longer to train doesn’t mean it’s not intelligent, kids develop slower than chimps.
Also intelligent doesn’t really mean anything, I personally think Intelligence is the ability to distillate unusable amounts of raw data and intuit a result beneficial to one’s self. But very few people agree with me.
I see intelligence as filling areas of concept space within an econiche in a way that proves functional for actions within that space. I think we are discovering more that "nature" has little commitment, and is just optimizing preparedness for expected levels of entropy within the functional eco-niche.
Most people haven't even started paying attention to distributed systems building shared enactive models, but they are already capable of things that should be considered groundbreaking considering the time and finances of development.
That being said, localized narrow generative models are just building large individual models of predictive process that doesn't by default actively update information.
People who attack AI for just being prediction machines really need to look into predictive processing, or learn how much we organics just guess and confabulate ontop of vestigial social priors.
But no, corpos are using it so computer bad human good, even though the main issue here is the humans that have unlimited power and are encouraged into bad actions due to flawed social posturing systems and the confabulating of wealth with competency.
A human lifetime worth of video is not anywhere close to equalling a human lifetime of actual corporeal existence, even in the perfect scenario where the AI is as capable as a human brain.
Strange to equate the other senses to performance in intellectual tasks but sure. Do you think feeding data from smells, touch, taste, etc. into an AI along with the video will suddenly make it intelligent? No, it will just make it more likely to guess what something smells like. I think it's very clear that our current approach to AI is missing something much more fundamental to thought than that, it's not just a dataset problem.
Properly following licensing, right?
No, see, because it's "learning like a human", and everybody knows that you're allowed to bypass any licensing for learning. /s
But seriously I don't know how they make the jump to these conclusions either.
Information wants to be free.
I mean, i agree, but artists want to eat too.
Humans don't live that long. That's only about 1.5 million 30 min videos, which isn't a huge amount for a whole day's worth of scraping.
Yeah this is honestly an order of magnitude less that I would've thought
Maybe they're running out
I would be lucky if I get to watch more than 10000 videos in my entire lifetime.
That’s only about 1.5 million 30 min videos
aka 2 videos from Quinton Reviews
instead of focusing on their products and improving them for everyone, some shitty ceo is pushing their shitty ai agenda down everyones throat.
Nvidia's biggest product is absolutely AI by a massive landslide, I'm pretty sure I read that the point of them downloading these videos and doing the training is to build a pipeline for their AI users to do the same with their own shit. (Can't be bothered to double-check cuz I really don't care)
So they aren't downloading all this video to make a crazy AI model. They're downloading all this video to make a tool for their AI customers to use, you may not agree but improving their product is exactly what they're doing.
So they use VMs to simulate user accounts, in future this will be blocked and whatever new AI startup is there won't have the option to do so. Competition blocked. Forever.
There's only a handful of video datasets and all of it is owned by Google through YouTube or big Hollywood companies like Disney and Netflix.
These companies are foaming at the mouth with rage thinking about what generative AI will do to their industry and how much it will help the currently non existant indie one. They will do whatever it takes to fence in the playbox and make sure they get to be the toll man.
This was never about AI getting to live or not, but who gets to own it. 404media is essentially a mouthpiece for these corporations, willingly or not, and the strengthening of copyright laws will not help the consumers or the small time creators. The only exception being laws that force copy left licenses onto models but that's not what is being pushed right now, as well as aocs Deepfake act which is well thought out imo.
Anyone should be permitted to train on YouTube and Netflix data, and Nvidia might even open source it in any case.
Nvidia does not have a strong history of open sourcing things, to say the least. That last bit sounds like pure hopium
Their nematron 320b model was released on what essentially is an open source licence (available for commercial use except if you are doing shady things like spamming and collecting biometric data).
Having a robust open source ecosystem directly benefits Nvidia since they sell more higher end consumer GPUs.
Obviously, there's a real chance that this isn't open sourced since it's a video model and there's huge money involved. Doesnt really change the fact that having YouTube and Netflix dictate who gets to make video models and at what cost isn't a good idea.
Can we stop with this bullshit? Nobody will buy into it. WE DON’T WANT IT.
It's not for you as a consumer.
It's to reduce your usefulness as a worker.
Which would be lovely, if our value wasn't calculated by our usefulness to the market.
Obligatory fuck AI and the illeterate bros pushing it.
What kind of videos, though? A lot of such material is very far from being proper educational material that we show other people to really teach them much, let alone educate them well enough to be anywhere trustworthy. This is a very processed material, with years of preparation once you consider the prior education of the individuals involved in the creative process - think of the past experiences silently influencing them, their initial knowledge on the subject obtained from somewhat basic facts from school or otherwise, their misconceptions, iterations that nobody knows about, and many other things that we don't usually directly associate with the act of working on something like a video, but that eventually do dictate a lot of the decisions and opinions put into it.
It's one thing that the AI has no intelligence in it whatsoever, but the fact that it's being pumped with information and "knowledge" in basically the reverse order doesn't help it become any better.
On the other hand, the entire thing is not about making something that works well, but something that sells well. And then there's people putting too much faith into the thing and trusting it with way too much stuff than they should (which is also the case with a lot of other tech, though, admittedly).
Some things of today are so damn unexciting.
I hope they aren’t on Comcast.
Can relate, I watched the English patient once.
Something like that was a plot point in Black Mirror. In that case it was with consciousnesses.