MIT scientists have just figured out how to make the most popular AI image generators 30 times faster
MIT scientists have just figured out how to make the most popular AI image generators 30 times faster
Scientists have built a framework that gives generative AI systems like DALL·E 3 and Stable Diffusion a major boost by condensing them into smaller models — without compromising their quality.
Partnered with Adobe research so we're never going to get the actual model.
More mediocre images for everyone!
While I think the realism of some models is fantastic and the flexibility of others is great it is starting to feel like we're reaching a plateau on quality. Most of the white papers I've seen posted lately are about speed or some alternate way of doing what ControlNet or inpainting can already do.
Have you seen the SD3 preview images? They're looking seriously impressive.
When the output of something is the average of the inputs it will naturally be mediocre. It will always look like the output of a committee by the nature of how it is formed.
Certain artists stand out because they are different from everyone else, and that is why they are celebrated. M.C. Escher has a certain style that when run through AI looks like a skilled high school student doing their best impression of M.C. Escher.
Now as a tool to inspire, AI is pretty good at creating mashups of multiple things really fast. Those could be used by an actual artist to create something engaging. Most AI reminds me of photoshop battles.
That's maybe because we've reached the limits of what the current architecture of models can achieve on the current architecture of GPUs.
To create significantly better models without having a fundamentally new approach, you have to increase the model size. And if all accelerators accessible to you only offer, say, 24gb, you can't grow infinitely. At least not within a reasonable timeframe.
Please link original article and paper when posting
Article: https://news.mit.edu/2024/ai-generates-high-quality-images-30-times-faster-single-step-0321
Pfft, I can do that, just run them on a computer that's 30 times faster!
Found the software engineer.
These kind of performance improvements have really cool potential for real time image/texture generation in games. I've already seen some games do this, but they usually rely on generating the images online.
ASCII and low graphic roguelike's have a lot of generation freedom where they can create very unique monsters/items/etc. However a lot of this flexibility is lost as you move to more polished games that require models and art assets for everything. This is also one of the many reasons that old-styled games are still popular, is because they often offer more variety and randomization than newer titles. I think generated art assets could be a cool way to bridge the gap though, and let more modern games have crazy unique monsters/items with visuals.
So can I run this without cloud?
Although I mainly use InvokeAI
I see your InvokeAI and raise you Stability Matrix
Edit: I wanted to edit my comment to leave some context for people.
Stability Matrix is an app that handles installing many different stable diffusion applications. (no more messing with InvokeAI's janky install script).
It also integrates with CivitAI and HuggingFace to directly download models and Lora and share them between your applications, saving you lots of diskspace.
but that is just normal Stable Diffusion, not the method used or mentioned here. So it isn't even what this news is about :/
The paper: https://arxiv.org/pdf/2311.18828.pdf
So more AI porn?
Faster AI porn
Gotta jack fast
This is nice, but the post ignores all the other research in this topic. SDXL Lightning can generate images in 2 steps.
1st step, describe the picture.
2nd step generate the picture.1st step: draw some circles
2nd step: draw the rest of the fucking owl
Now they can make a movie!