AMD Quietly Funded A Drop-In CUDA Implementation Built On ROCm: It's Now Open-Source
AMD Quietly Funded A Drop-In CUDA Implementation Built On ROCm: It's Now Open-Source
After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs. One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today.
From https://github.com/vosen/ZLUDA?tab=readme-ov-file#faq
Those benchmarks are impressive indeed. I certainly care much more about having extra VRAM than a little extra speed, comparing the 7900 XT or 7900 XTX to the RTX 4080.
I'd love to see some benchmarks for various LLMs and image generators.
Now we'll see if Nvidia drops CUDA immediately, or waits until next quarter.
Now let's get this working on Nvidia hardware :P
as pointless as it sounds it would be a great way to test the system and call alternative implementations of each proprietary Nvidia library. It would also be great for debugging and development to provide an API for switching implementations at runtime.
All I need to know about AMD is this:
Whenever amdgpu.ko is insmoded, the display (or the system itself in some cases) is unstable. When it works great now, one day it won't and the machine will inexplicably start crashing randomly or displaying garbage after an update.
This has held true for years for me on many machines I've installed Linux on, and it still does: not a week ago, I updated a laptop with a Renoir chipset in it (RX Vega 6) that had been stable for years, and now the display gets corrupted whenever I switch VT. Because amdgpu...
Not bashing on AMD or Nvidia. This has just been my reality. As a result, whenever I have a choice, I go with Intel graphics because it never causes me as much of a headache.
Do LLM or that AI image stuff run on CUDA?
Cuda is required to be able to interface with Nvidia GPUs. AI stuff almost always requires GPUs for the best performance.
Nearly all such software support CUDA, (which up to now was Nvidia only) and some also support AMD through ROCm, DirectML, ONNX, or some other means, but CUDA is most common. This will open up more of those to users with AMD hardware.
Thanks that is what I was curious about. So good news!
They are usually released for CUDA first, and if the projects got popular enough, someone will come in and port them to other platforms, which can take a while especially for rocm. Apple m series ports usually appear first before rocm, that's show how much the devs community dislike working with rocm with famous examples such as geohot throwing the towel after working with rocm for a while.
Yes, llama.cpp and derivates, stable diffusion, they also run on ROCm. LLM fine-tuning is CUDA as well, ROCm implementations not so much for this, but coming along.
ROCm DKMS modules
Huh? What are these?
Since when does ROCm require kernel modules? DRI exists?
Technically always has, ROCm comes with a "backported" amdgpu module and that's the one they supposedly test/officially validate with. It mostly exists for the ancient kernels shipped with old long-time support distros.
Of course, ROCM being ROCM, nobody is running an officially supported configuration anyway and the thing is never going to work to an suitably acceptable level. This won't change that, since it's still built on top of it.
This is the best summary I could come up with:
While there have been efforts by AMD over the years to make it easier to port codebases targeting NVIDIA's CUDA API to run atop HIP/ROCm, it still requires work on the part of developers.
The tooling has improved such as with HIPIFY to help in auto-generating but it isn't any simple, instant, and guaranteed solution -- especially if striving for optimal performance.
In practice for many real-world workloads, it's a solution for end-users to run CUDA-enabled software without any developer intervention.
Here is more information on this "skunkworks" project that is now available as open-source along with some of my own testing and performance benchmarks of this CUDA implementation built for Radeon GPUs.
For reasons unknown to me, AMD decided this year to discontinue funding the effort and not release it as any software product.
Andrzej Janik reached out and provided access to the new ZLUDA implementation for AMD ROCm to allow me to test it out and benchmark it in advance of today's planned public announcement.
The original article contains 617 words, the summary contains 167 words. Saved 73%. I'm a bot and I'm open source!
Presumably they did not want to see Cuda becoming the final de-facto standard that everyone uses. It nearly did at one point a couple of years ago, despite the lack of openness and lack of AMD hardware support.
i heavily rely on CUDA for many things i do on my personal computer. If this establishes itself as a reliable method to use all the funky CUDA stuff on AMD cards, my next card will 100% be AMD.
If there were a drop in equivalent to CUDA with AMD, I'd have several AMD cards, right now.
They stopped funding the replacement, not CUDA.
By funding an API-compatible product, they are giving CUDA legitimacy as a common API. I can absolutely understand AMD not wanting a competitors invention and walled-off product to be anything resembling an industry standard.
That is contradicted by the headline. This easy confusion between CUDA (the API) and CUDA (the proprietary software package that is one implementation of it) illustrates the problem with CUDA.
ZLUDA seems to be an effort to fix that problem, but I don't know what it's chances of success might be.