2y ago

How usable are AMD GPUs?

Heyho, I'm currently on a RTX3070 but want to upgrade to a RX 7900 XT

I see that AMD installers are there, but is it all smooth sailing? How well do AMD cards compare to NVidia in terms of performance?

I'd mainly use oobabooga but would also love to try some other backends.

Anyone here with one of the newer AMD cards that could talk about their experience?

EDIT: To clear things up a little bit. I am on Linux, and i'd say i am quite experienced with it. I know how to handle a card swap and i know where to get my drivers from. I know of the gaming performance difference between NVidia and AMD. Those are the main reasons i want to switch to AMD. Now i just want to hear from someone who ALSO has Linux + AMD what their experience with Oobabooga and Automatic1111 are when using ROCm for example.

47 comments

If you don't need cuda or ai, the 7900 is great.
- You can run CUDA apps on ROCm HIP. It’s easy.
  
  Whoa need to me. I'll have to dig in on that.
- Well that's the question...
  What you mean with "not needing ai"? I mean oobabooga and stable diffusion have AMD installers, and that's exactly what I am asking about. Therefore I post in community...
  To find out how good those AIs run on AMD
  
  Oops. I wasn't looking at the community just my main feed. Ok, so from what I understand the amd installer is a bit of a pain on Linux. If you're on windows it's probably a different story.
Wondering the same thing. All the ram in the 790xtx looks nice
You might also want to have a look at Koboldcpp or llama.cpp performance with ROCm. The LLMs seem mainly to be constrained by memory bandwith anyways. And not raw compute performance.
- Will do. Ty
Been using a rx 6600xt for a year now and I never had a single issue with drivers. used rocm once and it worked without issues
- I'm not asking about drivers or such. I'm asking about performance specifically in Oobabooga and/or StableDiffusion.
  Have you done anything with those?
  
  used stable diffusion and it took like 6 seconds to generate a 512x512px image
You probably want to use the amd driver out of the box on your linux distro + ROCm instead of whatever AMD gives you as a driver download on their landing page
Gaming wise the AMD card would win in rasterization performance but PyTorch is made for CUDA (Nvidia only) first instead of OpenCL/HID (which AMD uses)
I couldn't get my AMD card to run reliable in half-precision (16fp) which actually hurts performance A LOT in comparison to no-half or 32fp
interestingly enough setting up AMD cards on Linux with ROCm is actually easier compared to Windows
anyway my experience is mostly stable difussion and some early gpt4all stuff but oobabooga uses PyTorch too so its probably similar
- I've had AMD cards my whole life and only switched to NVidia 3 years ago where that whole local LLM and ImageAI thing wasn't even on the table...now i am just pissed that NVidia gives us so little VRAM to play with unless you pay the same price as used car -.-
  AMD drivers are available from within the kernel so yeah, i won't do any downloading for AMD drivers on Linux^^
  Oobabooga and Automatic1111 are my main questions - i could actually live with a downgrade in terms of performance if i then atleast can run the bigger models due to having way more VRAM. Can't even run 17b models on my current 8GB VRAM card...can't even make 1024x1024 images on Auto1111 without getting Issues aswell. If i can do those things but a bit slower, thats fine for me^^
  
  What sort of issues are you getting trying to generate 1024x1024 images in Stable Diffusion? I've generated up to 1536x1024 without issue on a 1070 (although it takes a few minutes) and could probably go even larger (this was in img2img mode which uses more VRAM as well - although at that size you usually won't get good results with txt2img anyway). What model are you using?
  
  I only ever used 7b large language models on my RX 6950 XT but PyTorch had or still has some nasty AMD VRAM bugs which didn't fully utilized all of my VRAM (more like only a quarter of it)
  it seems the sad truth is high performance/training of models are just not good on AMD cards as of now
Using a 6800xt on Linux for several months and I’m super happy with it. There hasn’t been anything I haven’t been able to do. AMA.
- what models are you using and how many iterations /s do you get on average with them?
  Do you also use StableDiffusion (Auto1111)? If yes, same question as above for that^^
  
  I use a ton of different ones. I can test specific models if you like.

47 comments