There is a serious security flaw in billions of Intel CPUs that can let attackers steal confidential data like passwords and encryption keys. Firmware updates can fix it, but at a potential significant performance loss.
I would say you'll be fine. Most games don't compile with avx-2 anyways since it'll crash if you run it on something that doesn't have them (which is a lot of CPUs) and AVX-512 is straight up only available on Xeons, Epyc and zen 4. Nobody is going to use that for consumer software.
The only game I can think of using AVX is a Skyrim mod for realistic physics, where the author provided binaries for AVX-2/AVX-512. So it won't affect most compiled applications much since you need to compile with it first (which almost nobody does).