Edited: It seems that this only happens to Celeste. Hollow Knight works just fine (haven't tested other games yet).
Edited 2: Thanks everyone! I've made it to work. See the comment.
Just want you guy to know, I managed to have it working thanks to this Redditor.
The issue here is that OBS and Celeste have to be running on the discrete GPU for Game Capture to work. I know this is also mentioned in the guide I referred, but initially, I only configured OBS to run on the discrete GPU. In my case, I also have to do that to Celeste, which is what I missed.
I always thought Celeste was running on my RTX3050, but all this time it just ran on my Intel integrated GPU without me noticing difference... lol
I have no idea if it would help, but if you're on windows you could also try to switch to the openGL version of Celeste. I don't know exactly what's different, but it changes some libraries or something, so there's a chance it'll work. (Don't quote me on that)
You can change it on steam from Library -> Right-click on game -> Properties -> Betas
i always switch games to my nvidia gpu anyway because it adds a lot of capture options without having to open obs. obviously if you dont have an nvidia gpu this doesnt apply