I always get them confused.

I would argue that science fiction and fantasy are the same thing and the only difference is the explanation for how all the cool stuff works.

Functionally, there's no real difference between a portal that takes the characters to another world using a wormhole and one that does it through through magic. Just like how there's no difference between Vulcans/Klingons/Wookies and Elves/Dwarves/Beastmen. Both are intelligent non-humans.