I'm guessing the cloudflared daemon isn't connecting to jellyfin. You want to use http://. Also is jellyfin the hostname of the VM? Using localhost or 127.0.0.1 might be better ways to specify the same VM without relying on DNS for anything.
Personal opinion, but I wouldn't bother with fail2ban, it's a bit of effort to get it to work with cloudflare tunnel and easy to lock yourself out. Cloudflare's own zero trust feature would be more secure and only need fiddling around cloudflare's dashboard.
You will want the actual IP address. Localhost can get lost in various circumstances. If Cloudflare tunnel service and Jellyfin are on the same virtual network it should be fine. But I wouldn't trust it.
But yes, your Cloudflare tunnel should only connect to http:// not https. It will serve https on the public side of things.