You could "pre-convert" convert them using Handbrake. This Programm uses your graphics card to transcode the media into a playable format like mp4 for example. Then your tv doesn't have to do anything other than just play the stream.
Handbrake would be a great tool for this. Another application that could work is tdarr, which is better at storing different transcoding parameters per directory.
I'm comfortable in a terminal but I'd prefer that I didn't have to do it manually, just set it in a configuration file somewhere and have it happen automatically.
I just went through a similar struggle. A large portion of my media wouldn't play properly on my TV or through a browser. I had considered re-encoding everything, but out of a 20TB library, assuming half of the media would need transcoding, that's a long time re-encoding media.
Ultimately, I upgraded the GPU in my Jellyfin server. Now I have no issues transcoding 4k streams on-the-fly and plenty of GPU overhead left for multiple streams if I need it. If it's in your budget, a GPU is a great way to go. My old Quadro P400 (I think it was $80 brand new a year or two ago) worked great for 1 or 2 4k streams as long as I disabled tone-mapping. My new RTX A2000 ($250 used) can transcode and tone-map a single 4k stream with tons of resources to spare.
If you look in my post history, I have a recent thread specifically about this.
I don't think there is any plan from Jellyfin to support pre-converting from what I remember reading. It is going to be quite resource-intense either way, but you might be able to schedule something during a time you wouldn't use the computer.
That said, what kind of machine you got? Just asking because it is not unpopular to use a raspberry pi as a server, but converting media on it will not work very well.
I have an old HP microserver with an AMD Turion II in it.
It would be nice to be able to transcode overnight and not while trying to watch. Space isn't an issue.