SSH is one of the protocols which are widely used on the Internet: developers use it to push code on a git server (such as GitHub), system administrators use it to connect to remote consoles in a secure way, some users use it as a VPN solution (thanks to TCP
This seems like a non-issue if you use FDE and proper file permissions. And even if you don't want to type your password at boot, you can use TPM for the FDE itself.
I prefer use Yubikey/Canokey instead of TPM. SSH key may be used outside or in other environment (such as my laptop, Android or iPad). And TPM support less algorithms than Yubikey, especially Ed25519 and RSA4096.