I've been quite stupid with this but never really had issues. Ever since I changed the open ssh port from 22 to something else, my server is basically ignored by botnets. These days I obviously also have some other tricks like fail2ban, but it was funny how effective that was.
Almost the same here. I also change some ssh settings: disable root login, disable password, allow only public key login. That's about it. I never had any problems.