I mean... "To fear"? No. But There are plenty legitimate to remove comments and posts that have nothing, and I mean absolutely nothing, to do with mod overreach or censorship.
removing derailing threads from heated discussions
removing the annoying "just asking questions" people from LGBTQ+ safe spaces
removing posts accidentally posted into the wrong community
removing troll posts
banning repeat troublemakers not willing to follow the rules
removing aggressive, sexist, racist,... posts
On the other hand, anti-moderation people only ever seem to come up with "but I want to be able to post whatever I want!"
"Free speech" in this context means: you can go create your own instance or community, with blackjack and hookers! And mods can use the tools at their disposal to enable the rest of us to not have to deal with bullshit.