I agree and would try this same approach. It can be difficult being firm in social situations as there are so many factors to consider. I feel like the risk is worth it to resolve the situation however. If they dont mean it maliciously then perhaps they will amend their behavior once they themselves arrive at the conclusion thoroughly that you have this boundary.