We don't let ours out because we're decent. Sometimes they'll argue through the screen, but they seem to be getting along a bit better now. Now they just stare each other down.
Not only decent. If you let your cat wander alone it is not your pet, it's just a cat you feed and don't care about. My cat ran away for 30 min and I was crazy worried she would be ran over, get lost, eat something she shouldn't etc
Literally had a neighbour's cat come to our house and hang out for several hours, almost on a daily basis. Sometimes he even stayed from afternoon to evening. If you let your cat out, you're sharing it with the whole neighbourhood.
My oldest cat needs to walk the perimeter of our house once a day because of the neighbor's outdoor cat. She tolerates a harness and lead (which I needed to prevent her from chasing things. The other two cats follow her progress from the windows.