I just don't mind my glasses that much that I want to put myself through this/take the risk/pay the cost. I've had them since I was a child, I'm used to them and as far as I know, that's still what has the least side/adverse effects.
I live in a sunny place, so I'm never outside without wearing my sunglasses. As you've pointed out they've saved my eyes from traumatic injury at least a dozen times over the years.
I wear safety glasses when I'm working around the house with anything that could be considered a power tool (kitchen mixer, drill, etc..) and those have saved me a few times as well.
But not needing glasses, now that could be a lifesaver. I have a close relative who is basically blind without his glasses. He's told me that if he's in an unfamiliar place and is woken up by the fire alarm, there's a good chance he can't find his way out without his glasses.