It's because most people don't actually like driving. They just do because they have to, and have become numb to the thought of it.
And being numb to it means that you only pay the bare minimum amount of attention. There's been stories about people accidentally hitting the gas when they meant to hit the brakes (Toyota's got actual records for that), and don't get me started how many times I started crossing the street, only to almost be run over by someone turning right and was only staring at the lights and not where he was going.
Driving safely requires patience, practice, and dedication. All things 95% of drivers don't want to bother with on something they see little more than as a chore.
This isn't the 60's where pleasure driving was a common thing.
Aging population + necessity of car ownership in most places (even areas of Toronto) = grandma/grandpa hitting the wrong pedal and going through the wall