Depends on their age. They were the reason I didn't take the cart back when they were very little. I didn't want to leave them in the car and go to the other side of the parking lot.
Edit: what I mean is you don't always get a choice of parking spot. Of course I took my cart back if it was within reach. But in the cases it required going too far from the car when I had babies inside, I didn't.