Yes, you read correctly, the 535 folks whom 80 percent of Americans dislike make these momentous decisions.
One thing I worry about is a contingent presidential election. That situation arises when no candidate gets a majority of electoral votes (270 of 538). Should this situation arise, Congress gets to pick the next president and vice president.