My best understanding is that it was a historic right of kings, governors, etc. The idea was to show mercy to those who may have been convicted wrongly, or to people convicted of laws that have been overturned.
Trump is abusing it in ways never done before to payback political supporters.
Just FYI this is not just an American thing. In Italy for example the President of the republic can cancel or amend sentences too, and it does happen relatively often, although not in the same partisan way as in the US (but that's also because the president in Italy is a neutral and representative position).