As a percentage of world population that's comparable to like 477,000 people dying at a football game today. Or possibly 1 million people, depending on what population estimate you use.
The population of Rome the city at the time is estimated at about one million people. So twenty thousand is like 2% - one out of every fifty people in the city died in that stadium.
Why do you find it difficult to believe that humans two thousand years ago could clount? Eratosthenes calculated the axial tilt of the damn planet three hundred years prior.
They could count, but they also tended to enormously overestimate how many people died. See for example this Wikipedia article. There’s also this article which looks like it would be interesting but I can’t get the full text of it:
“I do not think anyone in his senses would accept that!” Remarks on Numbers of Fallen Soldiers in Roman Historiography and commentarii