The previously fastest ran on Red Hat Enterprise Linux, the current fastest runs on SUSE Enterprise Linux.
The current third fastest (owned by Microsoft) runs Ubuntu. That's as far as I care to research.
I can't imagine Supercomputers to use a mainstream operating system such as Ubuntu. But clearly people even put Windows on it, so I shouldn't be surprised...
They do use Ubuntu, Red Hat and SUSE mostly.
But for customers like that, the companies are of course willing to adjust the distro to their needs, with full support.
Microsoft uses their own Linux distro now.