The ads are added in the app. If you cast, the Chromecast can't add apps (yet) so they'd have to make ad streams instead, and switch between the streams show-ad-show which would take several seconds of loading screen each way and so on. Which is a level of fuckery even they shied away from.
Kind of makes sense - casting is usually a webstream without extra dev effort. I'm not sure if you still can, but I used to circumvent twitch ads by casting from my phone. Not to excuse their shitty behaviour; I gave up on them years ago and started hosting my own content.