A who's who of Cardassian officers and Bajoran collaborators (Darhe'el, for example) were persona non grata at best, and arrested to face charges of war crimes at worst. I can imagine that, if there were no specific incidents which could be linked to Gul Dukat, perhaps he and other Cardassian officials would be tolerated... But as he was the head of the occupation, I'm not sure this makes sense.
Is the best explanation that this is merely a matter of convenience to normalize relations between Bajor and Cardassia, or is there a plausible justification for his semi-frequent visits to DS9 and/or Bajor in the early seasons? What real historical examples are analogous to his relationship with the Bajoran (provisional) government?
The political situation in the region is incredibly complicated - if Bajor is interested in prosecuting, they risk a renewed conflict with Cardassia, which neither they nor the Federation particularly want. On top of that, many Bajorans are skeptical of the Federation and its intentions, at least in the early days.
Plus...did Dukat ever "visit" DS9 (as in, physically board the station) without explicit authorization from the command staff?
The Federation also has no party in the fight. It's not implausible that if Bajor were to kick off a new conflict with Cardassia over DS9, they might just leave to avoid intervening in the conflict, and then the Bajorans would be in a pickle.
Yeah, at least in the beginning, Dukat only ever lurked nearby, menacingly. The first time he set foot on the station was to help Starfleet deal with what turned out to be the beginnings of the Maquis, during a conflict in the DMZ. That was also incredibly complicated. (Edit: and everyone was pretty pissed that he was the one Cardassia sent).
Everything on DS9 was incredibly complicated. That’s why I liked it so much. 
First time he steps foot back on the Station was in Cardassians when he’s there as a representative of the Cardassian government. Would have caused quite a stir if he was arrested then.
More of a stir than Darhe'el, who they were willing to arrest? (Though obviously that situation was much more complicated in light of who he actually turned out to be, it just serves as precedent)