That's just a pretext. As spoiled rich as he is, I doubt he is surprised to see people fighting back. He's just walking away because people in the administration are realizing how much of a fraud he really is and are stripping any power he had over it.
Given how much of a man child he is, I wouldn't be surprised if he's genuinely someone somewhere doesn't think he's important. He seems like the kind of person everyone hates.
Honestly I'm guessing it's actually that chaotically tearing down the government is too much work and he wants to go back to couch surfing at Xitter while doing drugs and playing video games all day.
also because hes becauses losing favor with trump and his administration, and give less of an involvement, during his meetings. additionally on another lemmy thread, hes just deflecting blame hes getting from the right as well as a "left attacking"