ME READING WARHAMMER 40K BOOKS IN 2018: this
setting is so fucking dumb. No way a society would let their
entire technological infrastructure be controlled by a handful
of fanatics that have convinced themselves that computer
code is literally God
Except they don't use AI in WH40K. They use cloned human brains for anything that can't be achieved with basic electronics. I'm pretty sure the AdMech actually consider AI a form of heresy against the Machine God and the Omnissiah.
They do bionically enhance the human brain with Neuralink-style interfaces, and even go as far as entombing a person in a pod of brain fluid to augment their interface to one specific machine. But full on AI is not kosher AFAIK
The Machine Spirit is (mostly) an abstraction. The AdMech as a whole is a collection of tech support geeks whose documentation and manfiles were converted to hymns 10,000 years ago. Very few members actually have any significant understanding of what they're doing anymore. They just know that if they speak the Holy Words of System Reset and then poke the button with a circle on it, the machine spirit dies and resurrects anew.
So, mostly, no, AI has been well and truly eradicated in 40k and most of what is referred to as "Machine Spirit" is just semi-basic technology being crabby like it already does in the modern day. When a modern programmer says his system is "thinking" or is "upset" he doesn't mean it literally - AdMech just forgot that language wasn't literal.
That said, there are a few suspiciously intelligent and well-informed Machine Spirits in a few places. A couple of them can outright hold a conversation.
Plenty of machines have AIs that are either hiding, hibernating, or degraded. There are also plenty of VI programs that are just very complex and sometimes will function in ways their users don't understand.
The rest... Well, belief matters when your species is made up of latent psychics. The best way to get a helpful machine spirit is to treat it with respect, so a decent number of machines that had VIs become something more with time. Not only will it empower the spirit, it will prevent it from falling to Chaos...
Well, no. The Imperium places the Emperor above all else. The Adeptus Mechanicus is the only part that idolizes machines. The rest idolizes a corpse on a toilet.