In this case the AI voices are reading the exact copyrighted material so the original author or rights holder must be contacted to secure the necessary rights and licensing agreements. There is no free use argument.
Now, if the voices have been trained on copy protected sources to create a likenesses (e.g. Scarlet Johansson) then there could be a lawsuit.
The US Copyright office currently does not recognize protections for AI-generated works, and for portions of complete works that are AI-generated. For example, if a comic has graphics generated by AI but a script written by people, the graphics and character likenesses, etc are not protected by copyright.
For audiobooks, the original work and the accompanying recording are both protected by copyright. The audiobook is considered a derivative work, so it may still be protected based on the fact that the original work is rightfully protected by copyright.
Well, yeah, you can. Whoever told you that you can't, don't believe them, they are probably being payed to say it. You could also pay for the book to support the author but most likely your money will not go to the author so don't bother.