So I took a png and used cjxl v0.10.2 and --distance==0 --effort==9 to convert it to a jxl file. And the result has color inaccuracies.
I checked that both files use RGB. In the Black&White part of the images I cannot see any differences, it seems to really be something about colours o this specific image...