AI models are using material from retracted scientific papers

Of course, because it doesn't understand anything and just spits out what it finds. Maybe it ranks certain sites as more reliable, but retractions, and where something is at in a process isn't taken into consideration.

My very first attempt to use it for work was to ask about the current mill levy exemption and it returned a value from a proposed bill that died in committee in the last session. It also presented that failed bill and then two documents with the correct value so it not only found the right documents that would easily answer the query but presented the wrong one as fact in the text vomit so anyone who didn't look at the documents would assume the text response was accurate.

It cited its sources after all!

2 comments

Of course, because it doesn't understand anything and just spits out what it finds. Maybe it ranks certain sites as more reliable, but retractions, and where something is at in a process isn't taken into consideration.
My very first attempt to use it for work was to ask about the current mill levy exemption and it returned a value from a proposed bill that died in committee in the last session. It also presented that failed bill and then two documents with the correct value so it not only found the right documents that would easily answer the query but presented the wrong one as fact in the text vomit so anyone who didn't look at the documents would assume the text response was accurate.
It cited its sources after all!