[Paper] ChatGPT is bullshit - Ethics and Information Technology

Abstract: Recently, there has been considerable interest in large language models: machine learning systems which produce human-like text and dialogue. Applications of these systems have been plagued by persistent inaccuracies in their output; these are often called “AI hallucinations”. We argue that these falsehoods, and the overall activity of large language models, is better understood as bullshit in the sense explored by Frankfurt (On Bullshit, Princeton, 2005): the models are in an important way indifferent to the truth of their outputs. We distinguish two ways in which the models can be said to be bullshitters, and argue that they clearly meet at least one of these definitions. We further argue that describing AI misrepresentations as bullshit is both a more useful and more accurate way of predicting and discussing the behaviour of these systems.

Large language models, like advanced chatbots, can generate human-like text and conversations. However, these models often produce inaccurate information, which is sometimes referred to as "AI hallucinations." Researchers have found that these models don't necessarily care about the accuracy of their output, which is similar to the concept of "bullshit" described by philosopher Harry Frankfurt. This means that the models can be seen as bullshitters, intentionally or unintentionally producing false information without concern for the truth. By recognizing and labeling these inaccuracies as "bullshit," we can better understand and predict the behavior of these models. This is crucial, especially when it comes to AI companionship, as we need to be cautious and always verify information with informed humans to ensure accuracy and avoid relying solely on potentially misleading AI responses.

by Llama 3 70B