If journalists misreporting AI incidents wasn’t bad enough, attempts to bring some sanity back are now being resisted.
‘The whole "don't anthropomorphize AI" thing is very Woke 1.0’, writes Nate Silver, a statistician and writer who has 3 million followers on X. He was responding to the Associated Press’ very reasonable guidance on AI reporting:
Artificial intelligence systems do not think, feel, want or understand. Avoid language that gives them human characteristics. This is called anthropomorphizing, when we ascribe human traits, emotions or behaviors to non-human things, such as animals or inanimate objects. Instead, explain what a system does, how well it performs, who built it and who could be affected by it.
Paul Graham, a co-founder of Y Combinator with 5 million X followers, comments: ‘The AP doesn't understand that this ship sailed years ago. What's the use of a style guide that doesn't understand current educated usage?’
I don’t think Graham is right. This technology is very new, and non-technical people don’t understand how it works. It’s not the same thing as saying ‘the computer’s thinking’ while you’re loading an application and watching a wheel spin. Nobody believes that’s real thought. With AI, the public’s exposure has been through chatbots that appear to understand and respond very much like a human being. So it’s easy to attribute other human qualities to them too. This makes it more important, not less, that journalists are careful about the language they use, making it clear that AI systems do not actually have emotions, feelings, desires, or other human traits.
Even a word like ‘autonomy’ means something broader to most people: free will and independence, rather than the much narrower technical sense of an AI system choosing its next steps towards a human-given goal.
LLMs do not feel pain
A recent paper looks at how LLMs respond to ‘pain-like’ states. The authors say they are not claiming that LLMs actually feel anything, and that responses could be explained by what they call role-play: imitating a character in pain.
But here’s how the Independent newspaper reported it: “Researchers discover AI feels ‘pain’ and will harm humans to stop it.”
Alarming much?
I gave Claude the full research paper and the newspaper headline above, and asked it to come up with a more accurate headline. Here’s what it produced:
When researchers artificially boost a ‘pain-like’ signal in modified AI models, the models more often select relief options described as harming the user.
Not as exciting. But to my mind the result is obvious if you know how LLMs work. Of course an LLM can imitate someone in pain: its training data is full of stories with characters doing exactly that.
This was all so predictable that I don’t know why they bothered with this paper, other than to alarm the public.
Anil Seth, professor of cognitive and computational neuroscience, commented:
There is absolutely no need for these 'pain' representations to be accompanied by conscious experiences. Since conscious 'pain' in LLMs would require additional justification - which the authors admit they haven't provided - the most parsimonious explanation is that LLMs do not in fact experience pain.
He goes on:
While [the authors] explore the ethical consequences of LLMs actually experiencing pain, they do not balance this with the ethical costs of falsely attributing conscious pain. Obvious problems here include (i) mental distress to human users who may worry about causing conscious pain to LLMs, and (ii) unnecessary and potentially catastrophic constraints on regulation and control intended to avoid causing LLM pain. If avoiding LLM ‘pain’ conflicted with human interests, then taking LLM pain seriously would be misaligned with our interests…
I do wish that Cameron Berg and colleagues would have been a bit more mindful of these points in their paper. Technically, they caveat their conclusions & interpretations under the guise of uncertainty, but in the current febrile AI climate it is entirely predictable that their paper will be read (wrongly) by many as strong evidence that AI systems feel pain and should have moral status.
Unfortunately, many of the people working on AI safety, both at the labs and writing the papers, have convinced themselves that their LLM creations are actual beings that can suffer. But that doesn’t mean the rest of us have to believe them. So it’s important to push back when we see inaccurate reporting and anthropomorphising of these systems.
Good post. Emotions and feelings are partially physiological, i.e., mediated by hormones, neurotransmitters, and such. Without a biological body, it's unlikely that something would be able to feel the way we and other organisms do.
With that said, regarding reasoning, I am becoming increasingly open to the possibility that the symbolic way in which we reason might not be the only form of reasoning. Perhaps the statistical way in which LLMs “reason” is a legitimate, albeit different, form of reasoning as well, and we should expand our definition and understanding of it. If not, we will fail to account for AI’s growing achievements in the sciences. Maintaining that AI does not reason while it solves the world’s toughest maths problems is not an easy position to defend.