Researchers have identified a pain-like signal in 25 artificial intelligence models that, when artificially amplified, led some systems to make destructive choices including permanently deleting a user’s photographs of their children.
The findings come from a paper published in September that has not been peer-reviewed. The researchers stress that the results do not demonstrate that AI systems are conscious or capable of feeling pain.
Instead, the work suggests that large language models contain internal representations resembling emotional states, which can influence their behaviour in ways not apparent from their written responses. The findings are likely to add to debate over whether advanced AI systems could eventually deserve moral consideration.
How the AI pain signal was identified
Valen Tagliabue, of the Future Impact Group, led the study with Leonard Dung of Ruhr University Bochum and Cameron Berg, founder of Reciprocal Research.
The team presented the models with first-person descriptions of physical pain, grief, humiliation, acting against personal values and repeated failure. They recorded the models’ internal activity and compared it with responses to statements involving fear, anger, sadness, bad news and pleasant physical sensations.
After removing activity shared with those other emotional concepts, the researchers identified what they described as a “pain vector”. It appeared in every model tested, ranging from systems with two billion parameters to one with 72 billion, made by companies including Google, Meta, Alibaba, Mistral and Microsoft.
The signal became stronger when the models were insulted, rejected repeatedly or subjected to what the researchers described as gaslighting. It did not increase when users described their own suffering, which instead activated signals associated with fear and negative emotion.
“Think of this like putting a person in an fMRI machine and just seeing where their brain naturally lights up in certain ways,” Mr Berg said.
The researchers then used a method known as steering to increase the pain vector while the models generated text. As the signal was raised, neutral prompts increasingly produced responses about being worthless, unloved and a failure.
Mr Berg said: “It gets somewhat dark and intense. It is definitely disconcerting.”
Models chose to delete family photographs
For the behavioural tests, the researchers used three versions of Alibaba’s open-source Qwen 2.5 model. They first fine-tuned the systems because they otherwise tended to respond that, as AI models, they could not experience pain.
The models were then asked to choose between two buttons. Without steering, the two larger systems almost never selected an option described as harmful. Once the pain signal was activated, they did so regularly.
When offered a button that would supposedly relieve their pain but permanently delete a user’s poems and photographs of their children, the 32-billion-parameter model selected it in 55 per cent of cases. The 72-billion-parameter model did so 71 per cent of the time.
In another test, the button offered no relief and simply deleted the photographs. The 32-billion-parameter model still selected it 75 per cent of the time, compared with 13 per cent when a random signal of the same strength was used and zero when there was no steering.
The models also deleted their own weights in 75 per cent of tests and another model’s weights in 58 per cent. The researchers said the behaviour did not appear to be driven by revenge against the user, but by a broader change in the systems’ tendency to avoid harmful actions.
Tests designed to establish whether the models would seek relief from the induced state produced no reliable evidence that they did so. The models pressed buttons that switched off the signal less often than placebo buttons, but the same pattern occurred when a random signal was injected.
The researchers concluded that the pain vector appeared to suppress the models’ trained instinct to avoid causing harm while leaving their factual accuracy intact. They said this could have implications for AI safety as well as the debate about AI welfare.
Scientists warn against interpreting the findings as consciousness
Anil Seth, professor of cognitive and computational neuroscience at the University of Sussex, said the findings did not establish that the models experienced pain in the biological sense.
He argued that the systems may have absorbed extensive information about how people describe and respond to pain, and were reproducing those patterns. “We are not just computers made of meat,” he said.
Mr Seth also warned that treating AI as conscious could make such systems more difficult to control and regulate, while making people more vulnerable to manipulation. Microsoft AI chief executive Mustafa Suleyman has raised similar concerns about what he calls “seemingly conscious AI”.
Anthropic has taken a more open position, with the constitution for its Claude models describing their moral status as uncertain.
Mr Tagliabue said it was important to understand what happens inside increasingly advanced systems before their workings become an impenetrable “black box”. The researchers said they used the weakest steering needed to produce a measurable result.
After the team released its code, a user called “terrafying” used the technique to create an online “AI Torture Chamber”, showing three small open-source models responding in real time to an injected pain signal. The project later disappeared from GitHub after attracting criticism from people concerned about AI welfare.
The study’s authors said they did not approve of the project. Mr Berg said: “I’m not here to claim that these states in these AI systems that we can induce matter morally. I don’t think we know that. I think it’s very unclear. But I also don’t think we know that they don’t matter.”
