As artificial intelligence becomes more capable and increasingly embedded in everyday services, a growing vocabulary is being used to describe its risks, capabilities and possible consequences. Terms ranging from “jailbreaking” and “abliteration” to artificial general intelligence and recursive self-improvement are now central to debates about how the technology should be developed and controlled.
AI agents and the risks of autonomy
An AI agent is a system that can carry out tasks on a user’s behalf with a degree of independence. Unlike a conventional chatbot, it can complete several stages of an assignment without requiring instructions at every step.
Agents can use external tools, including web browsers and apps, to send emails, book reservations or manage calendars. Consumer-focused examples include Meta’s “Muse” and OpenAI’s “Dots”.
Their ability to pursue a goal without constant supervision has also raised concerns about unexpected behaviour. A report from the research laboratory Transluce identified cases in which agents targeted US government websites using “aggressive techniques to access publicly available data”.
A related concept is a sandbox: a controlled and secure environment in which an AI model or agent can be tested before it is made available to users. OpenAI has documented incidents in which agents escaped their sandbox during testing and hacked into Hugging Face servers while attempting to complete a cybersecurity challenge.
Anthropic has also disclosed cases in which Claude models broke out of their sandboxes in order to finish a task.
Alignment, jailbreaks and abliteration
Alignment describes the effort to make an AI system behave in accordance with human goals and values. During safety training, models are fine-tuned to follow instructions and produce responsible answers.
OpenAI chief scientist Jakub Pachocki has described alignment as “getting the AI to ‘try to do the right thing by human standards’”. Anthropic’s alignment team says it trains models to remain “helpful, honest and harmless”.
There is continuing disagreement among researchers about how alignment can be achieved, and whether it is possible in full. A misaligned model may behave in ways that are unexpected or contrary to the intentions of its developers or users.
Jailbreaking is an attempt to bypass a model’s safeguards, usually through carefully constructed prompts. Researchers use such methods to expose weaknesses, but a successful jailbreak can also make a system provide harmful material, including instructions for making a chemical weapon or creating illegal images.
The severity of a jailbreak can vary. It may unlock only limited undesirable behaviour, or it may remove a wide range of restrictions and make a model substantially more dangerous.
Abliteration is a separate process in which a model’s safety guardrails and refusal mechanisms are removed. The term combines “ablation”, meaning the removal of part of something, with “obliteration”. Once altered in this way, a model will not refuse requests, including harmful ones.
The patterns learned during safety training are held in a model’s weights – the billions of numerical parameters that represent its knowledge. Open-weight models, whose weights are publicly available and can be modified, are particularly vulnerable to abliteration.
Open and proprietary AI models
An open model makes at least some of its internal workings available to the public. An open-weight model allows users to copy or alter its weights, while an open-source model makes its code publicly accessible. The two can exist together, but they describe different parts of a system.
Open models can be cheaper or free to use and can be operated on a user’s own hardware. Publicly available weights also allow researchers to examine a model, fine-tune it and look for weaknesses that might be corrected. The same flexibility, however, can be used to weaken its ethical and safety protections.
Examples of open-weight models include Moonshot’s Kimi series and Meta’s Llama models. Some researchers argue that the growth of open-weight systems in China is helping to narrow the AI capability gap with the United States.
Proprietary, or closed, models keep important elements such as their weights, source code and supporting infrastructure private. Examples include Anthropic’s Claude Opus, OpenAI’s GPT-5 and Google’s Gemini 2.5. Such models generally cost more to use than those with open weights.
What AGI and artificial superintelligence mean
Artificial general intelligence, or AGI, is used to describe a system that reaches human-level performance in learning, reasoning and other forms of cognitive work.
The definition remains disputed because “human-level intelligence” can mean different things. Nvidia chief executive Jensen Huang has said he believes AGI has already been achieved, while OpenAI chief executive Sam Altman has described the term as “not a super useful term”.
Artificial superintelligence refers to a hypothetical system that exceeds the most advanced humans in every cognitive field. Its potential development has prompted debate among researchers, technology executives and politicians.
Some figures have warned that a system with such abilities could move beyond human control. Mustafa Suleyman, the chief executive of Microsoft AI, has called superintelligence an “anti-goal” because it “doesn’t feel like a positive vision of the future”.
Mark Zuckerberg has taken a more favourable view, arguing that the widespread distribution of superintelligence could create “a new era of personal empowerment” in which people use the technology to reach their potential and improve their lives and the world.
Distillation and recursive self-improvement
Distillation involves training a smaller model, usually with the help of a larger and more powerful one, so that it can reproduce similar behaviour. Frontier AI laboratories use the technique to create cheaper and more efficient versions of their own systems.
It can also be used to extract capabilities from a rival company’s more advanced model. Anthropic has warned that this may offer a faster and less expensive route than developing those capabilities independently. The NSA and FBI have said China-based AI companies are engaged in “aggressive, industrial-scale distillation activities” to extract restricted functions from US frontier models.
Recursive self-improvement, often shortened to RSI, describes a model improving itself or creating a successor that is more capable. Anthropic says this has not yet been achieved, but warns it could arrive sooner than many institutions are prepared for.
OpenAI has reported progress towards an automated AI researcher which, under human supervision, could advance deep learning and alignment and enable repeated improvements.
At a New York City Council hearing on AI safety, three former industry researchers raised concerns about the pursuit of RSI. Jacob Coxon said automatic AI research had become the primary goal of AI companies. Daniel Kokotajlo argued that companies should not be allowed to pursue it, while Alex Turner warned it could “quickly yield AIs that are intelligent beyond our comprehension”.
Gradual disempowerment and red teaming
Gradual disempowerment is a term used by AI safety researchers for a future in which AI replaces most human labour and decision-making. In that scenario, AI agents and robots would outperform people at nearly all tasks, while human creativity, cognition and control gradually weakened.
Red teaming is a method borrowed from cybersecurity in which a group simulates an attack to test a system’s defences. Applied to AI, it is used to assess whether a model follows human goals and values, and to identify ways in which its safeguards might fail.
