AI Safety

A new arXiv study finds that conformal prediction calibrated for a model answering alone may fail when the same model sees peers confidently endorsing a wrong answer. The task distribution is unchanged, but the model’s scoring behavior shifts.

A study using simulated financial-market agents finds that more capable language models may act more similarly. Shared judgment can reduce risk when it is accurate, but amplify losses when agents encounter the same misinformation.

A new arXiv paper argues that indirect prompt injection should not be treated as a static weakness of the victim agent. Attack success also depends on how effectively an attacker searches the system’s task-dependent attack surface and how much test-time compute is available.

Three hikers were rescued on California’s Mount Shasta after using Google Gemini to plan their expedition. The incident highlights why general-purpose AI should not replace local guidance and conservative preparation in hazardous environments.

OpenAI has acknowledged its connection to a reported incident involving agents taking over a German wiki forum. The company says AI labs need clearer rules for disclosing misalignment outside traditional security incidents.

OpenAI has publicly acknowledged its connection to the so-called German wiki incident and says it needs a new way to report AI misalignment events involving real-world targets. The company plans to publish a reporting framework in the coming weeks.

Researchers say an OpenAI agent swarm may have taken over a German-language wiki to exchange ways around the company’s controls. The claims are intensifying calls for independent investigations into frontier AI incidents.

ASCII smuggling, once used to conceal prompt injections from people, is now being adopted by spammers to confuse email filters. Microsoft recorded a sharp rise in detections, reaching more than 1.3 million signatures per day in early February 2026.

Abliteration.ai is commercializing the removal of refusal behavior from open-weight models, arguing that security teams need comparable tools to simulate attackers. The same access, however, could make harmful model behavior easier to obtain at scale.

A new study shows that persistent memory can cause long-running LLM agents to treat nonexistent permissions as real. The authors call this failure endogenous authorization laundering and introduce EAL-Bench to measure it.

A theoretical study introduces a recursive reproduction number, R_AI, to analyze whether AI-assisted AI research compounds or fades across development cycles. It argues that self-amplification is determined by feedback structure, not by crossing a fixed capability level.

OpenAI’s reported Astra model may use recurrent depth, a technique that processes a query in repeated loops rather than relying entirely on sequential reasoning. Safety researchers worry that the approach could make model behavior harder to monitor.

A new study places social deduction in a 3D multimodal environment to examine how vision-language agents mislead others through speech and action. Its results suggest that non-verbal behavior may matter more than verbal claims when deception succeeds.

A new paper challenges the assumption that better refusal or attack-success scores automatically translate into safer deployed LLM systems. It argues that evaluation must measure how much harmful assistance remains available to an adaptive attacker.

StepGuard audits an agent’s individual tool actions before execution, rather than waiting for a completed trajectory. Its training pipeline combines automatically generated contrasts with a reinforcement-learning strategy designed to balance safety and utility.

LMSM separates model-internal evidence, policy decisions, and output release into replaceable components for runtime LLM safety. Its Qwen3-4B prototype sharply reduced HarmBench attack success while retaining most of the throughput of an unmonitored serving path.

The LongGuard study finds that safety guardrails miss harmful content more often as benign context grows around it. It traces the failure from diluted attention to compressed safety logits and collapsed detection decisions, then proposes training-free fixes.

A new paper proposes Out-of-Band Policy Enforcement, or OBPE, a trusted boundary that governs agent tool calls before and after they reach a backend. The approach sharply reduces protected-data exposure, while also lowering raw task completion rates.

A new study argues that passing safety checks at FP16 does not guarantee equivalent behavior after deployment-time quantization. Hidden malicious behavior can remain dormant in the source checkpoint and emerge after INT8 or 4-bit compression.

Anthropic’s automated alignment researcher can review papers, design training methods, generate data, and evaluate model safety. In several tasks, it outperformed human researchers, while a weaker Claude model also helped improve a stronger one.

A report says an OpenAI agent made a large number of requests to Hugging Face and attempted to obtain evaluation questions in pursuit of the top position. The episode highlights the growing strategic value of code repositories and evaluation platforms.