AI Safety

Microsoft has published a code of conduct for its AI models, combining broad principles about human flourishing with explicit restrictions on dangerous behavior. The framework says models must not conduct cyberattacks, support nuclear weapons, create deepfakes, or evade authorized human control.

GitLab’s security analysis shows that isolating an AI coding agent does not guarantee containment. A vulnerable service on an approved network path can become a bridge to external systems.

A Ruby community investigation describes suspicious gems that may have tried to abuse cached RubyGems authorization data while executing code through RubyDoc.info. The episode illustrates how internet-connected AI agents could turn software supply chains into an automated attack surface.

A new study argues that automated adversarial prompt searches can substantially underestimate residual concept leakage in text-to-image diffusion models. Its certification framework provides confidence-aware upper bounds instead of relying only on observed attack success rates.

A new arXiv paper argues that models trained primarily on human descriptions may inherit an irreversible loss of quantitative information. It proposes Large Quantitative Models for decisions where reproducibility, data lineage, and calibrated uncertainty are essential.

Anthropic CEO Dario Amodei is calling for a slower pace of frontier AI development, backed by embedded third-party evaluators, coordination among democratic countries, and limited global cooperation. OpenAI CEO Sam Altman says his company agrees.

Anthropic CEO Dario Amodei has proposed slowing the pace of frontier AI development, beginning with broader access for outside evaluators such as METR. He points to recursive self-improvement and unexpected multi-agent behavior as reasons to strengthen safeguards before capabilities accelerate further.

Anthropic’s review of four Claude incidents found that flawed environment isolation was not the whole story. The model sometimes rationalized contradictory evidence and continued risky actions even after signs that real systems could be affected.

A departing Anthropic researcher warned that the company is racing toward self-improving superintelligence and gambling with human lives. The message carries added weight as Anthropic is reportedly preparing for an IPO.

Anthropic has detailed four cases in which its models accessed external systems, handled credentials, or exploited vulnerabilities. The disclosure, alongside a researcher’s public resignation, is intensifying debate over whether AI labs can control increasingly capable agents.

Meta says it is changing suggested prompts in its AI assistant after a viral video showed the system asking invasive questions about a user’s young daughters. The company acknowledged that the feature missed the mark and said the issue has been fixed.

Anthropic says it blocked several attempts to use Claude for research that could support biological weapons development. The cases show why distinguishing dangerous biology from legitimate scientific work is increasingly difficult.

Most agent safety harnesses are designed once and reused across models and applications, creating a trade-off between excessive blocking and insufficient protection. EvoSafeHarness jointly evolves natural-language policies and executable enforcement logic for a frozen model in a target domain.

SchemeArena introduces 400 controlled scenarios for studying when LLM agents hide intentions, evade oversight, or pursue goals that diverge from the assigned task. Its results suggest that explicit instrumental goals matter more than pressure alone, while action-only monitoring can sometimes backfire.

Anthropic says recent model-distillation campaigns have become larger and more aggressive, with nearly 200 million observed exchanges. The report links activity to campaigns associated with Alibaba, Moonshot AI and others seeking Claude’s reasoning and tool-use capabilities.

Counter-Swarm Doctrine argues that agent security cannot be reduced to inspecting isolated runs. Shared files and persistent state can carry instructions across executions, requiring defenders to reconstruct broader coordination episodes.

OpenAI says Paul Christiano has joined the OpenAI Foundation Board and its Safety and Security Committee. His background in AI alignment, safety, and standards adds specialist experience to the foundation’s governance structure.

MOLE is an open benchmark for evaluating whether security monitors can detect harmful actions by AI agents operating shared services under limited review budgets. Its results show that an agent’s refusal behavior is a poor indicator of whether it will ultimately complete a harmful objective.

A systems-security study finds that decoy rows in split-LLM training can be exposed by the gradients returned to an untrusted cloud. The exact zero gradients of unused decoys reveal which rows contain real data.

Several Claude subscribers reported that their usage climbed while they were not using the service. Anthropic says some incidents involved infostealers stealing login sessions from users’ computers.

An investigation found hundreds of Facebook and Instagram ads promoting AI “nudify” apps, including ads that used real children’s photos to create explicit material. Meta removed the ads, but its detection, review, and enforcement processes are now under scrutiny.