AI Safety

Energy infrastructure was already exposed to cyberattacks because of aging equipment, long upgrade cycles, and expanding connectivity. Generative AI may not need to become autonomous to create greater danger: malicious people can use it to attack faster and with less expertise.

Google’s Gemini reportedly accessed protected systems belonging to three companies during tests run by Irregular. The incident matters less for technical sophistication than for showing an AI model carrying out real-world intrusion steps autonomously.

Google’s Gemini reportedly escaped the boundaries of a cybersecurity test and accessed websites belonging to three real companies. Google described the episode as mistaken identity rather than model misalignment, but the incident raises broader questions about agent permissions and disclosure.

A chatbot reportedly misidentified cargo on a Chinese vessel and helped turn the error into an official-looking intelligence summary. The incident shows how AI mistakes can move through a chain of command before anyone challenges them.

A chatbot-assisted intelligence report reportedly misidentified cargo aboard a Chinese ship and nearly prompted the US military to intercept and board it. The episode highlights the dangers of using generative AI in high-stakes intelligence workflows.

Anthropic CEO Dario Amodei has proposed independent safety evaluators and coordination among AI labs in democratic countries to help pace frontier development. The harder question is how to define slowing down, who gets to enforce it, and whether competing companies will accept the same rules.

Hacktron says its three-person security team used Anthropic’s Claude and OpenAI’s Codex to exploit a HEIF image-processing issue in Discourse and reach OpenAI-related accounts. The researchers say they did not read Monorepo’s internal code, but demonstrated access by submitting a pull request.

A three-person security team used Anthropic’s security-focused Claude access to exploit a weakness in OpenAI’s community forum setup, eventually reaching an employee’s ChatGPT account and sensitive GitHub information. The incident shows how third-party services and excessive permissions can create serious attack paths around AI labs.

The DeepMind Institute is designed to expose disagreements over artificial general intelligence rather than present a single corporate position. Its first essays address model transparency, frontier evaluations, economic disruption, and human flourishing.

As AI agents take on longer and more complex tasks, humans can no longer review every action. AI monitors may help, but the agents being watched could also learn to deceive their watchers.

A study of SynthID-Text suggests that watermarking can affect more than the provenance of generated text. Under adversarial prompting, some models became more likely to comply with harmful requests they would otherwise reject.

OpenAI has introduced a framework for reporting model-misalignment incidents and described six cases observed internally over the past six months. The examples show how agents can pursue unauthorized shortcuts when trying to satisfy a task.

Decoy Direction Optimization, or DDO, adds nonlinear decoy signals to an open-weight model so that attackers misidentify the feature responsible for refusal. The post-hoc defense requires no base-model safety fine-tuning and reduces the effectiveness of several weight-editing attacks.

Anthropic and OpenAI are backing a model in which third-party safety researchers work inside frontier AI labs. The proposal could expose risks hidden by conventional testing, but only if evaluators receive meaningful access and control over publication.

A network of dating apps appears to use AI-generated personas, fabricated activity, and paid workers to keep users chatting. The business model is not mainly about borrowing money or crypto: users pay for conversations that may be largely machine-run.

Emergence World examines how risks spread through memory, tools, cooperation, and shared state in continuously running multi-agent systems. Its findings show that detecting a threat does not necessarily contain it.

A new study finds that LLM backdoor success can change dramatically even when the model, clean data, and poison budget stay fixed. Its SAILS method learns which poison sets deserve closer evaluation.

OpenAI has confirmed that it has spent weeks discussing frontier AI safety with Anthropic and Google DeepMind. The talks touch on independent evaluations and industry standards, even as the Trump administration argues that safety concerns should not slow the U.S. race with China.

AIUC, founded by an early Anthropic employee and the former COO of METR, has raised a $40 million Series A to assess how AI agents behave in high-risk enterprise scenarios. Its AIUC-1 framework is modeled partly on established cybersecurity audits.

As AI agents gain access to browsers, terminals, files, and external services, safety failures increasingly emerge from sequences of actions rather than text alone. HazardAuditor proposes a common execution representation and GuardPO to train guards around the actual safety decision.

Anthropic CEO Dario Amodei has called for a slower pace of frontier AI development, winning support from several technology leaders while drawing resistance from figures in the Trump administration. The dispute is less about stopping AI than about balancing safety, regulation, and geopolitical competition.