PIMiner Uses Agents to Automate Prompt Injection Red Teaming

Introduction

As LLM agents become connected to tools, external documents, web pages, and user workflows, prompt injection has become one of the most important security problems in agentic AI. The paper “Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming” from Pennsylvania State University proposes PIMiner, an automated system for finding prompt injection weaknesses in LLM agents.

Key points

• The risk: Prompt injection can cause an agent to follow malicious or conflicting instructions embedded in external content, potentially overriding the user’s intended task.
• Why existing approaches fall short: The paper argues that many strong prompt injection red-teaming methods rely heavily on reinforcement learning. These attacker models can perform well in the settings they were trained on, but may generalize poorly to new target LLMs.
• What PIMiner changes: During training, PIMiner is exposed to a sequence of dataset and target-model pairs. From these experiences, it builds a strategy library from scratch rather than simply optimizing a single attacker model for one target.
• Transfer at test time: Once the library is learned, it can be used directly against a previously unseen target LLM without additional training.
• Low query budget: The system is designed to work with only a small number of queries to the target agent per test sample, with the paper giving 10 as an example.
• Reported results: On IPIArena, PIMiner reaches a 76.2% attack success rate against Gemini-2.5-Pro, 61.9% against GPT-5.1, and 42.9% against Claude-Sonnet-4.5. On AgentDojo, it reports 86.7%, 53.3%, and 40.0% respectively.

Why it matters

The main contribution is not merely a set of benchmark numbers. PIMiner reframes prompt injection red teaming as a process of accumulating reusable strategies that can transfer across models. For organizations deploying agents, this is important because new model backends and new tool integrations often change the security profile of a system.

The results also highlight a broader point: prompt injection remains difficult to defend against, even for frontier-scale models. Different targets show different levels of vulnerability, which suggests that agent security testing cannot rely on one model, one benchmark, or one manually curated attack set.

In practice, systems like PIMiner could help security teams generate failure cases, evaluate agent workflows before deployment, and collect data for improving defenses. The paper does not, based on the provided material, settle questions such as false positives, real-world coverage, or integration with production guardrails. Still, it points toward a more automated and transferable approach to LLM-agent security evaluation.

Source: Hugging Face Daily Papers