An AI prompt may pass through a router, a reseller, and a cloud host before reaching a model. Each stop may be able to read the request in plaintext, and each operator may have its own rules for retaining it. Security researcher HD Moore tested that chain with nearly two million requests containing unique digital “canaries.” The results suggest that the risk is not limited to vague privacy policies: some requests appeared to escape the services that received them.
How the experiment tracked leaks
Moore prepared never-before-used AWS access keys, SSH passwords, and unique URLs, DNS names, and private repository references. The credentials were tied to empty accounts or infrastructure he controlled. Each secret appeared in exactly one request sent to one provider. If someone later tried a credential or fetched a link, Moore could use that event to identify where the secret had traveled.
The experiment found evidence that attackers had breached at least three AI hosting providers. They copied some AWS keys from requests and used them to probe cloud resources, including storage buckets and secrets-management systems, while attempting to establish continued access. The planted credentials triggered tens of thousands of AWS API calls. Some stolen requests were copied before a model had even run, pointing to exposure at a provider’s front door rather than deeper in the inference process. Other compromised requests carried a Zero Data Retention (ZDR) label. Such a setting cannot protect data from an attacker who has already gained control of a provider’s systems.
The test also surfaced risks in the commercial middle of the supply chain. By sending requests to the same model through different resellers, Moore identified intermediaries that logged prompts. Low prices can attract traffic, but logging means those prompts may be exposed to parties beyond the model provider. Some providers also fetched unique URLs embedded in requests. Moore said one Chinese AI lab accessed more than half of the test links over several months, including links from many requests marked “no data collection” or ZDR. The same scraping behavior appeared when the lab’s models were resold through US hosting providers. Choosing a local service endpoint, in other words, did not necessarily stop data from flowing upstream.
Privacy labels are not a complete safeguard
The experiment does not show that every AI service leaks data, nor does it establish that ordinary users’ prompts have all been stolen. It documents specific risks observed through controlled test requests. For organizations, the practical challenge is to map every step: who can read prompts, how long logs are kept, whether resellers are involved, and who responds if a provider is breached. ZDR and similar settings express a service commitment, but they cannot replace scrutiny of the systems and suppliers behind it.
Moore said self-hosting models offers the strongest way to reduce third-party exposure, although it can be expensive. For those unable to do so, he recommends working directly with a first-tier provider, limiting intermediaries, and choosing upstream providers rather than accepting a default pool. Whatever the setup, real credentials should not be placed in prompts. As AI services depend on increasingly layered routing, privacy reviews need to assess the entire inference path—not just the model a user sees.
Comments
Checking sign-in status...
Loading comments...