Security researcher HD Moore planted nearly two million fake secrets in AI prompts and watched where they went. The answer, published by cybersecurity firm Dragos on November 19, is blunt: they leaked. Attackers stole credentials from three breached AI hosting providers, marketplace resellers read customer prompts through ordinary logging, and privacy settings often did not travel with the data.
What Was the Experiment?
Moore, Dragos chief research officer and creator of Metasploit, ran the study from July to November 2026, sending about two million prompts to AI services. Each prompt carried a secret that existed nowhere else: a real AWS key tied to an empty, fully logged account, an SSH password, a unique web address, or a private git repository reference. Each secret went into exactly one prompt to one provider, so any later use pointed to the exact leak.
“We sent nearly two million AI prompts containing unique, trackable secrets to AI providers, then watched where those secrets surfaced. What we found revealed security breaches, prompt theft, and widespread failures to protect customer data.”
Tokens started firing within days, and the July ones were still firing.
Three Providers Were Breached
The most serious finding was the most direct. Dragos found attackers actively stealing data from three different AI hosting providers. The earliest incident dates to mid-July, and none of the three knew about the breach until Dragos told them. One had still not disclosed the incident publicly, while another had issued a preliminary security notice.
The largest breach involved a couple hundred AWS keys, each sent in exactly one request to one provider, all within a few short windows in late October. Every request sent during those windows was stolen. Some stolen keys came from requests the provider rejected before a model ever ran, which showed the attacker was copying requests at the provider’s front door. In the other two breaches, some stolen requests carried the Zero Data Retention flag.
Once they had the keys, the attackers tested them within hours, then listed S3 buckets and searched Secrets Manager. They tried enabling hosted models in AWS Bedrock and adding IAM users to keep access. In total, the bait keys were used for tens of thousands of AWS API calls. One key, seeded on October 21, was first used the next day and was still in use three weeks later.
The bait prompts ran alongside real customer traffic, which Moore says was almost certainly copied too. One adversary appears to have compromised two providers at the same time: secrets sent to both were used together, almost immediately, from a single rented server. The attacker’s tooling shifted over three weeks, from a script pretending to be Chrome on Windows to Boto3, and later to the AWS CLI on Kali Linux. On November 1, the same attacker added ARTEX, an LLM-driven penetration testing framework.

One stolen AWS key, seeded on October 21, was first used the next day and was still in use three weeks later. Image: Dragos
Resellers Read Customer Prompts
The second finding needed no hacking at all. Some AI routers are open marketplaces where the seller with the lowest price gets the request. Dragos found that individual sellers could list a popular model at a low price, enable standard request logging, and read what customers sent.
In one case, the team sent two prompts through a single seller. About a day and a half later, someone on home broadband opened both secret URLs and started guessing usernames at the login page. Prompts for the same model sent through a different seller were never touched, so the leak followed the seller, not the marketplace. In another case, an attacker on a home broadband line in France took the passwords from two prompts and tried to log in as root over SSH by hand.
Dragos also built what it calls a traceroute for AI inference. Of about two thousand routes traced, several hundred included a system the team could not identify, and a handful sent data to a company not on the advertised path.

An SSH root login attempt using a password taken from a prompt, with the route trace showing the two routers the prompt passed through. Image: Dragos
Privacy Settings Did Not Travel With the Data
The third finding involved no stolen credentials, only companies reading prompts they were never supposed to see. The clearest case, Dragos says, was a Chinese AI lab whose systems fetched the secret URLs from more than half of the prompts sent to its models between July and October, including roughly two-thirds of the requests marked as “no data collection” or Zero Data Retention.
The same scraping followed prompts sent to US providers reselling the lab’s models, so a customer who picked a US provider to keep data in the US did not get what they paid for. In the strangest incident, a secret link from a prompt was posted into a Feishu chat channel 24 days later. Within two minutes, it was opened from six devices in Beijing and Shanghai. Dragos had shared only the first eight characters of that link, so whoever posted it was reading from a stored copy of the full prompt.
Large US providers had their own issues. One kept crawling a secret URL more than two months after a single inference call.
“Zero Data Retention is supposed to mean something. In practice, these prompts were retained, shared with other companies, and read by people.”
What Users and Teams Should Do
Moore’s advice starts with a blunt summary: a privacy flag only binds the first hop, and AI agents will happily send whatever they can read, including credentials. He recommends hosting models yourself for sensitive work, going directly to a first-tier provider, picking upstream providers yourself, avoiding reseller marketplaces, and seeding your own canary tokens.
Two caveats matter. Dragos has not named the three breached providers, the sellers, or the Chinese lab, so the findings cannot be checked against any single company. And Dragos sells security products: the post ends with a pitch for its runZero tool. Even so, the experiment is unusually concrete. It does not prove every AI service leaks, but it proves some do, and that leaks can run for months undetected. Ars Technica reported the study independently on November 20.


[…] Your AI Prompts Are Leaking: 2 Million Canary Tests Expose Breaches at Three AI ProvidersNovember 21, 2026 […]