AI coding agent leaks a secret key, exposing Together AI customer prompts

At AI cloud provider Together AI, an AI coding agent pushed a secret access key to a public GitHub project. Someone used the key to download stored customer prompts from June and July 2026, then combed through them for more passwords. The incident shows how risky it is to let AI agents work on your code without strict controls.

Industry Nov 21, 2026 6 min read Source: Together AI
Illustrative image, AI-generated

What's going on

Together AI is a US provider that lets companies and developers run open AI models in the cloud through an API. On November 21, 2026, it disclosed a security incident: an unauthorized party downloaded samples of prompts, meaning the text customers had sent to the service's AI models. The data comes from requests made in June and July 2026.

The cause is what makes this case stand out. According to Together AI, an AI coding agent (an AI program that writes code on its own and commits it to projects) uploaded a secret access key to a public GitHub project. That key is how the attacker got to the data. The disclosure is signed by CEO and co-founder Vipul Ved Prakash, and the company apologizes for the incident.

What exactly happened

Together AI runs a serverless inference service. Customers send requests to a model and pay per use, with no need to rent their own servers or GPUs. Together AI says it collected prompt samples for software testing and that their use for this purpose was "authorized." The disclosure doesn't say how that permission was obtained, for example through the terms of service or account settings. The samples were stored in a private repository on Hugging Face, the big platform for AI models and datasets.

The samples come from requests to three models:

Model Affected days (mainly)
Minimax M3 June 24, July 4, 8 and 14, 2026
GLM 5.2 same as above
Kimi 2.6 same as above

Together AI doesn't say which data belongs to which day or which model.

Then came the mistake. While working on the public GitHub project xorl, an AI coding agent created a commit, a saved change to the code. That commit contained a credential that granted access to the private Hugging Face repository. Together AI's automated scanner, which checks code for accidentally published secrets, caught the key, and the team reverted the commit.

That wasn't enough. During the exposure window, the period when the key was publicly visible, an unauthorized party grabbed it and downloaded the prompt samples. Together AI doesn't say how long that window lasted. One more important detail: reverting a commit doesn't automatically remove the key from the version history, and it doesn't invalidate it either. The disclosure also doesn't say whether or when the key was revoked.

The attacker went hunting for more passwords

The attacker didn't stop at downloading the data. According to Together AI, they deliberately searched the prompts for credentials to other services and checked whether those still worked. That fits a well-known problem: people using AI models often paste config files, code snippets or log files into their prompts, and those sometimes contain API keys or passwords.

Five of the credentials the attacker tested were canary credentials. These are decoy credentials planted on purpose that raise an alarm the moment someone uses them, much like canaries once warned miners about gas. They belonged to a security experiment run by Dragos, a firm that specializes in securing industrial systems. The disclosure doesn't say whether Dragos detected the misuse this way or whether the incident came to light some other way. It also leaves open whether the attacker found and used real, valid customer credentials alongside the decoys.

Investigation and scope

Together AI says it has brought in an outside forensics firm, specialists who reconstruct attacks technically. Law enforcement has also been notified. The company says there's no evidence of any other prompt leaks beyond this incident. It also has "no reason to believe" the attacker shared the data and assumes it has since been deleted. Together AI doesn't explain how it reached that conclusion.

The review of the content isn't finished yet. Together AI is still investigating whether the leaked prompts contained sensitive personal data or other customer data. The company says it will contact affected customers directly if needed.

The timeline is worth noting. The data dates from June and July 2026, and the disclosure came out on November 21, 2026. The company doesn't say when the attack itself happened or when it found out about it.

What Together AI is changing and what customers should do

In response, the company says it has tightened its controls around development workflows, credentials and customer data:

  • Scoped or short-lived credentials, and keyless access where possible. A key with limited permissions, or one that expires quickly, does less damage if it leaks.
  • Credential scanning on every pull request. Pull requests are proposed code changes reviewed before they're merged. The goal is to stop secrets from ever going public.
  • Zero Data Retention (ZDR) by default for new accounts. With ZDR, the provider doesn't store prompts and responses beyond the request itself.
  • New rules for using AI coding agents in development. Together AI doesn't describe what they look like.

Existing customers who send sensitive data through the service should turn on ZDR in their account settings themselves. For questions, Together AI points to security@together.ai.

Analysis

The incident highlights a risk that grows as AI coding agents spread. These tools often write and commit code faster than humans can review every change, and they can't reliably tell harmless config values from secrets. The fact that an AI company of all things is publicly admitting this kind of mistake makes the case a lesson for the whole industry. The underlying error is an old one, though: secret keys don't belong in code repositories. And a scanner that only fires after publication is too late when attackers scan public repositories for keys automatically.

It's also worth criticizing that Together AI kept real customer prompts stored for testing at all, and that ZDR wasn't the default until now. For you as a user, that means whether a provider stores your prompts often depends on settings you have to check yourself. And credentials, passwords or personal data should never go into a prompt, no matter the provider.

Plenty of questions remain open: how many customers and prompts are affected, how long the key was public, whether real customer credentials were misused, and what the assumption that the data was deleted is based on. So far, all of the information comes from Together AI alone. There's no independent confirmation yet.

Sources

#security#data leak#together ai#coding agents#privacy
Read the original source

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.