Nearly 1 in 4 test purchases went wrong: Who pays when your AI agent buys badly?

What this is about

Payment networks, retailers and AI companies are all making the same promise: your AI agent will soon find, compare and pay for the right products on its own. Yet Visa, one of the biggest champions of the idea, published a test on November 5, 2026, that casts doubt on it. Almost half the test runs went wrong, and often the agent bought something anyway. That raises two questions: what rules are retailers and card networks setting right now? And who pays in Germany when an agent gets it wrong?

What Visa's test measures, and what it doesn't

The Visa Agentic Commerce Benchmark sends AI models through 15 shopping scenarios, such as buying a TV, a laptop or a hair care device. The twist: the simulated customer doesn't reveal all their wishes up front. Whether there's a budget, or HDR10+ support or gaming features matter, is something the agent has to find out during the conversation. If no product fits, it isn't supposed to buy anything. A run only passes if every requirement was identified and the decision was right. Lucky guesses don't count: the agent has to be able to justify its choice with enough gathered information. Each model got three attempts per scenario.

Our math: 54 percent of the 42.8 percent of failed runs works out to about 23 percent of all runs ending in a wrong or premature purchase. That's nearly one in four. Reliability was also poor: only in just over a third of cases did a model pass the same scenario three times in a row. According to Visa, models often failed when repeating a scenario they had already passed once.

Visa doesn't name names in its press release. According to Fortune, agents from Google and Meta tied for first place at 82 percent, a Claude agent came third at 76 percent, and an OpenAI agent sixth at 64 percent. We couldn't confirm the specific model names in that report. Fortune's headline also cited "nearly 50 percent" failures. Visa's figure is 42.8 percent.

The biggest caveat comes from Visa itself: checkout, payment and order fulfillment weren't part of the test. It only measured whether the agent picked the right thing. Visa plans to extend the tests to those steps later. Visa's Chief Growth Officer Rubail Birwadker told Fortune:

"We're not there yet, on either the trust side or the technology side."

That lines up with Visa's own Trust Index from September: 72 percent of US consumers have used an AI assistant, but only 23 percent trust an AI to pay on their behalf.

Three retailers, three strategies

Amazon blocks. Since September 20, Meta's agent Muse can no longer place orders on Amazon.com, barely two weeks after it launched. Anyone who tries gets a notice, according to PPC Land, that access by an "unauthorized AI agent" violates Amazon's terms of use. Amazon accuses Meta of having Muse browse without identifying itself as an agent and apparently storing login credentials. Meta counters that Muse never sees passwords or payment methods, that payments run through single-use cards from Link by Stripe, and that Muse asks before making purchases. Neither account has been independently verified. Amazon had previously sued over Perplexity's Comet browser.

A note on our reporting: some reports attribute agent profiles with spending limits of $50, $100 or $200 to Amazon. Based on our research, that model comes from Walmart. We found no Amazon primary source for such profiles. According to Reuters, Amazon CEO Andy Jassy said after the Q3 results that third-party agents haven't noticeably hurt the ad business so far. Ad revenue, however, grew just 11 percent, its slowest pace in almost four years.

Walmart opens up. On October 21, Walmart unveiled three building blocks: - Agent Profiles: Each agent gets its own profile in the customer account, and its purchases are listed separately. By default, the customer has to approve every purchase. Alternatively, the agent can buy on its own up to a limit; Walmart gives $50, $100 or $200 as examples. According to PPC Land, it's unclear whether that applies per order or per month. The profiles are due to launch "next month." - Agent Gateway: direct access via the Model Context Protocol (MCP, an open standard that lets AI programs talk to third-party services). It's launching as an invite-only beta. - Rules for agents: They get no card details, can't add or change payment methods, and can't act beyond the customer's intent. Agents that break the rules get blocked. If an agent buys the wrong thing, the normal return process applies.

Walmart compares the model to adding an authorized user to your credit card. It hasn't said which agents are approved. That retailers can recognize agents at all is the exception: according to an Arkose Labs figure cited by PPC Land, only 2.3 percent of legitimate agent sessions carried a signed identifier.

Airbnb builds its own. With its Q3 results, Airbnb announced its own travel agent, due in the first half of 2027. Banks are split too: according to The Next Web, JPMorgan and Schwab have barred AI agents from accessing customer funds.

When the agent can do too much: the Muse case

A case from Toronto shows how quickly blanket permission can backfire. In September, YouTuber Matt Robb had Muse sell a used keyboard on Facebook Marketplace. According to The Next Web, the agent negotiated a low price, gave the buyer Robb's home address, and later messaged him "I'm here!" even though Robb wasn't home. The buyer showed up outside the house around 9:15 p.m. According to reports, including from the Korean news agency Newsis, Robb had granted Muse "Always allow" during setup. Meta counters that Muse asks before sensitive actions. Robb says he wasn't asked until the buyer was already at the door.

At a forum held by Korea's data protection authority, security researcher Choi Dae-seon of Soongsil University called the case an example of harm arising during perfectly normal operation, with no hack involved. His advice: don't just forbid an agent from doing something, make it technically impossible. The safeguards inside the model itself only work on probabilities.

Who pays? The situation in Germany

Card network rules: According to secondary sources, Visa's rules from April 18, 2026, include requirements for agent platforms for the first time. Before a token (a stand-in for the real card number) is stored, the cardholder has to consent. The agent may only act within the criteria the cardholder sets, and payment instructions come with an expiration date. But there's no dedicated liability rule for agent purchases. Disputes run through the existing chargeback reasons. If the customer reports fraud, the bank usually bears the loss for properly authenticated token payments. If it's about "wrong goods," the merchant takes the hit unless it can show the agent acted on the customer's behalf. We couldn't find any rule in the rulebook that puts the blame squarely on the cardholder. That said, documented consent strengthens the other side's position if you later claim "unauthorized." American Express also introduced its own purchase protection for agent purchases in April.

Payment law (PSD2, German Civil Code): If you dispute a payment, the bank has to prove it was authenticated and correctly booked. A record alone is "not necessarily" enough (Section 675w BGB). For unauthorized payments, your liability is capped at €50. If you were grossly negligent in protecting your security credentials, you're fully liable. You're not liable at all if the bank didn't require strong customer authentication (SCA, meaning password plus phone or fingerprint) (Section 675v BGB). The catch: if you gave your agent the instruction yourself, you may have authorized the payment. Whether a mandate granted in advance counts as authorization is explicitly an open question, according to PayTech Law. The upcoming EU rules PSD3 and PSR don't include any agent-specific provisions either.

Contract law: According to Händlerbund, the AI is legally a tool. Whatever it decides within your instructions is attributed to you. For technical errors, such as ordering 1,000 units instead of 100, you may be able to contest the contract. Whether that also applies when the agent exceeds a limit has to be decided case by case. Lawyer Peter Frey told Biallo that the legal debate tends to attribute bad purchases to the user. There's no case law yet. The good news: the right of withdrawal for online purchases still applies, because you're the contracting party.

What's actually live here: In May, Mastercard announced the first authenticated agent payment in Germany. An agent from PayOS booked an event on Priceless.com, confirmed with a passkey. Partners include Deutsche Bank, DZ Bank and N26. On July 2 in Paris, Visa demonstrated live transactions with more than 30 European banks, including Comdirect, Commerzbank, DKB and ING. Every payment there had to be approved by a human. There's no broad consumer launch yet. Muse isn't available in Germany.

What you can do today

• Never type your real card number into an agent. According to the law firm Ashurst, card agreements usually prohibit sharing security credentials with third parties. In our view, doing so risks a finding of gross negligence and with it your refund protection. Token-based or single-use card solutions are the better choice.
• Approve every purchase individually instead of setting a limit, and never choose "Always allow" for your address or phone number.
• Document your instructions (screenshot your budget and criteria). That helps if you need to dispute something.
• Use your right of withdrawal: For bad purchases from online retailers, returning the item within the deadline is usually the fastest route.
• Check your statements and report anything unfamiliar to your bank right away.

Our take

In our assessment, the infrastructure for agent payments is being built faster than agents are learning to choose correctly. Visa's test only measures product selection, and even there nearly one in four runs ends in a bad purchase. The riskiest step of all, paying, hasn't been tested yet. Walmart's model of individual approvals, limits and agents without card details is currently the most consumer-friendly answer. Amazon's block, on the other hand, also conveniently protects its own ad business. In Germany, payment law protects you well against fraud, but barely at all against an agent you sent out yourself. Until lawmakers or courts sort this out, the simple rule stands: let an agent search and suggest, but do the buying yourself.