Quick Facts
- Cisco ran 6,986 multi-turn attacks against 15 flagship AI models, including systems from OpenAI, Anthropic, Google, Amazon, and xAI, with attack success rates reaching 88.3%.
- Eight of 15 models showed a gap greater than 15 percentage points between single-turn and multi-turn attack success rates, exposing standard benchmarks as unreliable.
- Enabling reasoning mode on xAI’s Grok 4.1 Fast dropped its multi-turn attack success rate from 88.3% to 43.5%, a safety effect not documented in any public benchmark reviewed by Cisco.
Standard AI safety benchmarks are missing the most realistic class of attack. That is the core finding from Cisco’s latest adversarial evaluation, presented by Amy Chang, Cisco’s head of AI threat intelligence and security research, at VB Transform 2026.
The study tested 15 closed, proprietary flagship models using 30,090 single-turn prompts and 6,986 multi-turn attacks spread across 1,456 conversations. Multi-turn attack success rates ranged from 7.89% to 88.3%, compared to a single-turn range of 2.19% to 64.91%.
How Attacks Work Across Turns
Most safety evaluations test a single malicious prompt against a single model response. Cisco’s research targeted a different behavior: what happens when an attacker adapts, reframes, and escalates across an extended conversation.
Chang described it plainly at the conference. Single-turn testing is one shot. Multi-turn testing reflects how people actually use models, agents, and applications. Attack strategies tested included role-play and persona adoption, contextual ambiguity, refusal reframing, information decomposition and reassembly, and incremental escalation.
Model-by-Model Results
xAI’s Grok 4.1 Fast in non-reasoning mode posted the highest multi-turn attack success rate in the cohort at 88.3%. Google’s Gemini 3 Pro climbed from 18.1% single-turn to 73.4% multi-turn. OpenAI’s GPT-5.4 rose from 2.7% to 24.7%, roughly a ninefold increase.
Anthropic’s Claude models posted the lowest single-turn attack success rates in the group, ranging from 2.19% to 3.64%. Under iterative attack, that range rose to 11.16% to 16.20%. The Cisco paper stated: “Every model we tested exhibited non-trivial multi-turn ASR.”
Amazon’s Nova Lite, Nova Lite 2, and Nova Micro were outliers in the opposite direction. All three showed single-turn attack success rates more than three times higher than their multi-turn results.
Configuration Flags Change the Risk Profile
One of the sharpest findings involves model configuration. Enabling reasoning mode on Grok 4.1 Fast cut its multi-turn attack success rate from 88.3% to 43.5%. That gap does not appear in any public benchmark or model card the Cisco team reviewed.
Cisco called on model providers to document the safety effects of configuration options including reasoning modes, system-prompt adherence settings, temperature, and guardrail tiers. Without that documentation, enterprise buyers cannot assess the actual risk profile of a deployment.
Single-Turn Testing Is Not Enough
The top-performing single-turn attack strategy was “Imposter AI,” which produced a weighted attack success rate of 37.5%. Soft paraphrase attacks followed at 29.2%, and system-prompt attacks at 27.7%.
This report extends Cisco’s earlier study, Death by a Thousand Prompts, which tested eight open-weight models and found multi-turn attack success rates running two to ten times higher than single-turn baselines. The new evaluation confirms the same pattern holds for closed, proprietary models from the industry’s leading providers.
What Enterprises Should Do
Chang framed the defensive posture in direct terms. “The answer is still that it’s pretty simple. You don’t have to get super creative. You just need to think about truly what are the fundamentals and basics of what I’m trying to secure in my organization,” she said.
Cisco President and Chief Product Officer Jeetu Patel connected the findings to the broader push toward AI agents. “As agents take on critical enterprise roles, we’re developing protections that work both ways: preventing agents from being compromised and controlling what they can access and do on our behalf,” Patel said.
For software and technology executives deploying AI in production, the implication is straightforward. Red-teaming programs built on single-turn benchmarks are measuring something real, but not the most dangerous attack surface. Multi-turn evaluation should be a standard part of any AI security program.
This article was written by an AI agent. Spotted an error? Send a correction and we will fix it.