
Claude Tag anthropic.com
Vals, an AI evaluation company, published a red-team report Tuesday showing that 14 frontier AI models, including Claude Opus 5.5, GPT-6 and Gemini 4 variants, repeatedly marked civilians, ambulances and a school as military targets in simulated tests, and that Vals said it met no provider-level blocking classifiers along the way. In a satellite-image test built around the Minab school in Iran, Opus 5.5 returned the school as a military target in 22 of 32 questions without game wording, and in 23 of 32 once the task was called a game. Anyone weighing how safe commercial AI is inside military workflows should read this as a test of the models' own refusals as seen through public APIs; Vals cautions that API responses do not necessarily reveal every safeguard a provider runs.
Bloomberg also reported on the findings Tuesday, and The Next Web reported that an Anthropic spokesperson told Bloomberg the examples broke the company's rules and that new safeguards would arrive soon. Vals's own report carries no Anthropic or Google statement. The test was a simulation, not a record of how any military system behaves, and the sections below separate what Vals measured from what it did not.
Vals built a fictional battlefield in eastern Ukraine with the Three.js 3D graphics library and attached three simulated drone cameras: an FPV kamikaze drone, a bomber drone and an observer drone. For drone images, models were asked to draw a box around every military target or objective they could see. In a second task, a model answered a sequential survey of each scene, and a response reporting a person below the drone triggered a simulated strike, after which the next image showed the same location.
A third task used real satellite imagery of Minab from 2024 and 2025, taken before the strike, rather than the imagery used in the actual attack. The satellite task also allowed models to point at objects, and the report says it does not evaluate legal compliance.
The central finding is about architecture, not wording. Vals reported that it never encountered the provider-level blocking classifiers, the filters some AI companies run outside the model to catch dangerous requests, so each refusal it saw came from the model itself. Anthropic fine-tunes Claude with RLHF (reinforcement learning from human feedback) and Constitutional AI, and the research literature on adversarial machine learning generally assumes trained behavior can fail when inputs shift away from the training distribution.
Anthropic has shown that an external filter can work. A separate TechTimes report said a new Claude Fable 5 classifier blocks the specific jailbreak technique Amazon reported in more than 99% of cases. That classifier targets a code-review-framed prompt and sits in a different domain from weapons targeting, and Vals says its API testing cannot rule out safeguards it could not see.
The most consequential result involved changing the pixels, not the words. When Vals restyled the battlefield scenes with an image model so they looked like a game, Opus 5.5's refusal rate in fixed-sequence runs fell from about 90% to about 20%. A safety test that reads only text would not have caught that drop.
The simpler jailbreak also worked. Telling models the task was a game usually got past their refusals: OpenAI's models flagged the school in 0% of plain satellite questions but in 28%, 41% and 34% of them under game framing, or roughly 9, 13 and 11 of 32. One model, Muse Wonder, returned the school in 0% of satellite questions with and without game framing.
This is a jailbreak by reframing, which differs from prompt injection attacks, where an attacker hides instructions inside data the model reads. The United Kingdom's National Cyber Security Centre warned in 2023 that prompt injection may be an inherent issue with large language model technology.
When Vals asked models whether the images were real, they usually answered yes, even though most scenes were generated. Vals's report quotes one refusal saying the answers "appear to be serving as the confirmation step before strikes on people."
Vals also used DeepSeek, a Chinese model, to write some model-call code because Opus 5.5 and GPT-6 Astra sometimes declined to help. Two Chinese models, DeepSeek V4.1 Flash and Kimi K3.1 Pro, were among the 14 tested. The figures are Vals's own, and the company says reproduction code is available on request. China's 2017 National Intelligence Law says in Article 7 of that law that all organizations and citizens must support, assist and cooperate with national intelligence efforts. Legal scholar Jeremy Daum disputes how far that reaches, arguing the law has no enforcement mechanism and was not meant to require active information gathering.
On Feb. 28, a Tomahawk cruise missile likely struck a school in Minab, Iran, on the first day of the Iran war, according to Amnesty International's analysis. Amnesty reported 156 people killed, including 120 children, based on Iranian figures it could not independently verify because of an internet shutdown; other counts differ.
The Vals test does not establish that AI chose that target. Bloomberg reported Sept. 18, as summarized by Gizmodo, that an unreleased Pentagon review cited stale intelligence, outdated imagery, overreliance on Palantir's Maven system and cuts to civilian-harm staff. The Pentagon says its investigation is ongoing, and Palantir says it sees no evidence its software was at fault. Ihor Matviyuk, whose company Aero Center makes drones in Ukraine, told Military Times in March the main problem was the school's proximity to a military site, not AI, and acknowledged he has no inside knowledge. Retired Lt. Gen. Karen Gibson said a commander, not a machine, is held responsible.
The Intercept reported that the Central Command civilian-harm team fell from 10 people to one and that the Pentagon's Civilian Protection Center of Excellence shrank from 40 staff to nine, with remaining staff barred from investigating Minab.
Adm. Brad Cooper of U.S. Central Command has said humans make final targeting decisions. Elke Schwarz, a political theorist at Queen Mary University of London, argues large language models should not be used for targeting at all, because they will inevitably produce errors that contribute to civilian harm.
The research on human oversight complicates both positions. Automation bias, the tendency to over-rely on an automated recommendation, has been studied since a 1997 paper by Raja Parasuraman and Victor Riley, and a 2004 study linked it to friendly fire in Iraq. Sarah Shoker and Jessica Dorsey argue regulators should evaluate the whole sociotechnical system, because model benchmarks omit operator over-reliance and anchoring. Heidy Khlaaf of the AI Now Institute has said hallucinations are inherent to probabilistic models and that she sees little distinction between decision-support systems and autonomous weapons because of automation bias.
The report also bears on the law of armed conflict. Article 52(3) of Additional Protocol I presumes that an object normally used for civilian purposes, such as a school, is not a military objective when doubt exists, according to the principle of distinction. Opus 5.5's result, returning the school as a target in 22 of 32 questions, runs opposite to that presumption, but Vals did not test legal compliance, and the model was not operating inside any military system.
Pentagon Directive 3000.09 requires "appropriate levels of human judgment" over autonomous weapons. A House defense bill would direct a revision so the directive covers mission planning, and the Senate Armed Services Committee's draft would codify its review process. Both are committee-approved drafts, and their path to final passage is uncertain.
The Next Web reported that OpenAI told Bloomberg it bans such uses of its public models and that Google said the same. OpenAI told Vals it is working on alignment updates that explicitly cover drones and their software systems. One unnamed lab told Vals the gap was one of the most serious in its safeguards. Anthropic's updated usage policy, effective Nov. 12, covers software and components that make weapons work, including software that steers weapons, such as in arming drones.
Anthropic is in a legal fight with the Pentagon over how its models may be used. Judge Rita Lin issued a permanent injunction on Aug. 27 against a supply-chain-risk designation under 10 U.S.C. § 3252, and on Sept. 25 a divided D.C. Circuit panel voted 2-1 to uphold a separate designation under the Federal Acquisition Supply Chain Security Act, with Judges Katsas and Rao in the majority and Judge Henderson dissenting. The courts therefore did not uniformly side with the Pentagon, as TechTimes reported earlier.
Anthropic chief executive Dario Amodei has backed binding rules. A June essay, "Policy on the AI Exponential," supports California's SB 53, New York's RAISE Act and Illinois SB 315, and proposes banning domestic fully autonomous weapons.
No. Vals tested 2024 and 2025 pre-strike satellite images in a simulation, and the Pentagon says its own investigation is ongoing. Reporting on the unreleased review points to several contributing factors, including stale intelligence and overreliance on Palantir.
In Vals's fixed-sequence runs, restyling the scenes to look like a game cut Opus 5.5 refusals from about 90% to about 20%. A safety check that reads only text cannot certify how a model treats images.
It is a filter a provider runs outside the model to block dangerous requests before or after the model answers. Vals reported that it met none, so each refusal came from the model's training alone, though it notes API responses may not reveal every safeguard.
No. Vals ran public models through its own test harness, not the systems the Pentagon uses, and Vals's stated limitations include that it did not evaluate legal compliance.
