← all articles

what is adversarial distillation? openai's november 2026 defenses, explained

openai defines adversarial distillation as the systematic and unauthorized use of one model's outputs or reasoning to help train, reproduce, or improve another model. in a november 20, 2026 post, openai said that in its internal adaptive black-box evaluation, the average attack success rate against gpt-6.2 astra with its updated safeguards was below 1% for each attack type.

download the ReMynd desktop app to search text you saw in a focused window, by app and date.

what did openai publish?

openai's post, combatting adversarial distillation, explains how it detects and disrupts these attempts and how it strengthened protections for its models' protected reasoning. openai defines protected reasoning as the intermediate steps a model generates internally while working toward an answer, known as its chain of thought. it says these traces can contain information the final answer leaves out, which makes them especially valuable training data for another model.

the definition at the top of this page comes from openai's september 30, 2026 report on a campaign it disrupted. openai's november post adds that distillation itself is a common machine learning technique in which one model learns from data produced by another.

what do openai's numbers show?

openai tested its request level safeguards in an internal adaptive black-box evaluation, where automated ai red-teamers adjust their approach based on how the system responds. its table:

attack typegpt-6 astra, original safeguardsgpt-6.2 astra, updated safeguards
extracting protected reasoning from an earlier request54.0%0.2%
extracting protected reasoning from the current request22.9%0.9%

openai says the rates are aggregated across six attacker models, and lower is better. it says it is now applying selected safeguard improvements to gpt-6 astra as well. on enforcement, openai says its weekly enforcement actions against these campaigns have ranged in scale from thousands to tens of thousands of accounts.

how does openai defend against it?

openai describes two complementary levels of its safeguard stack:

  • request level: model behavior, classifier checks and api controls limit reasoning trace extraction. adversarial training teaches models to recognize extraction attempts and refuse to disclose protected reasoning, while preserving responses to legitimate requests. input classifiers detect extraction attempts, and output classifiers detect leaks in assistant messages and tool call arguments, for both streaming and non-streaming responses.
  • campaign level: openai looks for patterns in request level and account level signals that can reveal coordinated extraction attempts. when it identifies violations, it enforces against the network of accounts, and it shares indicators of compromise with other model providers.

on the api side, with gpt-6.2 astra and gpt-6.2 sol, openai ties a model's protected reasoning to the conversation history that produced it. if an earlier message is rewritten, the api drops the reasoning that followed it. openai also prevents older model families with weaker safeguards from reading gpt-6.2 astra and sol's protected reasoning.

why does openai treat it as a safety issue?

openai says that through adversarial distillation, a second model can learn advanced frontier capabilities without inheriting the safeguards that govern the original. it says the concern is especially clear in dual use domains like cybersecurity and biology.

in an internal study, openai fine tuned a student model on a dataset consisting primarily of trajectories from astra level teacher models working on coding and defensive cybersecurity tasks.

after supervised fine tuning, the student trained on exact reasoning traces generally outperformed the student trained on reasoning summaries. after reinforcement learning, the exact trace student still scored higher at the largest inference budget tested on both benchmarks shown, exploitbench and sec-bench pro, while the summary trained student scored higher at lower budgets.

what is the openai distillation policy?

openai's usage policies, effective the same day as the post, have a section on training and developing models. it says you must not use openai models or their outputs to generate, prepare, select, or supply training material or signals for models that are intended as alternatives to or substitutes for openai models or services. it also says you must not attempt to extract protected reasoning through unauthorized access, reverse engineering, or circumvention of safeguards.

the same section says you may use openai models for research, software development, model evaluation and benchmarking, and product development, within these boundaries. it adds that these restrictions do not prohibit uses expressly permitted by your applicable agreement with openai.

what changes for developers using the api?

openai's developer guide, keep the prefix unchanged, says that starting with newer frontier models, the api will drop reasoning items that follow a prefix change as part of its safeguards. a prefix change modifies earlier history, for example by shortening a tool result, rewriting a developer message, or replacing older turns with a summary.

when reasoning items are dropped, openai says the model may have to redo some reasoning, which can increase token usage, cost, and latency, and if the model can't recover the lost reasoning, output quality may also drop. its recommendation is to avoid changes to the prefix and adopt an append-only integration pattern as far as possible.

how does anthropic's approach compare?

anthropic published strengthening our defenses against distillation attacks on october 12, 2026. openai's post cites anthropic's related work in a footnote.

openaianthropic
main focusparticular emphasis on preventing extraction of protected reasoningalmost entirely on making claude's reasoning harder for distillers to access
headline resultlower extraction success against gpt-6.2 astra (table above)early evidence suggests data harvested by distillers down by around 90 to 99%
classifier false positivesnot stated in the postless than 0.1%
reasoning tied to historyrewriting an earlier message drops the reasoning that followedwith claude fable 5.1 and claude opus 5.5, preserved thinking prevents new api accounts (created on or after august 31, 2026) from editing claude's prior context

anthropic also reports that a pretrained claude model trained on full claude opus 4.8 transcripts scored 80.4% on swe-bench verified, against 71.8% when trained on summarized transcripts. in our view, the two companies' figures come from different tests, so they are not directly comparable. openai says it has also shared findings through the frontier model forum and appropriate government information sharing channels.

where does ReMynd fit?

ReMynd records your screen on a mac, runs ocr locally with apple vision, and keeps a searchable index on your computer. text comes from the focused window only, and video covers windows you have not excluded.

in our view, if you read a lot of model answers and policy pages, the useful part is finding them again: press control space, search in plain language, and jump to the exact moment, provided ReMynd was running then and the window was not excluded. on a mac it needs macos 14 or later on an apple silicon mac. ReMynd also runs on windows 11 (x64 or arm64).

for a different attack on language models, see what is prompt injection. for how models hold what you send them, see what is a context window.

limits of this guide

  • the rates, counts and scores here are openai's or anthropic's own, as of november 2026. we read their posts on november 21 and did not run any evaluation.
  • openai's extraction rates come from an internal evaluation, and anthropic calls its harvested data figure early evidence.
  • this guide covers the term as openai uses it. it does not cover other research that uses similar words.

download the ReMynd desktop app and search what you saw in a focused window, by app and date.

common questions

what is adversarial distillation? +
openai describes it as the systematic and unauthorized use of one model's outputs or reasoning to help train, reproduce, or improve another model. its concern is that the second model can learn frontier capabilities without the safeguards that govern the original.
is distillation against openai's usage policies? +
openai's usage policies, effective november 20, 2026, say you must not use openai models or their outputs to generate, prepare, select, or supply training material or signals for models that are intended as alternatives to or substitutes for openai models or services, and must not attempt to extract protected reasoning through unauthorized access, reverse engineering, or circumvention of safeguards. the same section says you may use openai models for research, software development, model evaluation and benchmarking, and product development, within these boundaries.
how much did openai cut reasoning extraction? +
in openai's internal adaptive black-box evaluation, aggregated across six attacker models, the average attack success rate for extracting protected reasoning from an earlier request fell from 54.0% against gpt-6 astra with its original safeguards to 0.2% against gpt-6.2 astra with its updated safeguards. these are openai's own figures.
does this change anything for api developers? +
openai's developer docs say that starting with newer frontier models, the api will drop reasoning items that follow a prefix change as part of its safeguards. the model may then have to redo some reasoning, which can increase token usage, cost, and latency, and if the model can't recover the lost reasoning, output quality may also drop. openai recommends an append-only integration pattern as far as possible.
what has anthropic reported about distillation? +
in an october 12, 2026 post, anthropic says early evidence suggests its new methods have reduced the amount of data harvested by distillers by around 90 to 99%. these are anthropic's own figures.