Adaptive Attack.

An adaptive attack is constructed with full knowledge of the target defence and its assumptions, and its loss function or constraints are tailored to bypass that defence. The term was popularised by Carlini and Wagner, whose 2017 study Adversarial Examples Are Not Easily Detected broke ten proposed detection methods once the attack was retargeted against each detector's specific rule.

Two landmark evaluations cemented adaptive attacks as mandatory. Athalye, Carlini and Wagner's ICML 2018 paper Obfuscated Gradients Give a False Sense of Security found that 7 of 9 defences accepted at ICLR 2018 relied on gradient masking — they merely made gradients uninformative rather than the model truly robust — and circumvented 6 of them completely using techniques such as BPDA (Backward Pass Differentiable Approximation, to get past non-differentiable pre-processing) and EOT (Expectation over Transformation, to defeat randomised defences). In 2020, Tramèr, Carlini, Brendel and Madry's On Adaptive Attacks to Adversarial Example Defenses (NeurIPS) broke 13 defences from ICLR, ICML and NeurIPS, showing that even authors who attempted adaptive evaluations often built weak, non-tailored attacks.

The lesson: any defence for adversarial examples, watermarking, or AI-content detection must be evaluated against an adversary aware of the defence who adapts its objective, ideally releasing attack code. Skipping this step routinely produces overstated robustness claims that fall to simple, principled attacks.

flowchart TD
  A[Proposed ML defence<br/>claims robustness] --> B[Generic attack<br/>e.g. standard PGD]
  B -->|Attack fails| C[Apparent robustness]
  C --> D{Adaptive evaluation?}
  D -->|Skipped| E[Overstated claim<br/>published]
  D -->|Applied| F[Attacker studies defence<br/>internals and assumptions]
  F --> G[Tailor loss + BPDA / EOT<br/>to the defence]
  G --> H{Defence survives?}
  H -->|No| I[Robustness refuted]
  H -->|Yes| J[Evidence of genuine<br/>robustness]