AI/ML Explained: Adversarial Examples & Robustness

An adversarial example is like a carefully engineered hash collision—two images that look identical to the human eye, yet the model outputs wildly different labels ("panda" → "gibbon"). The attacker doesn't guess randomly; like reverse-engineering a hash function, they compute exactly "which direction to nudge which pixels" to flip the class. Key insight: this is not a poorly-trained model, but a structural weakness of high-dimensional space.

In 2013 Szegedy et al. found an anomaly: add a human-imperceptible tiny perturbation to a correctly-classified image, and SOTA networks misclassify it with high confidence. Stranger still—these perturbations are not random noise, but carefully targeted. Why?

Goodfellow's 2014 linear hypothesis gives the cleanest explanation: neural networks are locally near-linear. Look at how one logit is computed: w·(x+η) = w·x + w·η. The attacker picks the worst-case perturbation η = ε·sign(w) (along the sign of the weights, nudging every dimension toward raising the logit), giving a change of w·η = ε·‖w‖₁. Here ε is the per-pixel change budget (small enough to be invisible), and ‖w‖₁ is the sum of absolute weights. The crux: ‖w‖₁ grows linearly with dimension n:

A 224×224×3 image has ~150k dimensions. Move each by just ε=0.007, and the accumulated logit change is a "sum of 150k small terms"—enough to push the decision across the class boundary. Fragility isn't a defect; it's the price of high-dimensional linearity.

import numpy as np
# Demonstrate "linear amplification": why moving each dim by ε flips the output
n = 150000                     # input dim (≈ pixels of a color image)
w = np.random.randn(n)         # weights of one logit w.r.t. input
eps = 0.007                    # per-dim budget, imperceptible

eta = eps * np.sign(w)          # worst-case: along the sign of weights
delta_logit = w @ eta          # logit change = ε·‖w‖₁

print(f"per-dim step: {eps}")          # 0.007
print(f"logit change: {delta_logit:.0f}")  # hundreds — grows linearly with dim
# Takeaway: in high-dim, an "invisible perturbation" accumulates into a "huge output shift"

During training you do gradient descent on the weights to lower loss; when attacking, you freeze the weights and instead do gradient ascent on the input to raise loss. Same autodiff machinery—just aimed at "changing the input" instead of "changing the model." FGSM is one greedy step; PGD is a constrained iteration—like a "one-shot heuristic" vs an "iterative optimizer with clamping."

FGSM (Fast Gradient Sign Method): one shot. x_adv = x + ε·sign(∇ₓL). Why take the sign rather than the gradient itself? Because the attack constraint is usually L∞ (each pixel changes ≤ ε); under this constraint, moving the full ε along each dimension's gradient sign is the single step that maximizes the loss increase. Fast, but crude.

PGD (Projected Gradient Descent): split FGSM into many small steps (step size α), and after each step project back into the ε-ball—clip back to the boundary if you exceed the budget. This "projection" is exactly the bound-clamping / rate-limiting you know: clamp values to [x−ε, x+ε]. PGD is the strongest first-order white-box attack today, and the de facto standard for evaluating robustness: a defense that can't withstand PGD is basically "fake robust."

import torch, torch.nn.functional as F

def fgsm(model, x, y, eps):
    x = x.clone().requires_grad_(True)
    loss = F.cross_entropy(model(x), y)   # loss w.r.t. the true label
    loss.backward()                        # gradient w.r.t. the INPUT, not weights
    return (x + eps * x.grad.sign()).detach()  # one step raising the loss

def pgd(model, x, y, eps, alpha, steps):
    x_adv = x.clone().detach()
    for _ in range(steps):                 # iterative FGSM
        x_adv.requires_grad_(True)
        loss = F.cross_entropy(model(x_adv), y)
        grad = torch.autograd.grad(loss, x_adv)[0]
        x_adv = x_adv.detach() + alpha * grad.sign()
        x_adv = torch.min(torch.max(x_adv, x - eps), x + eps)  # project into ε-ball
        x_adv = x_adv.clamp(0, 1)          # keep valid pixel range
    return x_adv

Ordinary training feeds "normal samples"; adversarial training manufactures the worst-case sample on the fly with PGD inside each batch, then learns from it. It's essentially bringing chaos engineering into the training loop: don't wait to be attacked in production—continuously inject worst-case perturbations during training, forcing the model to "get it right even in the worst case."

Madry 2017 formalized defense as a min-max robust optimization problem:

min_θ   𝔼_(x,y) [   max_{‖δ‖≤ε}   L(θ, x+δ, y)   ]

How to read it: the inner max_δ = find the worst perturbation within budget ε (approximated by PGD); the outer min_θ = do SGD on those worst-case samples and update the weights. This saddle-point view elegantly unifies attack and defense in one objective.

The cost is the robustness–accuracy trade-off: after adversarial training, accuracy on clean samples tends to drop. The intuition is that the model is forced to give up "highly predictive but brittle" non-robust features and keep only the stable ones—safer, but also more "dull."

Transferability is another key phenomenon: adversarial examples crafted on model A often fool a completely different model B. Why? Because different models learned the same non-robust features from the same data (Ilyas 2019). The practical consequence: an attacker needn't access your model—craft samples on their own surrogate model and hit you—black-box attacks become possible, like a zero-day that works across implementations.

# Adversarial training loop: inner max approximated by PGD, outer min by SGD
for x, y in loader:
    # (1) inner: craft this batch's "worst input" with PGD first
    x_adv = pgd(model, x, y, eps=0.03, alpha=0.007, steps=10)

    # (2) outer: do normal gradient descent on the worst input
    loss = F.cross_entropy(model(x_adv), y)
    opt.zero_grad()
    loss.backward()
    opt.step()
    # corresponds to  min_θ  E[ max_δ  L(θ, x+δ, y) ]
    # note: training cost ≈ normal × (PGD steps+1), since each step runs a full PGD

An empirical defense says "every attack we tried failed"—like "never been hacked" ≠ "secure"; certified robustness wants a provable guarantee. Randomized smoothing works remarkably like a quorum read: don't trust a single prediction; add lots of Gaussian noise to the input, sample thousands of times, and take the majority vote. A single adversarial flip can't overturn a strong consensus.

Pain point: the attack-defense arms race never ends—every empirical defense gets broken by a stronger attack months later. Certified robustness escapes this loop: it gives a radius R and a mathematical guarantee that no perturbation with L2 norm ≤ R can change the output.

Randomized smoothing (Cohen 2019) is the most practical method: wrap the base classifier f into a new one, g(x) = the majority-vote class of f after adding noise N(0, σ²I) to x. The theorem gives the certified radius:

R = σ · Φ⁻¹(p_A)

Meaning of the symbols: σ is the noise level added; p_A is the vote share of the top class under noise; Φ⁻¹ is the inverse CDF of the standard normal (translating "vote share" into "distance in std deviations"). Intuition: if you add lots of noise and the model still overwhelmingly votes A, then A's "vote reserve" is deep and a small perturbation can't move it. The closer p_A is to 1, the larger the certified radius.

import torch
from scipy.stats import norm   # norm.ppf is Φ⁻¹

def smoothed_predict(model, x, sigma=0.25, n=1000, k=10):
    votes = torch.zeros(k)               # ballot box for k classes
    for _ in range(n):                    # n noisy predictions (= quorum read)
        noisy = x + sigma * torch.randn_like(x)
        votes[model(noisy).argmax()] += 1   # cast one vote each time
    top = votes.argmax()
    p_A = (votes[top] / n).item()       # vote share of the top class
    radius = sigma * norm.ppf(p_A)        # certified radius R = σ·Φ⁻¹(p_A)
    return top.item(), radius           # within L2 radius R, guaranteed not to flip
    # a single adversarial pixel flip can't shake a 950:50 strong consensus

• Szegedy et al. 2013 — Intriguing Properties of Neural Networks — first discovery of adversarial examples; the field's origin
• Goodfellow et al. 2014 — Explaining and Harnessing Adversarial Examples — the linear hypothesis + FGSM, the classic mechanistic explanation
• Madry et al. 2017 — Towards Deep Learning Models Resistant to Adversarial Attacks — the foundational work on PGD and min-max robust optimization
• Cohen et al. 2019 — Certified Robustness via Randomized Smoothing — tight analysis of randomized smoothing, scalable to ImageNet
• Ilyas et al. 2019 — Adversarial Examples Are Not Bugs, They Are Features — the non-robust-features view that explains the root of transferability