AI/ML Explained: AI for Science

A protein is a 1D amino-acid sequence that deterministically folds into a 3D shape—but that "sequence → shape" mapping has no closed-form solution humans can compute. Backend analogy: like receiving a serialized byte stream (with no schema) and having to reconstruct its in-memory object graph. Measuring structure experimentally (X-ray crystallography, cryo-EM) is like "running one months-long, six-figure full physical scan"; AlphaFold is like "a trained model predicting it in minutes."

A protein's function is determined by its 3D shape (an enzyme's active pocket, an antibody's binding site). But measuring structure is painfully slow—about 100,000 solved in fifty years, versus hundreds of millions of known sequences. That's the sequence-structure gap. AlphaFold2 (Jumper et al. 2021) rests on two mechanisms:

(1) MSA (Multiple Sequence Alignment)—line up homologous versions of the same protein across species, treated as "distributed replicas left behind by evolution." The key insight: if two residues are adjacent in 3D (in contact), they tend to co-evolve—one mutates, the other mutates compensatorily to keep the structure stable. It's like two modules in a codebase that "always change together," hinting they're architecturally coupled. AlphaFold reads this co-evolution signal to infer which residues sit next to each other in space.

(2) Evoformer—an attention-based module that repeatedly exchanges information between a "sequence representation" and a "pairwise representation (who contacts whom)," then a structure module directly outputs every atom's 3D coordinates. AlphaFold3 (2024) extended this to protein-DNA-small-molecule complexes.

# Fold a sequence via the public ESMFold API on ESM Atlas (no GPU needed)
import requests

seq = "MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGDGTQDNLSGAEK"  # your sequence
r = requests.post(
    "https://api.esmatlas.com/foldSequence/v1/pdb/",
    data=seq, timeout=120,
)
with open("predicted.pdb", "w") as f:
    f.write(r.text)   # standard PDB file, drop it into PyMOL / py3Dmol

# The B-factor column at each ATOM line stores ESMFold's pLDDT confidence
# pLDDT > 90 = high confidence; < 50 = usually flexible/disordered—don't trust
print("Structure saved; filter by confidence before drawing conclusions")

Searching a vast crystal combinatorial space for "stable new materials" is like searching a huge configuration space for a valid config. Judging whether a candidate is stable traditionally means running DFT (Density Functional Theory)—like "actually running an expensive integration test," hours to days of CPU per material. GNoME uses a GNN to model the crystal as a graph (atoms are nodes, bonds are edges) and directly predict energy—like a "static analyzer" instantly judging whether a config compiles, sparing most of the expensive verification.

The bottleneck for new materials (batteries, superconductors, catalysts) is combinatorial explosion—which elements, which atomic arrangements yield an astronomical count you can't DFT one by one. GNoME (Merchant et al. 2023, Google DeepMind) isn't a one-shot predictor but an active learning loop:

The GNN first predicts stability over a huge candidate set → only the "most likely stable" small batch is sent to real DFT verification → the results (right or wrong) are fed back into the training set → the GNN gets sharper → predict the next batch. This loop is exactly the backend-familiar "cheap coarse filter + expensive precise verify + feedback refill" CI loop, isomorphic to RL's explore-exploit. The paper reports finding roughly 380,000 new stable crystal structures.

# Query a material's DFT formation energy via Materials Project API
from mp_api.client import MPRester

with MPRester("YOUR_MP_API_KEY") as mpr:   # free key at materialsproject.org
    docs = mpr.materials.summary.search(
        formula="Li3PO4",                       # a solid electrolyte
        fields=["material_id", "formation_energy_per_atom",
                "energy_above_hull"],
    )
for d in docs:
    # energy_above_hull == 0 -> on the convex hull -> thermodynamically stable
    stable = "stable" if d.energy_above_hull < 1e-6 else "metastable/unstable"
    print(d.material_id, round(d.formation_energy_per_atom, 3), stable)

Here AI doesn't hand you the answer; it "generates hypotheses for a human (or an automated evaluator) to verify"—like AI generating a batch of candidate query plans/indexes and you benchmarking which is actually faster. Math discovery has two very different AI routes, both following this template.

Route A — guiding human intuition (Davies et al. 2021, DeepMind × mathematicians): train a GNN on large datasets of math objects (e.g. knots) to surface "which invariants have unexpected correlations," feed that pattern as a hint to mathematicians, who then prove it into a rigorous theorem. One result was a new theorem in knot theory. Here AI is an intuition amplifier, not a proving machine—the proof is still done by humans.

Route B — evolutionary program search (FunSearch, Romera-Paredes et al. 2024): the key insight is to search for "programs" rather than "answers"—instead of storing "what the solution looks like," evolve "the function that constructs the solution" (think procedural vs declarative). Mechanism: an LLM acts as the mutation operator (rewriting/generating candidate programs), an automated evaluator acts as the fitness function (scoring each program), keep the high-scorers, iterate—it's a genetic algorithm, but the LLM generates the "offspring." It found constructions beyond the best known on the cap set problem (extremal combinatorics) and better heuristics for bin packing.

# FunSearch skeleton: LLM mutates a heuristic, evaluator scores, keep the best
from anthropic import Anthropic
client = Anthropic()  # needs ANTHROPIC_API_KEY

def evaluate(code):        # fitness: actually run the candidate, quantify quality
    ns = {}; exec(code, ns)
    return score_on_benchmark(ns["heuristic"])   # your scoring logic

best = ("def heuristic(x): return x", evaluate("def heuristic(x): return x"))
for _ in range(50):                                 # evolve 50 generations
    msg = client.messages.create(model="claude-opus-4-8", max_tokens=512,
        messages=[{"role": "user",
            "content": f"Improve this heuristic for a higher score, reply code only:\n{best[0]}"}])
    cand = extract_code(msg.content[0].text)
    try:
        s = evaluate(cand)
        if s > best[1]: best = (cand, s)   # keep only strict improvements
    except Exception: pass              # mutations may be syntactically broken; skip

Porting the large-model paradigm—"pretrain once, fine-tune everywhere"—into the natural sciences. The core counterintuition: data-driven models beat first-principles physical solvers on many problems, both more accurately and orders of magnitude faster—because real data holds regularities the model never wrote into equations.

Example 1 — GraphCast (Lam et al. 2023, Science): traditional weather forecasting numerically solves atmospheric PDEs on a supercomputer, like "recomputing every forecast from scratch." GraphCast uses a GNN trained on 40 years of reanalysis data to learn a function that maps the current state directly to a future state—like a precomputed materialized view, producing a 10-day forecast in one minute, beating top numerical systems on about 90% of verification targets.

Example 2 — ESM-2 / ESMFold (Lin et al. 2023, Science): treat a protein sequence as a "language" and pretrain with masked prediction in self-supervised fashion—exactly like training BERT. Scaled to 15 billion parameters, structural information "emerges": it folds proteins without any MSA, about an order of magnitude faster than AlphaFold (at a slight accuracy cost). This shows the "statistics of sequences" already encode structure.

Shared mechanism: large-scale self-supervised pretraining → learn a general domain "representation" → fine-tune on downstream tasks, wholly isomorphic to the LLM playbook.

# Extract representation vectors with a tiny ESM-2 (8M params, CPU-friendly)
import torch
from transformers import AutoTokenizer, AutoModel

name = "facebook/esm2_t6_8M_UR50D"          # smallest ESM-2
tok = AutoTokenizer.from_pretrained(name)
model = AutoModel.from_pretrained(name).eval()

seq = "MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQ"
inp = tok(seq, return_tensors="pt")
with torch.no_grad():
    out = model(**inp)
# one vector per residue; mean-pool for a whole-sequence "fingerprint"
emb = out.last_hidden_state.mean(dim=1)
print(emb.shape)   # torch.Size([1, 320]) -- attach your own task head and fine-tune

• Jumper et al. 2021 — Highly accurate protein structure prediction with AlphaFold (Nature) — the original AlphaFold2 paper, foundational for AI in structural biology
• Merchant et al. 2023 — Scaling deep learning for materials discovery (Nature) — GNoME, discovering new crystals via an active-learning loop
• DeepMind — FunSearch (2023/2024) — evolutionary program search with LLMs finds new math constructions (paper in Nature 625)
• Davies et al. 2021 — Advancing mathematics by guiding human intuition with AI (Nature) — AI guiding human intuition, a new knot-theory theorem
• Lam et al. 2023 — Learning skillful medium-range global weather forecasting (Science) — GraphCast weather foundation model
• Lin et al. 2023 — Evolutionary-scale prediction of atomic-level protein structure (Science) — ESM-2 / ESMFold protein language model