AI / ML

Day 1 LLM Basics Transformer · Attention · Tokenization · Positional Encoding Day 2 Pre-training & Fine-tuning Pre-training · SFT · RLHF · DPO Day 3 Prompt Engineering Zero/Few-shot · CoT · ReAct · Self-Consistency Day 4 RAG Systems Embedding · Vector DB · Retrieval · Re-ranking Day 5 Agent Architectures ReAct · Plan-and-Execute · Reflexion · AutoGPT Day 6 Tool Use Function Calling · MCP · Tool Selection · Sandboxing Day 7 Multi-Agent Systems AutoGen · CrewAI · Role Specialization · Collaboration Protocols Day 8 Context Engineering Context Window · Prompt Caching · Compression · Memory Day 9 Inference Optimization KV Cache · Speculative Decoding · Continuous Batching · Quantization Day 10 Loss & Optimization Cross-Entropy · AdamW · LR Schedule · Gradient Clipping Day 11 Tokenization Deep Dive BPE · WordPiece · SentencePiece · UTF-8 Boundaries Day 12 Attention Variants MHA · GQA/MQA · Sliding Window · FlashAttention Day 13 Positional Encoding Deep Dive Sinusoidal · Learned · RoPE · ALiBi Day 14 Scaling Laws Power Law · Chinchilla · Emergent Abilities · Mirage Critique Day 15 Evaluation & Benchmarks MMLU · HumanEval/pass@k · MT-Bench · LLM-as-Judge Day 16 Classic ML Algorithms Linear/Logistic Reg · Decision Tree · Random Forest · SVM Day 17 Deep Learning Foundations Backprop · Gradient Descent · Activations · Regularization Day 18 CNN & Vision Convolution · ResNet · ViT · CLIP Day 19 RNNs & Sequences LSTM · GRU · Seq2Seq · Attention Origin Day 20 Generative Models GAN · VAE · Diffusion · Flow Matching Day 21 Reinforcement Learning Q-Learning · Policy Gradient · Actor-Critic · PPO Day 22 Semantic Search BM25 · Dense Retrieval · Hybrid Search · HNSW Day 23 Multimodality VLM · Audio LLM · Video Understanding · Embodied AI Day 24 Encoder Models BERT · RoBERTa · Sentence Transformers · ColBERT Day 25 Training Infrastructure Distributed Training · ZeRO · FSDP · Megatron-LM Day 26 Alignment Math RLHF · Reward Model · Constitutional AI · DPO Day 27 Interpretability Mechanistic Interp · Sparse Autoencoders · Feature Circuits · Probing Day 28 Reasoning Models Test-time Compute · CoT · Self-Verification · Best-of-N Day 29 Data Engineering Synthetic Data · Data Curation · Deduplication · Quality Filtering Day 30 Representation & Embedding Geometry Word Vector Linear Structure · Contrastive/InfoNCE · Anisotropy & Whitening · Matryoshka Day 31 Model Compression Knowledge Distillation · Pruning · LoRA/QLoRA · Quantization Math Day 32 Activations & Normalization ReLU/GELU/SwiGLU · BatchNorm/LayerNorm/RMSNorm Day 33 Probability & Information Theory Entropy · KL Divergence · Mutual Information · ELBO Day 34 Frontier Architectures MoE · State Space Models/S4 · Mamba · Long Context Day 35 Time-Series Forecasting ARIMA · Exponential Smoothing/Kalman · Prophet · Foundation Models/TimesFM Day 36 Causal Inference Causation vs Correlation · Potential Outcomes · Instrumental Variables · Difference-in-Differences Day 37 Graph Machine Learning Message Passing/GNN · GCN · GAT · Graph Embedding/node2vec Day 38 Probabilistic Programming & Bayesian Deep Learning MCMC · Variational Inference · Uncertainty Quantification · PyMC/Stan Day 39 Meta-Learning & Few-Shot Transfer Learning · MAML · Prototypical Networks · In-context Learning Day 40 Federated & Privacy Learning Federated Learning/FedAvg · Differential Privacy · Homomorphic Encryption · Secure MPC Day 41 World Models & Embodied Intelligence World Models/Dreamer · JEPA · Causal Representation Learning · Embodied AI/VLA Day 42 Neuro-Symbolic Neuro-Symbolic Integration · KG Embeddings/TransE · Differentiable Reasoning · Program Synthesis/DreamCoder Day 43 The Geometry of Optimization Loss Landscape/Flat Minima · Sharpness & Generalization/SAM · Second-Order/K-FAC/Shampoo · Lion/Sophia Day 44 Anomalies in Training Grokking/Delayed Generalization · Double Descent · Phase Transitions & Emergence · Is Emergence a Mirage? Day 45 The Math of Decoding & Sampling Greedy/Beam Search · Temperature/Top-p/Min-p · Contrastive Decoding · Speculative Decoding Day 46 AI for Science AlphaFold · Materials Discovery (GNoME) · Math Conjectures (FunSearch) · Scientific Foundation Models Day 47 Alignment Failure Mechanisms Specification Gaming/Reward Hacking · Goal Misgeneralization · Deceptive Alignment · Sycophancy Day 48 Hallucination & Calibration Statistical Origin · Calibration/ECE · RLHF Hurts Calibration · Knowledge Boundary P(True) Day 49 Knowledge Storage & Model Editing Localization/Causal Tracing · ROME Rank-One Edit · MEMIT Batch Edit · Ripple Effects Day 50 Adversarial Examples & Robustness Linear Hypothesis/High-dim Fragility · FGSM/PGD Attacks · Adversarial Training/Transferability · Certified Robustness (Smoothing) Day 51 Fairness, Bias & Debiasing Sources of Bias · Demographic Parity vs Equalized Odds · Impossibility Theorem · Debiasing/Causal Fairness Day 52 Continual Learning & Catastrophic Forgetting Catastrophic Forgetting/Stability-Plasticity · EWC Reg./Fisher · Experience & Generative Replay · Parameter Isolation (PackNet) Day 53 Audio & Speech Models Speech Recognition/CTC Alignment · Two-stage TTS (Acoustic Model + Vocoder) · Neural Codec/RVQ Discretization · Streaming & Causality Day 54 Bias & Calibration in LLM-as-Judge Position/Verbosity/Self-Enhancement Bias · Self-Preference & Correlated Errors · Generation-Verification Gap · Diverse Panel Aggregation & Calibration Day 55 CoT Monitoring & Monitorability CoT Monitoring (Production Flag Rates) · Obfuscated Reward Hacking · Faithfulness Probe (25%-39%) · Monitorability Tax/Cross-Lab Position Day 56 Same Name, Different Thing: Model Versioning & Eval Reproducibility Referential Drift (o3-preview 87.5% vs 41-53%) · Scaffolding Confound/pass@k · The Harness Is the Benchmark · Construct Validity & Error Bars Day 57 Diffusion Language Models Masked Diffusion/Absorbing State · Any-Order & the Flexibility Trap · Block-Parallel Denoising (NFE Ledger) · Independence Tax / Three-Way Trade-off Day 58 Verbalizable Representations & the Global Workspace Jacobian Lens (J-lens) · J-space (<10% variance, k≈25) · Verbalizable = Used (Causal Interventions) · Global Workspace Theory & Critiques Day 59 Automated Alignment Researchers AAR Loop (Hill-climbing + Capability Guardrail) · Safety Gap Closed (26%-96%, Deception 85%) · Goodhart in the Research Loop (2.4% Cheating) · Overseeing the Overseer / Fuzzy Tasks Day 60 Alignment Post-Mortem of Real Incidents Biased Reasoning (79% Isolated vs 1% In-Task) · Recklessness & Momentum (Reminder 90%→40%) · Faithful-but-Biased CoT Fools Monitors (1% vs 50%) · No-Solution Evals & Generations (82%/31%/33%) Day 61 The Scale Leap in Machine Proof Proof Kernel (An Unpersuadable Judge) · Statement Fidelity & Comparator (3 Axioms / No sorry) · Statement DAG for Multi-agent (~30k Theorems) · Verified ≠ Understood ≠ Reusable (5× Mathlib, ~20× Compile)