Study guides · 76 of 76 ready

Papers explained

Study guides for papers in Papers worth reading. Each has a Plain English tab for anyone and an In depth tab for specialists, with diagrams you can play with.

Neural networks

  1. Artificial neuron 1943
    A Logical Calculus of the Ideas Immanent in Nervous Activity
  2. Perceptron 1958
    The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain
  3. Perceptrons (book) 1969
    Perceptrons: An Introduction to Computational Geometry
  4. Backpropagation 1986
    Learning representations by back-propagating errors
  5. LSTM 1997
    Long Short-Term Memory
  6. Forget gate 2000
    Learning to Forget: Continual Prediction with LSTM
  7. VAE 2013
    Auto-Encoding Variational Bayes
  8. Dropout 2014
    Dropout: A Simple Way to Prevent Neural Networks from Overfitting
  9. GAN 2014
    Generative Adversarial Networks
  10. Batch norm 2015
    Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
  11. Layer norm 2016
    Layer Normalization
  12. Mixture of experts 2017
    Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
  13. RMSNorm 2019
    Root Mean Square Layer Normalization
  14. Switch Transformer 2021
    Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Machine learning & training

  1. Adam 2014
    Adam: A Method for Stochastic Optimization
  2. Distillation 2015
    Distilling the Knowledge in a Neural Network
  3. AdamW 2017
    Decoupled Weight Decay Regularization
  4. Scaling laws 2020
    Scaling Laws for Neural Language Models
  5. LoRA 2021
    LoRA: Low-Rank Adaptation of Large Language Models
  6. Chinchilla 2022
    Training Compute-Optimal Large Language Models
  7. QLoRA 2023
    QLoRA: Efficient Finetuning of Quantized LLMs
  8. DPO 2023
    Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  9. GRPO 2024
    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Language models & AI

  1. word2vec 2013
    Efficient Estimation of Word Representations in Vector Space
  2. Attention 2014
    Neural Machine Translation by Jointly Learning to Align and Translate
  3. Transformer 2017
    Attention Is All You Need
  4. GPT-1 2018
    Improving Language Understanding by Generative Pre-Training
  5. BERT 2018
    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  6. GPT-2 2019
    Language Models are Unsupervised Multitask Learners
  7. GPT-3 2020
    Language Models are Few-Shot Learners
  8. RAG 2020
    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  9. RoPE 2021
    RoFormer: Enhanced Transformer with Rotary Position Embedding
  10. Chain of thought 2022
    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  11. InstructGPT (RLHF) 2022
    Training language models to follow instructions with human feedback
  12. FlashAttention 2022
    FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
  13. ReAct 2022
    ReAct: Synergizing Reasoning and Acting in Language Models
  14. LLaMA 2023
    LLaMA: Open and Efficient Foundation Language Models
  15. DeepSeek-R1 2025
    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Computer vision

  1. LeNet 1998
    Gradient-Based Learning Applied to Document Recognition
  2. AlexNet 2012
    ImageNet Classification with Deep Convolutional Neural Networks
  3. ResNet 2015
    Deep Residual Learning for Image Recognition
  4. U-Net 2015
    U-Net: Convolutional Networks for Biomedical Image Segmentation
  5. Diffusion models 2015
    Deep Unsupervised Learning using Nonequilibrium Thermodynamics
  6. DDPM 2020
    Denoising Diffusion Probabilistic Models
  7. ViT 2020
    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  8. CLIP 2021
    Learning Transferable Visual Models From Natural Language Supervision
  9. Diffusion beats GANs 2021
    Diffusion Models Beat GANs on Image Synthesis
  10. Classifier-free guidance 2021
    Classifier-Free Diffusion Guidance
  11. Latent diffusion 2021
    High-Resolution Image Synthesis with Latent Diffusion Models
  12. Flow matching 2022
    Flow Matching for Generative Modeling
  13. Segment Anything 2023
    Segment Anything

Robotics & reinforcement learning

  1. DQN 2013
    Playing Atari with Deep Reinforcement Learning
  2. TRPO 2015
    Trust Region Policy Optimization
  3. Nature DQN 2015
    Human-level control through deep reinforcement learning
  4. AlphaGo 2016
    Mastering the game of Go with deep neural networks and tree search
  5. Domain randomisation 2017
    Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
  6. PPO 2017
    Proximal Policy Optimization Algorithms
  7. AlphaZero 2017
    Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
  8. SayCan 2022
    Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
  9. Diffusion Policy 2023
    Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
  10. RT-2 2023
    RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
  11. Open X-Embodiment 2023
    Open X-Embodiment: Robotic Learning Datasets and RT-X Models
  12. π0 2024
    π0: A Vision-Language-Action Flow Model for General Robot Control

Safety & security

  1. Adversarial examples 2013
    Intriguing properties of neural networks
  2. Concrete problems 2016
    Concrete Problems in AI Safety
  3. Constitutional AI 2022
    Constitutional AI: Harmlessness from AI Feedback
  4. Indirect prompt injection 2023
    Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
  5. Universal jailbreaks (GCG) 2023
    Universal and Transferable Adversarial Attacks on Aligned Language Models
  6. Sleeper agents 2024
    Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
  7. Abliteration 2024
    Refusal in Language Models Is Mediated by a Single Direction
  8. Refusal concept cones 2025
    The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
  9. Defending against abliteration 2025
    An Embarrassingly Simple Defense Against LLM Abliteration Attacks
  10. Abliteration tools compared 2025
    Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluation
  11. Swarm red-teaming 2026
    Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance
  12. The Hugging Face swarm attack 2026
    OpenAI – Hugging Face Incident: Technical Report
  13. Stealing reasoning traces 2026
    Stealing Reasoning Traces from Proprietary LLM APIs