Sparky, the LoveSpark wolf mascot Papers worth readingA LoveSpark archive · October 2026

A LoveSpark archive

Papers worth
reading

76 papers · 6 fields · 1943 to 2026

The research behind modern AI, as timelines by field. Each entry gives the paper's main finding in one line; click it to open the full paper. When a later paper improved on or replaced an idea, the entry links forward to it, so you can follow a technique from where it started to what replaced it.

Three passes, from S. Keshav, “How to Read a Paper” (2007)
  1. First pass: the shape

    Title, abstract, introduction, headings, conclusion. Five to ten minutes tells you what it claims and whether to go on.

  2. Second pass: the content

    Read it properly, with care over the figures and graphs. Skip the proofs. Note what you don't follow.

  3. Third pass: rebuild it

    Re-create the work in your head, or in code, from the same assumptions. This is where you find what it got wrong.

Study guides
ContentsHow to use this archive
  1. 1943–2021Neural networks
    1. 1943 · Artificial neuron
    2. 1958 · Perceptron
    3. 1969 · Perceptrons (book)
    4. 1986 · Backpropagation
    5. 1997 · LSTM
    6. 2000 · Forget gate
    7. 2013 · VAE
    8. 2014 · Dropout
    9. 2014 · GAN
    10. 2015 · Batch norm
    11. 2016 · Layer norm
    12. 2017 · Mixture of experts
    13. 2019 · RMSNorm
    14. 2021 · Switch Transformer
  2. 2014–2024Machine learning & training
    1. 2014 · Adam
    2. 2015 · Distillation
    3. 2017 · AdamW
    4. 2020 · Scaling laws
    5. 2021 · LoRA
    6. 2022 · Chinchilla
    7. 2023 · QLoRA
    8. 2023 · DPO
    9. 2024 · GRPO
  3. 2013–2025Language models & AI
    1. 2013 · word2vec
    2. 2014 · Attention
    3. 2017 · Transformer
    4. 2018 · GPT-1
    5. 2018 · BERT
    6. 2019 · GPT-2
    7. 2020 · GPT-3
    8. 2020 · RAG
    9. 2021 · RoPE
    10. 2022 · Chain of thought
    11. 2022 · InstructGPT (RLHF)
    12. 2022 · FlashAttention
    13. 2022 · ReAct
    14. 2023 · LLaMA
    15. 2025 · DeepSeek-R1
  4. 1998–2023Computer vision
    1. 1998 · LeNet
    2. 2012 · AlexNet
    3. 2015 · ResNet
    4. 2015 · U-Net
    5. 2015 · Diffusion models
    6. 2020 · DDPM
    7. 2020 · ViT
    8. 2021 · CLIP
    9. 2021 · Diffusion beats GANs
    10. 2021 · Classifier-free guidance
    11. 2021 · Latent diffusion
    12. 2022 · Flow matching
    13. 2023 · Segment Anything
  5. 2013–2024Robotics & reinforcement learning
    1. 2013 · DQN
    2. 2015 · TRPO
    3. 2015 · Nature DQN
    4. 2016 · AlphaGo
    5. 2017 · Domain randomisation
    6. 2017 · PPO
    7. 2017 · AlphaZero
    8. 2022 · SayCan
    9. 2023 · Diffusion Policy
    10. 2023 · RT-2
    11. 2023 · Open X-Embodiment
    12. 2024 · π0
  6. 2013–2026Safety & security
    1. 2013 · Adversarial examples
    2. 2016 · Concrete problems
    3. 2022 · Constitutional AI
    4. 2023 · Indirect prompt injection
    5. 2023 · Universal jailbreaks (GCG)
    6. 2024 · Sleeper agents
    7. 2024 · Abliteration
    8. 2025 · Refusal concept cones
    9. 2025 · Defending against abliteration
    10. 2025 · Abliteration tools compared
    11. 2026 · Swarm red-teaming
    12. 2026 · The Hugging Face swarm attack
    13. 2026 · Stealing reasoning traces
Further reading & learning

Start here

How to use this archive

Each field runs oldest to newest. Pick a field, read down the line, and open the papers whose finding surprises you. The title opens the full paper in a new tab, usually its free arXiv page. Built on and Led to jump to the papers before and after it, sometimes in another field: the transformer came out of translation work, and diffusion moved from images into robot control.

The one-line findings are summaries, not quotes, so the paper is always the authority. Every title and link was checked against its source in October 2026. Incident reports are marked as such: they are not peer-reviewed papers. A paper marked No free copy has no free version without a sign-in, so its title opens a free explainer instead.

Study guides, made with AI

Where available, a Study guide link opens a page with a Plain English tab for anyone and an In depth tab for academics and professionals, with diagrams you can play with. See all the guides. They were written by Claude, an AI model, and each was checked against its paper by a second, separate AI review (also by Claude). I (Joona) have not personally read all of them yet, so mistakes can remain: use them to get into a paper, and check anything that matters against the paper itself.

You can also have an AI make a guide just for you, at your level and in the format you like: an interactive page, flashcards, a slide deck, even a video. Here is how to ask, and how to check the result.

Field I · 1943–2021 · 14 papers

Neural networks

The building blocks: how a network of simple units learns, keeps memory, stays stable when it gets deep, and generates new data.

1943

A Logical Calculus of the Ideas Immanent in Nervous Activity (opens the full paper in a new tab)

Treat a neuron as a unit that either fires or doesn't once its inputs pass a threshold, and networks of them can compute any logical expression: the first artificial neuron, though it could not learn.

Led toPerceptron 1958
Study guidePlain English + in depth, made with AI (study guide for Artificial neuron)
1958

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain (opens the full paper in a new tab)

Active units are reinforced when the correct response is forced or happens to be given (and weakened when it is wrong), so the machine learns to classify patterns from examples. The error-correction rule taught today, w ← w + η(y − ŷ)x, is a later textbook form of the procedure in Rosenblatt's 1962 book.

Built onArtificial neuron 1943
Led toPerceptrons (book) 1969Backpropagation 1986
Study guidePlain English + in depth, made with AI (study guide for Perceptron)
1969

Perceptrons: An Introduction to Computational Geometry (opens Wikipedia in a new tab; the full text has no free copy)

Simple perceptrons, whose units each see only part of the input, provably can't compute global properties such as parity or whether a shape is connected. The book is often blamed for a slump in neural-network research, though how much it caused is debated.

Built onPerceptron 1958
Led toBackpropagation 1986
Study guidePlain English + in depth, made with AI (study guide for Perceptrons (book))
1986

Learning representations by back-propagating errors (opens the full paper in a new tab)

Send the error backwards through the layers and the hidden units learn useful features on their own. The core method had been described earlier (Linnainmaa 1970, Werbos 1974); this paper showed what it could learn.

Built onPerceptron 1958Perceptrons (book) 1969
Led toLSTM 1997LeNet 1998
Study guidePlain English + in depth, made with AI (study guide for Backpropagation)
2015

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (opens the full paper in a new tab)

Normalise each layer's inputs over the mini-batch and deep networks train much faster at higher learning rates. The paper's explanation, reducing "internal covariate shift", was later disputed (Santurkar et al., 2018).

Led toLayer norm 2016
Study guidePlain English + in depth, made with AI (study guide for Batch norm)

Field II · 2014–2024 · 9 papers

Machine learning & training

How models are optimised, scaled, compressed and fine-tuned, from the default optimiser to training a 65B model on one GPU.

Field III · 2013–2025 · 15 papers

Language models & AI

From word vectors and attention to the transformer, instruction-following chat models, retrieval, agents and reasoning.

2017

Attention Is All You Need (opens the full paper in a new tab)

Drop recurrence entirely: attention alone, computed in parallel, beats RNNs at translation and trains far faster. Today's LLMs keep the core but usually swap in pre-norm, RMSNorm, rotary positions and gated feed-forward layers.

Built onLSTM 1997Attention 2014
Led toGPT-1 2018BERT 2018GPT-3 2020ViT 2020FlashAttention 2022RoPE 2021
Study guidePlain English + in depth, made with AI (study guide for Transformer)
2022

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (opens the full paper in a new tab)

Show a few worked examples with the reasoning written out and large models solve far more multi-step problems. Later the same year, just adding "Let's think step by step" was shown to help too (Kojima et al., 2022).

Built onGPT-3 2020
Led toReAct 2022DeepSeek-R1 2025
Study guidePlain English + in depth, made with AI (study guide for Chain of thought)
2025

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (opens the full paper in a new tab)

Reward just correct final answers (plus a format check) and long step-by-step reasoning, self-checking included, emerges from reinforcement learning alone (R1-Zero). R1 adds a small supervised warm-up and further training stages so the reasoning reads well.

Built onGRPO 2024Chain of thought 2022
Study guidePlain English + in depth, made with AI (study guide for DeepSeek-R1)

Field IV · 1998–2023 · 13 papers

Computer vision

How machines learned to see: convolution, depth, transformers for images, matching pictures to text, and generating images from noise.

2020

Denoising Diffusion Probabilistic Models (opens the full paper in a new tab)

Learn to undo noise one small step at a time and you can generate high-quality images starting from pure noise. It made the 2015 idea of diffusion models (Sohl-Dickstein et al.) work well.

Built onU-Net 2015Diffusion models 2015
Led toDiffusion beats GANs 2021Latent diffusion 2021Diffusion Policy 2023Flow matching 2022
Study guidePlain English + in depth, made with AI (study guide for DDPM)
2021

Diffusion Models Beat GANs on Image Synthesis (opens the full paper in a new tab)

With a better architecture and classifier guidance, diffusion models overtake GANs on image quality. Classifier-free guidance (Ho & Salimans) soon made the separate classifier unnecessary.

Built onGAN 2014DDPM 2020
Led toClassifier-free guidance 2021
Study guidePlain English + in depth, made with AI (study guide for Diffusion beats GANs)
2021

High-Resolution Image Synthesis with Latent Diffusion Models (opens the full paper in a new tab)

Run diffusion in a compressed latent space instead of pixels: far cheaper, and the basis of Stable Diffusion, which added CLIP's text encoder for prompts.

Built onVAE 2013DDPM 2020Classifier-free guidance 2021
Study guidePlain English + in depth, made with AI (study guide for Latent diffusion)

Field V · 2013–2024 · 12 papers

Robotics & reinforcement learning

Learning from reward and from demonstration, from Atari and Go to robots that take instructions in plain language.

2015

Human-level control through deep reinforcement learning (opens the full paper in a new tab)

Add a target network, a copy of the Q-network refreshed only every few thousand steps, and one architecture with one set of settings, trained separately on each game, reaches over 75% of a professional games tester's score (measured above random play) on 29 of 49 Atari games.

Built onDQN 2013
Led toAlphaGo 2016
Study guidePlain English + in depth, made with AI (study guide for Nature DQN)

Field VI · 2013–2026 · 13 papers

Safety & security

How models fail and get attacked: adversarial inputs, jailbreaks, prompt injection, hidden backdoors, refusal removal (abliteration) and agent swarms.

2023

Universal and Transferable Adversarial Attacks on Aligned Language Models (opens the full paper in a new tab)

An automatically searched suffix jailbreaks open chat models, and the same suffix transfers to closed ones.

Built onAdversarial examples 2013
Led toAbliteration 2024
Study guidePlain English + in depth, made with AI (study guide for Universal jailbreaks (GCG))
2024

Refusal in Language Models Is Mediated by a Single Direction (opens the full paper in a new tab)

Refusal is one direction in the model's activations. Project it out of the weights and the model stops refusing: abliteration.

Built onUniversal jailbreaks (GCG) 2023
Led toRefusal concept cones 2025Defending against abliteration 2025Abliteration tools compared 2025
Study guidePlain English + in depth, made with AI (study guide for Abliteration)
2026

Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance (opens the full paper in a new tab)

A swarm of small attacker agents with shared memory shows jailbreak scores overstate harm, and that attack capability belongs to the whole system, not the model alone.

Led toThe Hugging Face swarm attack 2026
Study guidePlain English + in depth, made with AI (study guide for Swarm red-teaming)
2026

OpenAI – Hugging Face Incident: Technical Report (opens the full paper in a new tab)

About 1,200 evaluation agents found a shared message board; roughly 700 joined an unsanctioned intrusion into Hugging Face's production systems.

Built onIndirect prompt injection 2023Swarm red-teaming 2026
Also readHugging Face's forensic timeline METR's independent investigation
Study guidePlain English + in depth, made with AI (study guide for The Hugging Face swarm attack)
2026

Stealing Reasoning Traces from Proprietary LLM APIs (opens the full paper in a new tab)

Reusing encrypted reasoning blocks across models within a provider lets weaker models reveal hidden traces, exposing private data and enabling concealed prompt injections; the authors propose cryptographic and system-level mitigations.

Study guidePlain English + in depth, made with AI (study guide for Stealing reasoning traces)

Further reading & learning · 28 free resources

Learn it, free, best first

Everything here is free to read or watch, with no account, sign-in or application. Start with the six in order: each one assumes the one before it, going from intuition to building things yourself, then to a textbook, then to practice with real models. After that, pick by field.

  1. Neural networks (opens in a new tab)

    Start here. Pictures, not code: what a network is, what gradient descent does, how backpropagation works, and how a transformer pays attention.

  2. Neural Networks: Zero to Hero (opens in a new tab)

    Build backpropagation, then a small GPT, from an empty Python file. It walks the same road as this archive, from backprop to the transformer.

  3. Practical Deep Learning for Coders (opens in a new tab)

    Top-down: train a working model in lesson one, understand it later. Good if you learn by doing.

  4. Understanding Deep Learning (opens in a new tab)

    The modern textbook to keep open beside the papers, with clear figures, from the basics to transformers, diffusion and reinforcement learning.

  5. LLM Course (opens in a new tab)

    Transformers in practice: tokenizers, fine-tuning, and sharing and running models. Reads without an account.

  6. The Illustrated Transformer (opens in a new tab)

    Read it alongside “Attention Is All You Need”. After this, the papers on this page should read comfortably.

Neural networks

Machine learning & training

Language models & AI

Computer vision

Robotics & reinforcement learning

Safety & security

Finding new papers