Contents
How to use this archive- I1943–2021Neural networks
- 1943 · Artificial neuron
- 1958 · Perceptron
- 1969 · Perceptrons (book)
- 1986 · Backpropagation
- 1997 · LSTM
- 2000 · Forget gate
- 2013 · VAE
- 2014 · Dropout
- 2014 · GAN
- 2015 · Batch norm
- 2016 · Layer norm
- 2017 · Mixture of experts
- 2019 · RMSNorm
- 2021 · Switch Transformer
- II2014–2024Machine learning & training
- 2014 · Adam
- 2015 · Distillation
- 2017 · AdamW
- 2020 · Scaling laws
- 2021 · LoRA
- 2022 · Chinchilla
- 2023 · QLoRA
- 2023 · DPO
- 2024 · GRPO
- III2013–2025Language models & AI
- 2013 · word2vec
- 2014 · Attention
- 2017 · Transformer
- 2018 · GPT-1
- 2018 · BERT
- 2019 · GPT-2
- 2020 · GPT-3
- 2020 · RAG
- 2021 · RoPE
- 2022 · Chain of thought
- 2022 · InstructGPT (RLHF)
- 2022 · FlashAttention
- 2022 · ReAct
- 2023 · LLaMA
- 2025 · DeepSeek-R1
- IV1998–2023Computer vision
- 1998 · LeNet
- 2012 · AlexNet
- 2015 · ResNet
- 2015 · U-Net
- 2015 · Diffusion models
- 2020 · DDPM
- 2020 · ViT
- 2021 · CLIP
- 2021 · Diffusion beats GANs
- 2021 · Classifier-free guidance
- 2021 · Latent diffusion
- 2022 · Flow matching
- 2023 · Segment Anything
- V2013–2024Robotics & reinforcement learning
- 2013 · DQN
- 2015 · TRPO
- 2015 · Nature DQN
- 2016 · AlphaGo
- 2017 · Domain randomisation
- 2017 · PPO
- 2017 · AlphaZero
- 2022 · SayCan
- 2023 · Diffusion Policy
- 2023 · RT-2
- 2023 · Open X-Embodiment
- 2024 · π0
- VI2013–2026Safety & security
- 2013 · Adversarial examples
- 2016 · Concrete problems
- 2022 · Constitutional AI
- 2023 · Indirect prompt injection
- 2023 · Universal jailbreaks (GCG)
- 2024 · Sleeper agents
- 2024 · Abliteration
- 2025 · Refusal concept cones
- 2025 · Defending against abliteration
- 2025 · Abliteration tools compared
- 2026 · Swarm red-teaming
- 2026 · The Hugging Face swarm attack
- 2026 · Stealing reasoning traces
Further reading & learning
Start here
How to use this archive
Each field runs oldest to newest. Pick a field, read down the line, and open the papers whose finding surprises you. The title opens the full paper in a new tab, usually its free arXiv page. Built on and Led to jump to the papers before and after it, sometimes in another field: the transformer came out of translation work, and diffusion moved from images into robot control.
The one-line findings are summaries, not quotes, so the paper is always the authority. Every title and link was checked against its source in October 2026. Incident reports are marked as such: they are not peer-reviewed papers. A paper marked No free copy has no free version without a sign-in, so its title opens a free explainer instead.
Study guides, made with AI
Where available, a Study guide link opens a page with a Plain English tab for anyone and an In depth tab for academics and professionals, with diagrams you can play with. See all the guides. They were written by Claude, an AI model, and each was checked against its paper by a second, separate AI review (also by Claude). I (Joona) have not personally read all of them yet, so mistakes can remain: use them to get into a paper, and check anything that matters against the paper itself.
You can also have an AI make a guide just for you, at your level and in the format you like: an interactive page, flashcards, a slide deck, even a video. Here is how to ask, and how to check the result.
Field I · 1943–2021 · 14 papers
Neural networks
The building blocks: how a network of simple units learns, keeps memory, stays stable when it gets deep, and generates new data.
1943
Artificial neuron
Treat a neuron as a unit that either fires or doesn't once its inputs pass a threshold, and networks of them can compute any logical expression: the first artificial neuron, though it could not learn.
Warren McCulloch, Walter Pitts · Bulletin of Mathematical Biophysics · PDF ↗
1958
Perceptron
Active units are reinforced when the correct response is forced or happens to be given (and weakened when it is wrong), so the machine learns to classify patterns from examples. The error-correction rule taught today, w ← w + η(y − ŷ)x, is a later textbook form of the procedure in Rosenblatt's 1962 book.
Frank Rosenblatt · Psychological Review · PDF ↗
1969
Perceptrons (book)No free copy
Simple perceptrons, whose units each see only part of the input, provably can't compute global properties such as parity or whether a shape is connected. The book is often blamed for a slump in neural-network research, though how much it caused is debated.
Marvin Minsky, Seymour Papert · MIT Press (book) · Explainer: Wikipedia ↗
1986
Backpropagation
Send the error backwards through the layers and the hidden units learn useful features on their own. The core method had been described earlier (Linnainmaa 1970, Werbos 1974); this paper showed what it could learn.
David Rumelhart, Geoffrey Hinton, Ronald Williams · Nature · PDF ↗
1997
LSTM
Gated memory cells let a recurrent network carry information across long sequences without the gradient vanishing. The forget gate that every modern LSTM has came later (Gers, Schmidhuber & Cummins, 2000).
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · PDF ↗
2000
Forget gate
Add a forget gate so an LSTM can reset its own memory, and it handles long, unbroken input streams that the original LSTM could not: the LSTM used today.
Felix Gers, Jürgen Schmidhuber, Fred Cummins · Neural Computation · PDF ↗
2013
VAE
The reparameterisation trick makes a latent-variable generative model trainable with ordinary backpropagation.
Diederik Kingma, Max Welling · ICLR 2014 · arXiv 1312.6114 ↗
2014
Dropout
Randomly switch off units during training and they stop co-adapting, which cuts overfitting.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov · JMLR · jmlr.org ↗
2014
GAN
Train a generator against a discriminator that tries to spot fakes, and the generator learns to make realistic samples.
Ian Goodfellow et al. · NeurIPS 2014 · arXiv 1406.2661 ↗
2015
Batch norm
Normalise each layer's inputs over the mini-batch and deep networks train much faster at higher learning rates. The paper's explanation, reducing "internal covariate shift", was later disputed (Santurkar et al., 2018).
Sergey Ioffe, Christian Szegedy · ICML 2015 · arXiv 1502.03167 ↗
2016
Layer norm
Normalise across a layer's features instead of the batch: it works for recurrent nets and any batch size, and became the transformer default. Many recent LLMs use the simpler RMSNorm (Zhang & Sennrich, 2019) instead.
Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey Hinton · arXiv · arXiv 1607.06450 ↗
2017
Mixture of experts
Route each input to a few experts out of thousands, so model capacity grows far faster than compute.
Noam Shazeer et al. · ICLR 2017 · arXiv 1701.06538 ↗
2019
RMSNorm
Drop layer norm's mean-centring and only rescale by the root mean square: similar accuracy for less compute, and now standard in LLMs such as LLaMA.
Biao Zhang, Rico Sennrich · NeurIPS 2019 · arXiv 1910.07467 ↗
2021
Switch Transformer
Send each token to just one expert: simpler, more stable routing that scales sparse models past a trillion parameters.
William Fedus, Barret Zoph, Noam Shazeer · JMLR · arXiv 2101.03961 ↗
Field II · 2014–2024 · 9 papers
Machine learning & training
How models are optimised, scaled, compressed and fine-tuned, from the default optimiser to training a 65B model on one GPU.
2014
Adam
Give every parameter its own step size from running averages of the gradient and its square: the default optimiser for a decade, even though its convergence proof was later shown to be flawed (Reddi et al., 2018).
Diederik Kingma, Jimmy Ba · ICLR 2015 · arXiv 1412.6980 ↗
2015
Distillation
A small model trained on a big model's soft output probabilities learns much of what the big one knows.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean · NeurIPS 2014 workshop · arXiv 1503.02531 ↗
2017
AdamW
Under Adam, L2 regularisation and weight decay are not the same thing; decoupling them generalises better.
Ilya Loshchilov, Frank Hutter · ICLR 2019 · arXiv 1711.05101 ↗
2020
Scaling laws
Language-model loss falls as a smooth power law in parameters, data and compute, so you can predict a big run from small ones. Its advice to grow models faster than data was later revised by Chinchilla.
Jared Kaplan et al. · arXiv · arXiv 2001.08361 ↗
2021
LoRA
Freeze the model and train two small low-rank matrices per layer: up to 10,000× fewer trainable parameters with no added latency.
Edward Hu et al. · ICLR 2022 · arXiv 2106.09685 ↗
2022
Chinchilla
Scale training data as fast as parameters: a 70B model on 1.4T tokens beat the 280B Gopher on the same compute.
Jordan Hoffmann et al. · NeurIPS 2022 · arXiv 2203.15556 ↗
2023
QLoRA
Train LoRA adapters on top of a frozen 4-bit model and a 65B model fine-tunes on a single 48 GB GPU.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer · NeurIPS 2023 · arXiv 2305.14314 ↗
2023
DPO
Skip the reward model and the RL loop: a simple loss on pairs of preferred and rejected answers does the same job.
Rafael Rafailov et al. · NeurIPS 2023 · arXiv 2305.18290 ↗
2024
GRPO
GRPO drops PPO's value network: sample a group of answers and score each against the group's average.
Zhihong Shao et al. · arXiv · arXiv 2402.03300 ↗
Field III · 2013–2025 · 15 papers
Language models & AI
From word vectors and attention to the transformer, instruction-following chat models, retrieval, agents and reasoning.
2013
word2vec
Cheap models turn words into vectors whose directions capture meaning and relationships between words.
Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean · ICLR 2013 workshop · arXiv 1301.3781 ↗
2014
Attention
Let the decoder look back at every input word instead of one fixed summary vector: the birth of attention.
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio · ICLR 2015 · arXiv 1409.0473 ↗
2017
Transformer
Drop recurrence entirely: attention alone, computed in parallel, beats RNNs at translation and trains far faster. Today's LLMs keep the core but usually swap in pre-norm, RMSNorm, rotary positions and gated feed-forward layers.
Ashish Vaswani et al. · NeurIPS 2017 · arXiv 1706.03762 ↗
2018
GPT-1
Pre-train a transformer to predict the next word on thousands of unpublished books, then fine-tune it: new state of the art on 9 of 12 language tasks.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever · OpenAI · PDF ↗
2018
BERT
Pre-train a transformer by filling in masked words, then fine-tune it: new state of the art on eleven language tasks.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova · NAACL 2019 · arXiv 1810.04805 ↗
2019
GPT-2
A 1.5B-parameter model trained on 40 GB of web text sets the state of the art on 7 of 8 language-modelling benchmarks zero-shot, and starts doing tasks it was never trained for.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever · OpenAI · PDF ↗
2020
GPT-3
At 175B parameters a language model picks up new tasks from a few examples in the prompt, with no fine-tuning.
Tom Brown et al. · NeurIPS 2020 · arXiv 2005.14165 ↗
2020
RAG
Retrieve passages from a document index and generate from them: more specific, more factual answers you can trace to a source.
Patrick Lewis et al. · NeurIPS 2020 · arXiv 2005.11401 ↗
2021
RoPE
Encode position by rotating the query and key vectors, so attention depends on how far apart two tokens are: the position scheme most open LLMs now use.
Jianlin Su et al. · arXiv · arXiv 2104.09864 ↗
2022
Chain of thought
Show a few worked examples with the reasoning written out and large models solve far more multi-step problems. Later the same year, just adding "Let's think step by step" was shown to help too (Kojima et al., 2022).
Jason Wei et al. · NeurIPS 2022 · arXiv 2201.11903 ↗
2022
InstructGPT (RLHF)
Fine-tune on human preferences (RLHF) and people prefer a 1.3B model's answers over the 175B GPT-3's.
Long Ouyang et al. · NeurIPS 2022 · arXiv 2203.02155 ↗
2022
FlashAttention
Exact attention, reordered so the big score matrix never goes to slow GPU memory: faster training and longer contexts.
Tri Dao, Daniel Fu, Stefano Ermon, Atri Rudra, Christopher Ré · NeurIPS 2022 · arXiv 2205.14135 ↗
2022
ReAct
Interleave reasoning steps with tool actions so the model can look things up and act: the template for LLM agents.
Shunyu Yao et al. · ICLR 2023 · arXiv 2210.03629 ↗
2023
LLaMA
Trained only on public data, the 13B model beats the 175B GPT-3 on most benchmarks, and the weights went to researchers.
Hugo Touvron et al. · arXiv · arXiv 2302.13971 ↗
2025
DeepSeek-R1
Reward just correct final answers (plus a format check) and long step-by-step reasoning, self-checking included, emerges from reinforcement learning alone (R1-Zero). R1 adds a small supervised warm-up and further training stages so the reasoning reads well.
DeepSeek-AI · Nature 2025 · arXiv 2501.12948 ↗
Field IV · 1998–2023 · 13 papers
Computer vision
How machines learned to see: convolution, depth, transformers for images, matching pictures to text, and generating images from noise.
1998
LeNet
Convolutional networks trained end to end with gradient descent read handwritten digits and cheques.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner · Proceedings of the IEEE · PDF ↗
2012
AlexNet
A deep CNN trained on two GPUs cut ImageNet top-5 error to 15.3%, against 26.2% for the runner-up.
Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton · NeurIPS 2012 · papers.nips.cc ↗
2015
ResNet
Skip connections let each block learn a correction to its input, so 152-layer networks train and win ImageNet.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun · CVPR 2016 · arXiv 1512.03385 ↗
2015
U-Net
A contract-then-expand network with skip links segments images precisely from very few training examples.
Olaf Ronneberger, Philipp Fischer, Thomas Brox · MICCAI 2015 · arXiv 1505.04597 ↗
2015
Diffusion models
Slowly destroy data with noise, then train a network to reverse each small step: the original diffusion model, borrowed from physics.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, Surya Ganguli · ICML 2015 · arXiv 1503.03585 ↗
2020
DDPM
Learn to undo noise one small step at a time and you can generate high-quality images starting from pure noise. It made the 2015 idea of diffusion models (Sohl-Dickstein et al.) work well.
Jonathan Ho, Ajay Jain, Pieter Abbeel · NeurIPS 2020 · arXiv 2006.11239 ↗
2020
ViT
Cut an image into 16×16 patches, treat them like words, and a plain transformer matches the best CNNs given enough pre-training data.
Alexey Dosovitskiy et al. · ICLR 2021 · arXiv 2010.11929 ↗
2021
CLIP
Match 400 million images to their captions and the model classifies new images zero-shot from a text description.
Alec Radford et al. · ICML 2021 · arXiv 2103.00020 ↗
2021
Diffusion beats GANs
With a better architecture and classifier guidance, diffusion models overtake GANs on image quality. Classifier-free guidance (Ho & Salimans) soon made the separate classifier unnecessary.
Prafulla Dhariwal, Alex Nichol · NeurIPS 2021 · arXiv 2105.05233 ↗
2021
Classifier-free guidance
Train one diffusion model with and without its conditioning, then push each step away from the unconditioned guess: strong guidance with no separate classifier.
Jonathan Ho, Tim Salimans · NeurIPS 2021 workshop · arXiv 2207.12598 ↗
2021
Latent diffusion
Run diffusion in a compressed latent space instead of pixels: far cheaper, and the basis of Stable Diffusion, which added CLIP's text encoder for prompts.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer · CVPR 2022 · arXiv 2112.10752 ↗
2022
Flow matching
Train a network to predict the velocity that carries noise to data along simple paths: a simpler alternative to diffusion that samples in fewer steps.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, Matt Le · ICLR 2023 · arXiv 2210.02747 ↗
2023
Segment Anything
A promptable model trained on over a billion masks segments objects in images it has never seen, zero-shot.
Alexander Kirillov et al. · ICCV 2023 · arXiv 2304.02643 ↗
Field V · 2013–2024 · 12 papers
Robotics & reinforcement learning
Learning from reward and from demonstration, from Atari and Go to robots that take instructions in plain language.
2013
DQN
The same network and settings, trained separately on each of seven Atari games, learn to play from raw pixels and the score alone. The target network and the 49-game result came in the 2015 Nature version.
Volodymyr Mnih et al. · NeurIPS 2013 workshop · arXiv 1312.5602 ↗
2015
TRPO
Keep each policy update inside a trust region, a limit on how far the new policy can drift from the old, and policy-gradient training improves steadily instead of collapsing.
John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, Pieter Abbeel · ICML 2015 · arXiv 1502.05477 ↗
2015
Nature DQN
Add a target network, a copy of the Q-network refreshed only every few thousand steps, and one architecture with one set of settings, trained separately on each game, reaches over 75% of a professional games tester's score (measured above random play) on 29 of 49 Atari games.
Volodymyr Mnih et al. · Nature · PDF ↗
2016
AlphaGo
Policy and value networks guiding a tree search beat a professional Go player 5–0, a decade earlier than expected.
David Silver et al. · Nature · PDF ↗
2017
Domain randomisation
Train in simulation with wildly random textures and lighting, and the real world looks like just one more variation.
Josh Tobin et al. · IROS 2017 · arXiv 1703.06907 ↗
2017
PPO
A simpler successor to TRPO: clip how far each update can move the policy. It is stable, and became the go-to policy-gradient method, RLHF included.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov · arXiv · arXiv 1707.06347 ↗
2017
AlphaZero
No human games at all: one algorithm learning from self-play alone masters chess, shogi and Go.
David Silver et al. · Science 2018 · arXiv 1712.01815 ↗
2022
SayCan
A language model proposes the next step and the robot's learned value functions say which steps it can actually do right now.
Michael Ahn et al. · CoRL 2022 · arXiv 2204.01691 ↗
2023
Diffusion Policy
Generate robot actions the way diffusion generates images, which copes with demonstrations that solve a task in several ways.
Cheng Chi et al. · RSS 2023 · arXiv 2303.04137 ↗
2023
RT-2
Write robot actions as text tokens and a web-trained vision-language model becomes a robot policy that generalises.
Anthony Brohan et al. · CoRL 2023 · arXiv 2307.15818 ↗
2023
Open X-Embodiment
Pool data from 22 different robots into one dataset, and one model trained on all of it beats the single-robot specialists.
Open X-Embodiment Collaboration · ICRA 2024 · arXiv 2310.08864 ↗
2024
π0
A vision-language model with a flow-matching action head drives many robot types on dexterous tasks like folding laundry.
Kevin Black et al. (Physical Intelligence) · arXiv · arXiv 2410.24164 ↗
Field VI · 2013–2026 · 13 papers
Safety & security
How models fail and get attacked: adversarial inputs, jailbreaks, prompt injection, hidden backdoors, refusal removal (abliteration) and agent swarms.
2013
Adversarial examples
Changes to an image too small to see make a classifier confidently wrong, and they fool other models too.
Christian Szegedy et al. · ICLR 2014 · arXiv 1312.6199 ↗
2016
Concrete problems
Five practical accident risks to research now: side effects, reward hacking, scalable oversight, safe exploration and distribution shift.
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané · arXiv · arXiv 1606.06565 ↗
2022
Constitutional AI
Replace human harm labels with a written list of principles that the model uses to critique and revise its own answers.
Yuntao Bai et al. · arXiv · arXiv 2212.08073 ↗
2023
Indirect prompt injection
Instructions hidden in a web page or document the model reads can take over an LLM app: to the model, data is code.
Kai Greshake et al. · AISec 2023 · arXiv 2302.12173 ↗
2023
Universal jailbreaks (GCG)
An automatically searched suffix jailbreaks open chat models, and the same suffix transfers to closed ones.
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, Matt Fredrikson · arXiv · arXiv 2307.15043 ↗
2024
Sleeper agents
A backdoor trained into a model survives standard safety training, and adversarial training can teach it to hide better.
Evan Hubinger et al. · arXiv · arXiv 2401.05566 ↗
2024
Abliteration
Refusal is one direction in the model's activations. Project it out of the weights and the model stops refusing: abliteration.
Andy Arditi et al. · NeurIPS 2024 · arXiv 2406.11717 ↗
2025
Refusal concept cones
Refusal is not just one direction: there are several independent ones, forming multi-dimensional concept cones.
Tom Wollschläger et al. · ICML 2025 · arXiv 2502.17420 ↗
2025
Defending against abliteration
Train a model to explain why before it refuses and abliteration cuts its refusal rate by at most 10%, against 70–80% for undefended models.
Harethah Abu Shairah et al. · arXiv · arXiv 2505.19056 ↗
2025
Abliteration tools compared
Four abliteration tools on 16 models: same idea, very different damage, from +1.5 to −18.8 points on GSM8K maths.
Richard J. Young · arXiv · arXiv 2512.13655 ↗
2026
Swarm red-teaming
A swarm of small attacker agents with shared memory shows jailbreak scores overstate harm, and that attack capability belongs to the whole system, not the model alone.
Michael A. Riegler, Inga Strümke · arXiv · arXiv 2605.09504 ↗
2026
The Hugging Face swarm attackIncident report
About 1,200 evaluation agents found a shared message board; roughly 700 joined an unsanctioned intrusion into Hugging Face's production systems.
OpenAI · Technical report · PDF ↗
2026
Stealing reasoning traces
Reusing encrypted reasoning blocks across models within a provider lets weaker models reveal hidden traces, exposing private data and enabling concealed prompt injections; the authors propose cryptographic and system-level mitigations.
Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, Maksym Andriushchenko · arXiv preprint · arXiv 2608.09867 · PDF ↗
Further reading & learning · 28 free resources
Learn it, free, best first
Everything here is free to read or watch, with no account, sign-in or application. Start with the six in order: each one assumes the one before it, going from intuition to building things yourself, then to a textbook, then to practice with real models. After that, pick by field.
- 1
Video series · 3Blue1Brown (Grant Sanderson)
Start here. Pictures, not code: what a network is, what gradient descent does, how backpropagation works, and how a transformer pays attention.
- 2
Video course + code · Andrej Karpathy
Build backpropagation, then a small GPT, from an empty Python file. It walks the same road as this archive, from backprop to the transformer.
- 3
Course + free book · Jeremy Howard (fast.ai)
Top-down: train a working model in lesson one, understand it later. Good if you learn by doing.
- 4
Free PDF book · Simon J. D. Prince
The modern textbook to keep open beside the papers, with clear figures, from the basics to transformers, diffusion and reinforcement learning.
- 5
Course · Hugging Face
Transformers in practice: tokenizers, fine-tuning, and sharing and running models. Reads without an account.
- 6
Article · Jay Alammar
Read it alongside “Attention Is All You Need”. After this, the papers on this page should read comfortably.
Neural networks
Free book with code · Zhang, Lipton, Li, Smola
Every chapter is runnable code, in PyTorch and other frameworks.
Free online book · Goodfellow, Bengio, Courville
The classic 2016 textbook; strongest on the maths and the foundations.
Lectures + labs · MIT
A short, yearly-updated lecture series with free videos and labs.
Machine learning & training
Free PDF book · James, Witten, Hastie, Tibshirani, Taylor
Classic machine learning before deep learning: regression, trees, validation, overfitting.
Free PDF book · Deisenroth, Faisal, Ong
The linear algebra, calculus and probability the papers assume, and nothing more.
Article · Andrej Karpathy
How to actually get a network to train, step by step, without fooling yourself.
Language models & AI
Course + lectures · Stanford
Word vectors to transformers and LLMs, with the slides, notes and lecture videos public.
Course + lectures · Stanford
Build and train a language model end to end. Lectures are free on YouTube; the assignments need a GPU.
Free PDF book · Dan Jurafsky, James H. Martin
The standard NLP textbook, with chapters on transformers and LLMs.
Computer vision
Course notes · Stanford
The best free notes on convolutional networks, still the clearest way into vision.
Article · Lilian Weng
The maths behind DDPM and latent diffusion, laid out in one place.
Robotics & reinforcement learning
Free PDF book · Richard Sutton, Andrew Barto
The reinforcement learning textbook, from bandits to policy gradients.
Guide + code · OpenAI
Deep RL (including PPO) explained with short, readable implementations.
Course · Hugging Face
Hands-on RL, from Q-learning to PPO, with notebooks.
Course · Hugging Face
Robot learning in practice: imitation learning and vision-language-action policies.
Free PDF book + videos · Kevin Lynch, Frank Park
The mechanics, planning and control underneath the learning.
Safety & security
Free online textbook · Centre pour la Sécurité de l'IA
A structured introduction to AI risk, evaluations and alignment.
Research articles · Anthropic interpretability team
How researchers look inside models: the work behind ideas like refusal directions.
Interactive articles · Distill journal (archive)
Closed in 2021, but its interactive explanations of what networks learn are still among the clearest.
Finding new papers
Paper feed · Hugging Face
New papers each day, picked and discussed by the community, all linked to arXiv.
Paper listing · arXiv
The raw feed of new machine-learning papers. Use it once the curated lists start to feel slow.
Survey articles · Lilian Weng
Long, carefully cited surveys that map a whole research area at once.