This repository contains code samples for the book Neural Networks and Deep Learning.
Neural Networks: Zero to Hero
This course teaches how to build neural networks from scratch in code.
About Neural Networks: Zero to Hero
Neural Networks: Zero to Hero is a course by Andrej Karpathy that teaches building neural networks from scratch in code. It starts with backpropagation and builds toward models such as GPT, with a focus on language models.
It lists prerequisites of solid Python programming and intro-level math including derivatives and Gaussians. Lessons cover micrograd, character-level language models, WaveNet-style networks, GPT, and a GPT tokenizer, and it points learners to a Discord channel.
Key features
- Build micrograd to explain backpropagation and neural network training
- Implement bigram character-level language model using torch.Tensor
- Implement multilayer perceptron character-level language model with training basics
- Explain activations, gradients, BatchNorm and diagnostic visualizations
- Manually backpropagate through 2-layer MLP without PyTorch autograd
- Build GPT and Tokenizer from scratch, including Byte Pair Encoding
Best for: People with solid Python programming and intro-level math who want to build neural networks in code.
Good for
- Starts with backpropagation basics and progresses to GPT and tokenizers
- Uses language models to teach concepts described as transferable to other areas
- Points to a Discord channel for learning with others
Watch out for
- Requires solid Python programming and intro-level math
- GPT lesson assumes familiarity with earlier makemore videos and PyTorch basics
- Lists residual connections, Adam optimizer and dilated convolutions as not yet covered
Neural Networks: Zero to Hero at a glance
- Category
- Education & Learning › AI Learning & Guides
- Type
- Website
- Pricing
- Not confirmed. Check the site
- Platforms
- Web
- Open source
- Not stated
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Neural Networks: Zero to Hero alternatives
Tools and sites you could use instead, ranked by how closely they match.
This Coursera course teaches supervised machine learning, regression, and classification with Python.
The Hundred-Page Language Models Book teaches machine learning foundations, transformers, and large language models.
Notebooks and code for the book "Introduction to Machine Learning with Python".
Spinning Up provides documentation and learning resources for deep reinforcement learning.
Machine Learning Mastery shares guidance for developers learning to apply machine learning.
Popular in AI Learning & Guides
Well-known listings from the same topic.
NN Visualnn-visual.comTutorialSearchtutorialsearch.ioTutor AItutorai.meAnimated AIanimatedai.github.ioMade With MLmadewithml.comR-bloggersr-bloggers.com
Neural Networks: Zero to Hero FAQ
What do I need to know before starting?
Solid programming in Python and intro-level math, such as derivatives and Gaussians.
Do I need to watch the videos in order before the GPT lesson?
The GPT lesson recommends watching the earlier makemore videos first to get comfortable with autoregressive language modeling, tensors, and PyTorch nn.
Is there a place to discuss the course with others?
Yes, the course points learners to a Discord channel.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.