This open book collects methods, tools, and instructions for training, fine-tuning, and running machine learning models.
rasbt/LLMs-from-scratch
This repository walks through implementing a ChatGPT like language model in PyTorch, step by step.
About rasbt/LLMs-from-scratch
This GitHub repository contains the code for the book Build a Large Language Model (From Scratch). It walks through creating a GPT-like language model in PyTorch, with each stage of development explained in the book's text, diagrams, and examples. The code covers developing, pretraining, and finetuning a model.
The method is intended for training a small but functional model for educational purposes, and the source says it mirrors the approach used for large-scale foundational models such as those behind ChatGPT. The book also includes code for loading the weights of larger pretrained models for finetuning. The source states that a strong foundation in Python programming is the most important prerequisite.
Key features
- Chapter-by-chapter notebooks and scripts for building a GPT-like model
- Code for pretraining a small model for educational purposes
- Code for finetuning, including text classification and instruction following
- Code for loading weights of larger pretrained models for finetuning
- Appendices on PyTorch basics, training loop additions, and LoRA finetuning
- Setup guide and troubleshooting guide included in the repository
Best for: Readers with solid Python skills who want to understand how large language models work by building one in PyTorch.
Good for
- Code is organized by chapter, following the structure of the book
- Includes setup and troubleshooting documentation within the repository
- Covers loading weights from larger pretrained models for finetuning
Watch out for
- A strong foundation in Python programming is the most important prerequisite
- The model built is small and intended for educational purposes
rasbt/LLMs-from-scratch at a glance
- Category
- Education & Learning › Learn to Code
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Not stated
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
rasbt/LLMs-from-scratch alternatives
Tools and sites you could use instead, ranked by how closely they match.
This LLM course combines learning roadmaps with Google Colab notebooks for studying large language models.
Open source
24 alternatives
This tutorial walks through training and deploying a machine learning REST API that predicts cryptocurrency prices in ten steps.
Open source
24 alternatives
Jupyter notebooks implement and derive machine learning algorithms from first principles using NumPy.
Open source
24 alternatives
A Python learning playground and cheat sheet organizes example scripts by topic, with explanations for the code.
Open source
24 alternatives
Data Science IPython Notebooks collects notebooks about data science, including TensorFlow deep learning.
Popular in Learn to Code
Well-known listings from the same topic.
Codelabscodelabs.developers.google.comScratchscratch.mit.eduTelegram Bot APIcore.telegram.org/botsStudent Developer Packeducation.github.com/packHacksplaininghacksplaining.comQB64SourceCodeqb64tutorial.com
rasbt/LLMs-from-scratch FAQ
How can I get a copy of the code?
Click the Download ZIP button on the repository page, or run git clone --depth 1 https://github.com/rasbt/LLMs-from-scratch.git in a terminal.
Where can I find help with installing Python and setting up the environment?
The README.md file in the setup directory gives guidance on installing Python, Python packages, and the code environment. A troubleshooting guide is also included.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.