This article shares practical tips for fine tuning large language models with low rank adaptation, drawn from experiments.
Publication
Patterns for Building LLM-based Systems & Products
This article outlines patterns for building LLM systems, covering evaluation, retrieval augmented generation, fine tuning, caching, and guardrails.
About Patterns for Building LLM-based Systems & Products
This article describes seven patterns for integrating large language models into systems and products: evals, RAG, fine-tuning, caching, guardrails, defensive UX, and collecting user feedback. The patterns are organized by improving performance versus reducing cost and risk, and by proximity to data versus proximity to the user.
It defines evals as measurements, including benchmark data and metrics, used to assess performance and detect regressions. It discusses benchmarks such as MMLU, EleutherAI Eval, HELM and AlpacaEval, and mentions metrics such as BLEU, ROUGE, BERTScore and MoverScore.
Key features
- Explains evals to measure performance and detect regressions
- Describes RAG to add recent, external knowledge
- Covers fine-tuning for specific tasks
- Explains caching to reduce latency and cost
- Describes guardrails to ensure output quality
- Covers defensive UX and collecting user feedback
Best for: Practitioners integrating large language models into systems and products.
Good for
- Organizes patterns by performance versus cost/risk and data versus user proximity
- Draws on academic research, industry resources and practitioner know-how
- Describes evaluation benchmarks and metrics such as MMLU, HELM and BLEU
Patterns for Building LLM-based Systems & Products at a glance
- Official website
- eugeneyan.com/writing/llm-patterns
- Type
- Publication
- Platforms
- Web
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Similar publications
Other listings in Science, Math & Technology with the most in common.
This guide explains how to run LLMs locally with llama.cpp across different hardware.
Publication
Stanford MLSys Seminar Series presents talks on systems for machine learning and includes a podcast.
Publication
Open-access peer-reviewed edited volume edited by Witold Kosinski, presenting improvements and concepts in evolutionary algorithms with chapters readable online.
Freemium
PublicationPopular in Science, Math & Technology
Well-known listings from the same topic.
MathNetmathnet.mit.eduOneZoomonezoom.orgThrustCurvethrustcurve.orgWikispeciesspecies.wikimedia.orgTheAlgorithmsthe-algorithms.comHot Nozzle Societyhotnozzlesociety.org
Patterns for Building LLM-based Systems & Products FAQ
What patterns does the article cover?
Evals to measure performance, RAG to add recent external knowledge, fine-tuning for specific tasks, caching to reduce latency and cost, guardrails to ensure output quality, defensive UX to manage errors gracefully, and collecting user feedback to build a data flywheel.
What are evals according to the article?
A set of measurements, including benchmark data and metrics, used to assess a model's performance on a task and detect regressions.
How are the patterns organized?
Along a spectrum from improving performance to reducing cost and risk, and from closer to the data to closer to the user.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.