DeepSeek-VL2 supports multimodal understanding through a mixture-of-experts vision-language model.
Open source
24 alternativesmicrosoft/LoRA
Microsoft’s loralib implements low rank adaptation for large language models.
About microsoft/LoRA
This GitHub repository contains the source code for loralib, a Python package that implements Low-Rank Adaptation (LoRA) for large language models. LoRA learns pairs of rank-decomposition matrices while the original model weights stay frozen. According to the source, this reduces the number of trainable parameters and the storage needed for models adapted to specific tasks, and it allows task-switching during deployment without adding inference latency.
The repository also includes examples of how to use loralib with PyTorch models, including Hugging Face models. It supports PyTorch only. To use the LoRA checkpoints, the original pre-trained checkpoint from Hugging Face is also required. The source reports results on the GLUE benchmark that are comparable or superior to full fine-tuning when using RoBERTa and DeBERTa, and it describes comparisons with other adaptation methods on GPT-2.
Key features
- Python package loralib for adapting large language models using low-rank matrices
- Trains rank-decomposition matrix pairs while freezing the original model weights
- Examples of integrating loralib with PyTorch models, including Hugging Face models
- Example implementations for GPT-2, RoBERTa, and DeBERTa v2 in separate example folders
- Downloadable LoRA checkpoints for RoBERTa, DeBERTa, and GPT-2 models
- Installable with pip install loralib
Best for: Machine learning researchers and developers who fine-tune large language models in PyTorch and want to reduce trainable parameters and storage.
Good for
- Reduces the number of trainable parameters and the storage needed for task-adapted models
- The source reports results on the GLUE benchmark comparable or superior to full fine-tuning for RoBERTa and DeBERTa
- Includes step-by-step example directories for reproducing the reported results
Watch out for
- Supports PyTorch only
- The original pre-trained checkpoint from Hugging Face is still required to use the LoRA checkpoints
- Reproducing the paper's results requires following the instructions in the example directories
microsoft/LoRA at a glance
- Website
- github.com/microsoft/LoRA
- Category
- AI & Machine Learning › Models & ML Research
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
microsoft/LoRA alternatives
Tools and sites you could use instead, ranked by how closely they match.
This repository accompanies research on improving language model factuality and reasoning through multi-agent debate.
This repository curates research on large language models for code generation.
Open source
24 alternatives
SWE-bench evaluates whether language models can resolve real GitHub issues.
Open source
24 alternatives
This repository lists open language models identified as available for commercial use.
Open source
24 alternatives
Qwen is a series of chat and pretrained language models proposed by Alibaba Cloud.
Open source
24 alternativesPopular in Models & ML Research
Well-known listings from the same topic.
Falcon LLMfalconllm.tii.aeERNIEernie.baidu.comTokenVersetoken-verse.github.iowTAObittensor.comAlpacacrfm.stanford.edu/2023/03/13/alpaca.htmlSmolVLAhuggingface.co/blog/smolvla
microsoft/LoRA FAQ
Which deep learning framework does loralib support?
The source states that only PyTorch is supported for now.
Do I still need the original model to use the LoRA checkpoints?
Yes. The source notes that the original pre-trained checkpoint from Hugging Face is still needed to use the LoRA checkpoints.
How is loralib installed?
The source gives pip install loralib as the install command. It also shows an alternative of installing directly from the GitHub repository with pip.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.