Dedupe

github.com/dedupeio/dedupe AI Data & Analytics Open-source projectOpen source

A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

github.com/dedupeio/dedupe
Dedupe homepage screenshot

About Dedupe

Dedupe is an open-source Python library that uses machine learning for fuzzy matching, deduplication and entity resolution on structured data. It takes human-provided training data and derives rules to find similar records in a dataset.

It can remove duplicate names and addresses, link separate lists such as customer information and order history without shared IDs, and group records where names were entered slightly differently. It is installed with pip install dedupe and used through its Python API.

Key features

  • Fuzzy matching of structured records using machine learning
  • Deduplication of duplicate names and addresses
  • Record linkage across lists without unique IDs
  • Learns matching rules from human-provided training data
  • csvdedupe command-line tool for CSV deduplication and linking
  • Installable with pip install dedupe
Best for: Python developers and data workers who need to deduplicate structured records and link datasets without shared identifiers.
Good for
  • Released as open source under the MIT License
  • Learns dataset-specific matching rules from human training examples
  • Provides example projects and a CSV command-line tool
Watch out for
  • Requires Python to install and use the library
  • Needs human-provided training data to learn matching rules
  • Designed for structured data

Dedupe at a glance

Type
Open-source project
Pricing
Not confirmed. Check the site
Open source
Yes · Source code
Language
English
Last checked
Oct 5, 2026 · Online when checked
Listed
Oct 5, 2026

Dedupe alternatives

Tools and sites you could use instead, ranked by how closely they match.

All 24 alternatives

Popular in AI Data & Analytics

Well-known listings from the same topic.

All 302

Dedupe FAQ

How do I install Dedupe for use?
Run pip install dedupe, then review the API and examples to start a project.
What is csvdedupe?
A command-line tool for de-duplicating and linking CSV files.
How do I report a bug?
Report it through the repository issues page.
Is something wrong or out of date?

Tell us and an editor will check it.

Own this website?

Claim the listing and suggest updates yourself. An editor checks every claim first.

Claim this listing