Pachyderm

Pachyderm provides data-focused pipelines and version control for datasets and related workflows.

github.com/pachyderm/pachyderm
Pachyderm homepage screenshot

About Pachyderm

Pachyderm is an open-source project for building data-centric pipelines with version control for datasets and related workflows. Pipelines are data-driven and trigger automatically when data changes are detected, while immutable data lineage and versioning are tracked for any data type. Processing is parallelized across multi-stage, language-agnostic pipelines, with autoscaling built on Kubernetes for resource orchestration, and data is held in standard object stores with automatic deduplication.

Key features

  • Data-driven pipelines trigger automatically when data changes are detected
  • Immutable data lineage with data versioning for any data type
  • Autoscaling and parallel processing built on Kubernetes
  • Multi-stage, language-agnostic pipeline processing
  • Uses standard object stores with automatic deduplication
  • Runs on major cloud providers and on-premises installations
Best for: Data engineering and data science teams that need to automate complex, versioned data transformations across varied data types.
Good for
  • Open source and released under the Apache-2.0 license
  • Supports deployment on major cloud providers and on-premises installations
  • Combines data versioning, lineage tracking and pipeline automation
Watch out for
  • No pricing information is stated in the source
  • Anonymized usage metrics are reported automatically unless disabled via the METRICS environment variable
  • Autoscaling and parallel processing are built on Kubernetes

Pachyderm at a glance

Type
Open-source project
Pricing
Not confirmed. Check the site
Open source
Yes · Source code
Language
English
Last checked
Oct 6, 2026 · Online when checked
Listed
Oct 5, 2026

Pachyderm alternatives

Tools and sites you could use instead, ranked by how closely they match.

All 24 alternatives

Popular in AI Infrastructure & MLOps

Well-known listings from the same topic.

All 341

Pachyderm FAQ

What license is Pachyderm released under?
The repository lists an Apache-2.0 license.
Does Pachyderm collect usage data?
It automatically reports anonymized usage metrics. These can be disabled by setting the METRICS environment variable to false in the pachd container.
Where can Pachyderm be deployed?
It can be run locally or deployed on AWS, GCE or Azure, and it also runs on-premises.
Is something wrong or out of date?

Tell us and an editor will check it.

Own this website?

Claim the listing and suggest updates yourself. An editor checks every claim first.

Claim this listing