Heritrix
Heritrix 3 documentation describes configuration and use of the Heritrix web crawler.
About Heritrix
Heritrix 3 is a web crawler. Its documentation explains installation, running the crawler, starting crawl jobs, and managing jobs through a user interface and REST API.
It covers job settings such as crawl scope, limits, politeness, retry policy, bandwidth, link extraction, authentication, proxies, checkpointing and recovery, along with log files and crawl reports for monitoring. The site notes that additional documentation still lives on the GitHub wiki while guides are migrated here.
Key features
- Configure crawl scope, limits, politeness and retry policy
- Extract links from HTML, CSS, JS, PDF, XML and other formats
- Fetch content via FTP, HTTP/2, HTTP/3, SFTP and WHOIS
- Checkpoint crawls and restart or recover from checkpoints
- Manage crawl jobs through REST API
- Configure HTTP, SOCKS5 proxies and DNS over HTTP
Best for: Operators and developers configuring and running Heritrix 3 crawl jobs.
Good for
- Documents crawl configuration including scope, politeness, bandwidth and authentication
- Includes operational guidance for checkpointing, recovery, logs and reports
- Describes REST API operations for creating, launching, pausing and terminating jobs
Watch out for
- Documentation notes that additional material remains on the GitHub wiki during migration
Heritrix at a glance
- Website
- heritrix.readthedocs.io
- Category
- Development › Developer Tools
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Platforms
- Web
- Open source
- Not stated
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Guides for Developer Tools
Heritrix alternatives
Tools and sites you could use instead, ranked by how closely they match.
Build native user interfaces with Slint, an open-source declarative GUI toolkit for Rust, C++, JavaScript, and Python.
Open source
24 alternatives
Sign files and verify digital signatures with Minisign, a small command-line signing tool.
Open source
24 alternatives
Decompiler Explorer compares output from multiple decompilers in a web browser.
Open source
24 alternatives
GPT-Migrate helps move a codebase from one programming language or framework to another.
Open source
24 alternatives
Describe interface ideas and see rendered UI components with OpenUI, a prototyping tool.
Open source
24 alternatives
StatPing provides status pages for monitoring websites and applications.
Open source
24 alternativesPopular in Developer Tools
Well-known listings from the same topic.
AppDevToolsappdevtools.comCommand Not Foundcommand-not-found.comTestFlighttestflight.apple.com/join/A8GwCnq8Sourcegraphabout.sourcegraph.comProject Wallaceprojectwallace.comCarbon Calculatorwebsitecarbon.com
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.