grab-site

github.com/ArchiveTeam/grab-site Everyday Web Utilities Open-source project

grab-site crawls websites for archiving, saves output as WARC, and provides a dashboard and dynamic ignore patterns.

Visit grab-site 5 alternatives

grab-site homepage screenshot

About grab-site

grab-site is a preconfigured crawler for backing up websites. Give it a starting URL and it recursively crawls the site and writes WARC files, using a fork of wpull internally. It runs on specific Python 3.7 or 3.8 versions with set dependencies, with install steps for Ubuntu, Debian, NixOS and macOS, and experimental Windows 10 support.

Pricing is not clearly stated.

Key features

Dashboard shows crawls, URLs being grabbed and queue size
Add ignore patterns while a crawl is running
Default global ignore set plus optional sets for forums and similar
Duplicate page detection skips links from duplicate content
URL queue kept on disk instead of memory

grab-site at a glance

Website: github.com/ArchiveTeam/grab-site
Category: Online Tools › Everyday Web Utilities
Type: Open-source project
Pricing: Not confirmed. Check the site
Platforms: Linux, macOS, Windows
Details verified: Oct 22, 2026 · Official source
Open source: Not stated
Language: English
Last checked: Oct 5, 2026 · Online when checked
Listed: Oct 5, 2026

crawler archiving warc

grab-site alternatives

Tools and sites you could use instead, ranked by how closely they match.

All 5 alternatives

Scrapling

github.com/D4Vinci/Scrapling

Scrapling provides adaptive web scraping for tasks ranging from single requests to full-scale crawls.

open source web scraping data extraction

Open source

17 alternatives

Crawly

crawly.diffbot.com

Crawly extracts structured content from websites by crawling pages across a site.

web scraping crawler structured data

Awesome Web Scraping

github.com/lorien/awesome-web-scraping

Awesome Web Scraping collects web scraping libraries, tools, APIs, and manuals across programming languages.

web scraping crawler captcha

Spider Suite

github.com/spidersuite/SpiderSuite

SpiderSuite is a cross-platform web crawler designed for security professionals.

security security tools crawler

Waymore

github.com/xnl-h4ck3r/waymore

Waymore searches multiple web archives and data sources, including the Wayback Machine, Common Crawl, and URLScan.

open source python vaults

Open source

14 alternatives

Popular in Everyday Web Utilities

Well-known listings from the same topic.

Barcodrod.iogithub.com/MarkHopper24/barcodrod.io QRcodlyqrcodly.de IUSinstantusername.com Your Countdown Toyourcountdown.to QRCode Monkeyqrcode-monkey.com wallabagwallabag.org

Is something wrong or out of date?

Tell us and an editor will check it.

Own this website?

Claim the listing and suggest updates yourself. An editor checks every claim first.

Claim this listing

Listing maintained by Chris McClure and the WebIndexly editors · Updated Oct 22, 2026 · How we rank