Web Scraping

Agent Reach lets AI agents read and search services including Twitter, Reddit, YouTube, GitHub, and Bilibili through one command-line tool.

Track brand visibility across six AI models using the open-source, local-first GEO/AEO Tracker dashboard.

GenFuse turns chat prompts into AI automations on a visual canvas for handling repetitive work.

Awesome Web Scraping collects web scraping libraries, tools, APIs, and manuals across programming languages.

Web Scraping FYI explains web scraping technologies, concepts, and techniques in a technical knowledge base.

Crawl and analyze websites for SEO, security, accessibility and performance with this cross-platform tool.

Markdown Web Browser renders URLs with headless Chrome and returns OCR-backed Markdown with provenance for agents and pipelines.

changedetection.io monitors webpages for content changes and can alert users about updates such as price drops or restocks.

Run a self-hosted search results API for Google, Bing, Yandex and other engines, with browser rendering and page extraction.

Scrapling provides adaptive web scraping for tasks ranging from single requests to full-scale crawls.

Automate browser tasks and scrape websites with Browserflow, a visual tool that requires no coding.

Ui.Vision RPA builds reusable browser and desktop macros, with automation, OCR, and AI and MCP options.

Collect and structure website data with Octoparse, a no-code web scraping tool.

PromptLoop automates web research, scraping, and CRM data enrichment for go-to-market and B2B sales teams.

CreatorHub provides a web panel for monitoring, collecting, and transferring content across Douyin, Xiaohongshu, and Kuaishou.

Browse AI extracts web data, monitors page changes, and turns website content into APIs through a no-code scraper.

All The Places provides web scrapers that produce consistent geodata about businesses and other places.

Chat4Data extracts structured information from webpages through chat prompts in a Chrome web-scraping plugin.

Crawl4AI turns website content into Markdown for language models and AI agents through an open-source crawling framework.

Import.io extracts website content into structured data and offers scraping and data services.

Scrapeless combines an agent browser, AI scraper, web unlocker, and APIs for building web-data workflows.

Crawly extracts structured content from websites by crawling pages across a site.

This Cloudflare Workers-based web scraper extracts text from selected page elements or reads an attribute.

BrowserAct creates reusable website scrapers from a data request, runs them on live sites, and returns structured results.