Marketplace

By Use Case Marketplace

Compare By Use Case products across curated subcategories and trusted providers

Products
15,460
Subcategories
699

Selected subcategory

Web Crawling

Compare By Use Case products across curated subcategories and trusted providers

Explore By Use Case with structured category paths, practical filters, and independent product data

Showing 1–12 of 56

Active filtersOpen sourceClear all

Crawl4AI

by UncleCode

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

Web CrawlingOpen Source Developer ToolsOpen Source

Isbot

by omrilotan

🤖/👨‍🦰 Recognise bots/crawlers/spiders using the user agent string.

Web CrawlingOpen Source Developer ToolsOpen Source

Linkchecker

by wummel

check links in web documents or full websites

Web CrawlingOpen Source Developer ToolsOpen Source

Scrapling

by Karim shoair

Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be!

Web CrawlingOpen Source Developer ToolsOpen Source

XSRFProbe

by Pinaki

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

Web CrawlingOpen Source Developer ToolsOpen Source

PHP Spider

by Matthijs van den Bos

A configurable and extensible PHP web spider

Web CrawlingOpen Source Developer ToolsOpen Source

FictionDown

by ma jia cong

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

Web CrawlingOpen Source Developer ToolsOpen Source

Corpuscrawler

by Google

Crawler for linguistic corpora

Web CrawlingOpen Source Developer ToolsOpen Source

NewPipeExtractor

by Team NewPipe

NewPipe's core library for extracting data from streaming sites

Web CrawlingOpen Source Developer ToolsOpen Source

lxSpider

by 李玺

爬虫案例合集。包括但不限于《淘宝、京东、天猫、豆瓣、抖音、快手、微博、微信、阿里、头条、pdd、优酷、爱奇艺、携程、12306、58、搜狐、各种指数、维普万方、Zlibraty、Oalib、小说、招标网、采购网、小红书、大众点评、推特、脉脉、知乎》

Web CrawlingOpen Source Developer ToolsOpen Source

Weibo Crawler

by Chen Lei

新浪微博爬虫,用python爬取新浪微博数据,并下载微博图片和微博视频

Web CrawlingOpen Source Developer ToolsOpen Source

Diskover Community

by Diskover Data

Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch

Web CrawlingOpen Source Developer ToolsOpen Source

Frequently asked questions