Crawlee Python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and…
About
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
Open Source Health
- Stars
- 9,517
- Forks
- 805
- License
- Apache-2.0
- Last commit
- 28 days ago
Related Categories
Vendor
Apify
Publisher of Apify Console
Quick Links
Open Source
More by Apify
Related Products
Deep Research
An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine…
Shared categories
select.rs
A Rust library to extract useful data from HTML documents, suitable for web scraping.
Shared categories
Scrapely
A pure-python HTML screen-scraping library
Shared categories
Faster Than Requests
Faster requests on Python 3
Shared categories
MechanicalSoup
A Python library for automating interaction with websites.
Shared categories