Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
http://example.com/boundary/babies.html
http://example.com/birthday/bite.php
https://www.example.com/#birth
https://www.example.net/bag?bedroom=box
http://basketball.example.com/books#activity
http://example.edu/blood
http://www.example.edu/afterthought?brick=believe&afterthought=addition
http://arm.example.com/
https://www.example.com/berry
http://example.com/border/alarm.php
http://example.net/
https://www.example.com/
http://www.example.com/brick/addition
https://example.com/bed.html
http://www.example.org/airplane
https://www.example.com/#babies
https://www.example.com/amount
http://www.example.com/
http://bubble.example.com/advertisement.php
http://example.net/
http://adjustment.example.com/bikes/boy.html
http://example.com/
https://www.example.com/beginner/battle
http://bee.example.com/art/birth
https://www.example.com/bubble/amount.php?ants=brass&approval=brother#bite
http://www.example.net/?beds=bone
http://www.example.com/
http://www.example.com/
http://anger.example.org/bone/action.htm?baby=balance
https://www.example.com/apparel
https://www.example.com/badge/bike
http://www.example.com/apparel
https://border.example.edu/
https://www.example.com/
https://arm.example.com/baseball/bird.htm?animal=amusement
http://www.example.com/animal.html
http://example.com/arm.php
https://www.example.org/
http://bubble.example.com/basketball.php?ants=bed&believe=beef#box
https://www.example.org/#anger
http://apparatus.example.edu/argument
https://www.example.com/
http://www.example.com/
http://www.example.com/art.aspx
https://www.example.com/agreement/brother.aspx?attraction=bite
http://anger.example.com/?account=bottle&bath=behavior
https://www.example.com/#acoustics
https://www.example.com/addition.php
https://www.example.com/brass
https://www.example.com/?branch=blow&actor=bath
http://www.example.com/blow.php
http://example.com/authority
http://www.example.com/birthday.html#bed
http://boot.example.com/bottle?agreement=approval&believe=army
https://www.example.com/
http://www.example.com/
https://example.edu/action/achiever
https://www.example.com/branch/book.php?brass=babies
http://www.example.com/
https://back.example.edu/activity
https://www.example.edu/birth.php
http://adjustment.example.com/book
http://www.example.com/?book=baby&bait=board
http://www.example.com/
https://example.com/
https://www.example.com/
http://www.example.com/#behavior
https://www.example.net/base
https://example.com/act
http://www.example.edu/
http://www.example.net/
https://www.example.com/?art=bear
https://www.example.com/basin/bed.aspx#act
http://www.example.com/
http://example.net/baseball/addition?blood=blade&apparatus=anger
https://www.example.com/afternoon/appliance.html
https://apparatus.example.com/bedroom.html
https://www.example.com/
https://www.example.com/
http://www.example.com/birds
http://example.org/?boat=believe
https://www.example.com/#angle
https://www.example.com/
https://example.com/#believe
http://birthday.example.com/bottle?activity=alarm&bite=bottle
https://example.net/
https://achiever.example.com/
http://www.example.com/boundary/believe
http://example.com/appliance/base.html
http://example.com/bikes.php
http://base.example.com/
http://www.example.com/
http://www.example.com/
https://www.example.com/base?birth=basketball&afternoon=actor
http://www.example.net/bead
https://www.example.org/?alarm=birds
https://www.example.com/art
https://www.example.com/
https://example.net/action/box.php
http://www.example.com/
https://x.com/elonmusk/status/1783231058462617962