Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
add links to public actors
Glenn Goossens3109b8dc6bb5
diff --git a/docs/guides/getting_started.md b/docs/guides/getting_started.mdindex ae3a6d74b..d144dbb37 100644--- a/docs/guides/getting_started.md+++ b/docs/guides/getting_started.md@@ -633,7 +633,7 @@ And that's it! No more parsing the links from HTML using Cheerio, filtering themWe hear you young padawan! First, learn how to crawl, you must. Only then, save data, you can!### Making a store crawler-Fortunately, we don't have to travel to a galaxy far far away to find a good candidate for learning how to scrape structured data. The <a href="https://apify.com/store" target="_blank">Apify Store</a> is a store of public actors that anyone can grab and use. You can find ready-made solutions for crawling Google Places, Amazon, Google SERPs, Booking, Kickstarter and many other websites. Feel free to check them out! It also poses a great place to practice our jedi scraping skills since it has categories, lists and details. That's almost like our imaginary `online-store.com` from the previous chapter.+Fortunately, we don't have to travel to a galaxy far far away to find a good candidate for learning how to scrape structured data. The <a href="https://apify.com/store" target="_blank">Apify Store</a> is a store of public actors that anyone can grab and use. You can find ready-made solutions for crawling <a href="https://apify.com/drobnikj/crawler-google-places" target="_blank">Google Places</a>, <a href="https://apify.com/vaclavrut/amazon-crawler" target="_blank">Amazon</a>, <a href="https://apify.com/apify/google-search-scraper" target="_blank">Google Search</a>, <a href="https://apify.com/petr_cermak/booking-scraper" target="_blank">Booking</a>, <a href="https://apify.com/jaroslavhejlek/instagram-scraper" target="_blank">Instagram</a>, <a href="https://apify.com/petrpatek/tripadvisor-scraper" target="_blank">Tripadvisor</a> and many other websites. Feel free to check them out! It also poses a great place to practice our jedi scraping skills since it has categories, lists and details. That's almost like our imaginary `online-store.com` from the previous chapter.### The importance of having a planSometimes scraping is really straightforward, but most of the times, it really pays out to do a little bit of research first. How is the website structured? Can I scrape it only with HTTP requests (read "with `CheerioCrawler`") or would I need a full browser solution? Are there any anti-scraping protections in place? Do I need to parse the HTML or can I get the data otherwise, such as directly from the website's API. Jakub, one of Apify's founders wrote a <a href="https://blog.apify.com/web-scraping-in-2018-forget-html-use-xhrs-metadata-or-javascript-variables-8167f252439c" target="_blank">great article about all the different techniques</a> and tips and tricks so make sure to check that out!