Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Replaced www.apify.com to apify.com in docs

Jan Curn03bb67109d4d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ed4353d38..416853dc8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -457,7 +457,7 @@
0.5.7 / 2018-03-06
==================
-- A lot of new stuff. Everything is backwards compatible. Check https://www.apify.com/docs/sdk/apify-runtime-js/latest for reference
+- A lot of new stuff. Everything is backwards compatible. Check https://apify.com/docs/sdk/apify-runtime-js/latest for reference
0.5.0 / 2018-02-08
===================
diff --git a/MIGRATIONS.md b/MIGRATIONS.md
index b2bc1b4ae..19036a012 100644
--- a/MIGRATIONS.md
+++ b/MIGRATIONS.md
@@ -5,7 +5,7 @@
Originally, AutoscaledPool was just a single class that handled everything. Now it's split into 3 different classes.
As such, the configuration changed dramatically. The API is the same however.
-Mainly the parameter `workerFunction` was replaced with combination of 3 parameters `runTaskFunction`, `isTaskReadyFunction` and `isFinishedFunction`. For more information and examples check <a href="https://www.apify.com/docs/sdk/apify-runtime-js/latest#AutoscaledPool" target="_blank">AutoscaledPool documentation</a>.
+Mainly the parameter `workerFunction` was replaced with combination of 3 parameters `runTaskFunction`, `isTaskReadyFunction` and `isFinishedFunction`. For more information and examples check <a href="https://sdk.apify.com/docs/api/autoscaledpool#docsNav" target="_blank">AutoscaledPool documentation</a>.
If you were using any of the following options, you should see the current documentation and migrate accordingly:
diff --git a/README.md b/README.md
index 20f4e01db..93c55f144 100644
--- a/README.md
+++ b/README.md
@@ -10,8 +10,8 @@ to maintain queues of URLs to crawl, store crawling results to a local filesyste
rotate proxies and much more.
The SDK is available as the <a href="https://www.npmjs.com/package/apify" target="_blank"><code>apify</code></a> NPM package.
It can be used either stand-alone in your own applications
-or in <a href="https://www.apify.com/docs/actor" target="_blank">actors</a>
-running on the <a href="https://www.apify.com/" target="_blank">Apify Cloud</a>.
+or in <a href="https://apify.com/docs/actor" target="_blank">actors</a>
+running on the <a href="https://apify.com/" target="_blank">Apify Cloud</a>.
**View full documentation, guides and examples on the dedicated <a href="https://sdk.apify.com" target="_blank">Apify SDK project website</a>**
@@ -77,7 +77,7 @@ The Apify SDK is available as the <a href="https://www.npmjs.com/package/apify">
</li>
<li>
<a href="https://sdk.apify.com/docs/api/requestqueue"><code>RequestQueue</code></a>
- - Represents a queue of URLs to crawl, which is stored either on a local filesystem or in the <a href="https://www.apify.com" target="_blank">Apify Cloud</a>.
+ - Represents a queue of URLs to crawl, which is stored either on a local filesystem or in the <a href="https://apify.com" target="_blank">Apify Cloud</a>.
The queue is used for deep crawling of websites, where you start with
several URLs and then recursively follow links to other pages.
The data structure supports both breadth-first and depth-first crawling orders.
diff --git a/docs/guides/getting_started.md b/docs/guides/getting_started.md
index 11978c36d..64955823b 100644
--- a/docs/guides/getting_started.md
+++ b/docs/guides/getting_started.md
@@ -621,9 +621,9 @@ We can see that some of the information is available directly on the list page,
Knowing that we will use plain HTTP requests, we immediately know that we won't be able to manipulate the website in any way. We will only be able to go through the HTML it gives us and parse our data from there. This might sound like a huge limitation, but you might be surprised in how effective it might be. Let's get on it!
#### The start URL(s)
-This is where we start our crawl. It's convenient to start as close to our data as possible. For example, it wouldn't make much sense to start at `www.apify.com` and look for a `library` link there, when we already know that everything we want to extract can be found at the `www.apify.com/library` page.
+This is where we start our crawl. It's convenient to start as close to our data as possible. For example, it wouldn't make much sense to start at `apify.com` and look for a `library` link there, when we already know that everything we want to extract can be found at the `apify.com/library` page.
-Once we look at the `www.apify.com/library` page more carefully though, we see that the categories themselves produce URLs that we can use to access those individual categories.
+Once we look at the `apify.com/library` page more carefully though, we see that the categories themselves produce URLs that we can use to access those individual categories.
```
https://apify.com/library?category=ENTERTAINMENT
@@ -769,7 +769,7 @@ In the previous chapter, we've used the `Apify.utils.enqueueLinks()` function li
await enqueueLinks({
$,
requestQueue,
- pseudoUrls: ['http[s?]://[[-\w.]+]apify.com[.*]']
+ pseudoUrls: ['http[s?]://apify.com[.*]']
});
```
While very useful in that scenario, we need something different now. Instead of finding all the `<a href="..">` links that match the `pseudoUrl`, we need to find only the specific ones that will take us to the actor detail pages. Otherwise, we'd be visiting a lot of other pages that we're not interested in. Using the power of DevTools and yet another `enqueueLinks()` parameter, this becomes fairly easy.
diff --git a/docs/readme/introduction.md b/docs/readme/introduction.md
index 98cd10fff..75834704a 100644
--- a/docs/readme/introduction.md
+++ b/docs/readme/introduction.md
@@ -10,7 +10,7 @@ to maintain queues of URLs to crawl, store crawling results to a local filesyste
rotate proxies and much more.
The SDK is available as the <a href="https://www.npmjs.com/package/apify" target="_blank"><code>apify</code></a> NPM package.
It can be used either stand-alone in your own applications
-or in <a href="https://www.apify.com/docs/actor" target="_blank">actors</a>
-running on the <a href="https://www.apify.com/" target="_blank">Apify Cloud</a>.
+or in <a href="https://apify.com/docs/actor" target="_blank">actors</a>
+running on the <a href="https://apify.com/" target="_blank">Apify Cloud</a>.
**View full documentation, guides and examples on the dedicated <a href="https://sdk.apify.com" target="_blank">Apify SDK project website</a>**
diff --git a/docs/readme/overview.md b/docs/readme/overview.md
index 77e9e4cf9..16a42ccf1 100644
--- a/docs/readme/overview.md
+++ b/docs/readme/overview.md
@@ -36,7 +36,7 @@ The Apify SDK is available as the <a href="https://www.npmjs.com/package/apify">
</li>
<li>
<a href="https://sdk.apify.com/docs/api/requestqueue"><code>RequestQueue</code></a>
- - Represents a queue of URLs to crawl, which is stored either on a local filesystem or in the <a href="https://www.apify.com" target="_blank">Apify Cloud</a>.
+ - Represents a queue of URLs to crawl, which is stored either on a local filesystem or in the <a href="https://apify.com" target="_blank">Apify Cloud</a>.
The queue is used for deep crawling of websites, where you start with
several URLs and then recursively follow links to other pages.
The data structure supports both breadth-first and depth-first crawling orders.
diff --git a/test/webdriver.js b/test/webdriver.js
index 77e209a6b..da76d7a72 100644
--- a/test/webdriver.js
+++ b/test/webdriver.js
@@ -335,7 +335,7 @@ describe('Apify.browse()', function () {
console.log('HEREE');
let browser;
const opts = {
- url: 'https://www.apify.com/',
+ url: 'https://apify.com/',
headless: false,
browserName: 'chrome',
extraChromeArguments: ['--disk-cache-dir=/Users/jan/Projects/apify-runtime-js/_profiles/xxx1'], //, '--disk-cache-size=10000000'],
diff --git a/website/core/Footer.js b/website/core/Footer.js
index 36488a1cc..fdff1e47a 100644
--- a/website/core/Footer.js
+++ b/website/core/Footer.js
@@ -73,7 +73,7 @@ class Footer extends React.Component {
</div>
<div>
<h5>More</h5>
- <a href="https://www.apify.com" target="_blank">Apify Cloud</a>
+ <a href="https://apify.com" target="_blank">Apify Cloud</a>
<a href="https://docusaurus.io" target="_blank">Docusaurus</a>
<a href={this.props.config.repoUrl} target="_blank">GitHub</a>
</div>