Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

fix: hardcode public API URL in key-value store URL getter (#1108)

Martin Adámek4e2bd76078b2
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cdf335aa6..f53a58db0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,10 @@
====================
- **BREAKING**: Require Node.js >=15.10.0 because HTTP2 support on lower Node.js versions is very buggy.
+1.3.3 / 2021/08/04
+====================
+- Fix public URL getter of key-value stores
+
1.3.2 / 2021/08/02
====================
- Fix `headerGeneratorOptions` not being passed to `got-scraping` in `requestAsBrowser`.
diff --git a/src/constants.js b/src/constants.js
index 683fa5d22..62d287132 100644
--- a/src/constants.js
+++ b/src/constants.js
@@ -25,6 +25,12 @@ export const EXIT_CODES = {
*/
export const ACTOR_EVENT_NAMES_EX = { ...ACTOR_EVENT_NAMES, PERSIST_STATE: 'persistState' };
+/**
+ * Base URL of Apify's API endpoints.
+ * @type {string}
+ */
+export const APIFY_API_BASE_URL = 'https://api.apify.com/v2';
+
/**
* Additional number of seconds used in CheerioCrawler and BrowserCrawler to set a reasonable
* handleRequestTimeoutSecs for BasicCrawler that would not impare functionality (not timeout before crawlers).
diff --git a/src/storages/key_value_store.js b/src/storages/key_value_store.js
index f5e853cf0..7d0705b0a 100644
--- a/src/storages/key_value_store.js
+++ b/src/storages/key_value_store.js
@@ -10,6 +10,7 @@ import * as ApifyClient from 'apify-client';
// @ts-ignore
import { ApifyStorageLocal } from '@apify/storage-local';
import { Configuration } from '../configuration';
+import { APIFY_API_BASE_URL } from '../constants';
/* eslint-enable no-unused-vars,import/named,import/no-duplicates,import/order */
/**
@@ -266,7 +267,7 @@ export class KeyValueStore {
* @return {string}
*/
getPublicUrl(key) {
- return `${this.config.get('apiBaseUrl')}/v2/key-value-stores/${this.id}/records/${key}`;
+ return `${APIFY_API_BASE_URL}/key-value-stores/${this.id}/records/${key}`;
}
/**