Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Bump client version, update tets, bump version, update changelog
Ondra Urbanff3163928a0b
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 67b63945a..598425952 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -3,7 +3,8 @@In this release we've changed a lot of code, but you may not even notice.We've updated the underlying `apify-client` package which powers all communication withthe Apify API to version `1.0.0`. This means a completely new API for all internal calls.-If you use `Apify.client` calls in your code, visit the client docs for a migration guide. <----- TODO+If you use `Apify.client` calls in your code, this will be a large breaking change for you.+Visit the client docs for a migration guide. <----- TODOUntil now, local emulation of Apify Storages was part of the SDK. We've moved the logicinto a separate package `@apify/storage-local` which shares interface with `apify-client`.@@ -18,14 +19,22 @@ instead of `LaunchPuppeteerOptions` and don't realize it. Before this version, Slet you know and just silently continue with Chromium. Now, it will throw an error sayingthat `useChrome` is not an allowed property of `PuppeteerPoolOptions`.+Based on developer feedback, we decided to remove `--no-sandbox` from the default Puppeteer+launch args. It will only be used on Apify Platform. This gives you the chance to use+your own sandboxing strategy.+`LiveViewServer` and `puppeteerPoolOptions.useLiveView` were never very user-friendlyor performant solutions, due to the inherent performance issues with rapidly taking manyscreenshots in Puppeteer. We've decided to remove it. If you need similar functionality,try the `devtools-server` NPM package, which utilizes the Chrome DevTools Frontend forscreen-casting live view of the running browser.-Finally, we cleaned up some old deprecations.+Full list of changes:+- **BREAKING:** Updated `apify-client` to `1.0.0` with a completely new interface.+ This means that all client calls made using `Apify.client` will break.+- **BREAKING:** Removed `--no-sandbox` from default Puppeteer launch arguments.+ This will most likely be breaking for Linux and Docker users.- **BREAKING:** Function argument validation is now more strict and will not accept extraparameters which are not defined by the functions' signatures.- **DEPRECATED:** `puppeteerPoolOptions.useLiveView` is now deprecated.diff --git a/jest.config.js b/jest.config.jsindex 44e5ec7b8..5de837f5e 100644--- a/jest.config.js+++ b/jest.config.js@@ -4,7 +4,6 @@ module.exports = {testEnvironment: 'node',testRunner: 'jest-circus/runner',verbose: true,- testEnvironment: 'node',rootDir: path.join(__dirname, './'),testMatch: ['**/?(*.)+(spec|test).[tj]s?(x)'],transform: {diff --git a/package.json b/package.jsonindex c9ceb4b30..ebcbe5bcc 100644--- a/package.json+++ b/package.json@@ -60,7 +60,7 @@"@types/puppeteer": "^3.0.2","@types/socket.io": "^2.1.11","@types/tough-cookie": "^4.0.0",- "apify-client": "1.0.0-beta.10",+ "apify-client": "1.0.0-beta.11","apify-shared": "^0.5.0","cheerio": "^1.0.0-rc.3","content-type": "^1.0.4",diff --git a/test/utils.test.js b/test/utils.test.jsindex 6c479b971..bd2eb5b74 100644--- a/test/utils.test.js+++ b/test/utils.test.js@@ -20,7 +20,7 @@ describe('utils.newClient()', () => {expect(client.constructor.name).toBe('ApifyClient');expect(client.token).toBe('token');- expect(client.baseUrl).toBe('http://www.example.com:1234/path');+ expect(client.baseUrl).toBe('http://www.example.com:1234/path/v2');});test('uses correct default if APIFY_API_BASE_URL is not defined', () => {