Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Add release docs
Ondra Urban78d7eb6c3ac3
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6716e2841..fb3c964b9 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,7 @@NEXT====================+- **BREAKING CHANGE** Removed support for Web Driver (Selenium) since no further updates are planned.+ If you wish to continue using Web Driver, please stay on Apify SDK version ^0.14.15- Improve logging of memory overload errors.- Fix multiple log lines appearing when a crawler was about to finish.- Add `Apify.utils.puppeteer.enqueueLinksByClickingElements()` function which enables youdiff --git a/CHECKLIST.md b/CHECKLIST.mddeleted file mode 100644index 44ba38e43..000000000--- a/CHECKLIST.md+++ /dev/null@@ -1,26 +0,0 @@--# Checklist of things to do to publish new version---## Beta release--Bump `package.json` version, updated `CHANGELOG.md` and commit to develop branch.--Run `publish.sh` to publish the package on NPM--Upgrade packages in [Apify Docker images](https://github.com/apifytech/apify-actor-docker),-**make sure the `puppeteer` version matches the version in Apify SDK** !!! After commit, the new BETA-images are built automatically.--On major version upgrade, ensure that [Apify CLI templates](https://github.com/apifytech/apify-cli/tree/master/src/templates)-use the latest version.--Upgrade packages in Apify integration tests and run them using:--```-./check.sh BETA_PACKAGES=true ./check.sh prod 2-```--## Latest release--TODOdiff --git a/RELEASE.md b/RELEASE.mdnew file mode 100644index 000000000..ef8c77343--- /dev/null+++ b/RELEASE.md@@ -0,0 +1,52 @@+# How to release new versions of Apify SDK+Release of new versions is managed by Travis CI. On pushes to the `master` branch, prerelease versions+are automatically produced. Latest releases are triggered manually through the GitHub release tool.+After creating a release there, Travis will automatically produce a latest version of the package.++## TLDR;+- To release `beta`, just push to `master`. If it breaks with a `Version already exists error` increment version+ in `package.json` and push again.+- To release `latest`, go to releases on GitHub, draft and publish a release. If you don't know how, read below.++## Prerelease (beta) versions+On each push to the `master` branch, a new prerelease version is automatically built and published+by Travis CI. The process is as follows:++1. Travis build is triggered by a push to `master` (typically a merge of a PR).+2. Travis lints the source code, checks that documentation is built and runs tests in Node.js 8, 10 and 12.+3. If all is well, a new prerelease version is published to NPM (`${VERSION}-beta.${COUNTER}`),+ where `VERSION` is the version in package.json and `COUNTER` is a zero based index of existing prereleases.+ Example: `0.15.1-beta.3`.+4. The package is tagged with the `beta` NPM tag and a Git tag is associated with the triggering commit.+5. A build of Apify docker images is triggered that updates the `beta` packages to use the newly published package.+6. All done and ready to use.++### Updating a release version+When releasing breaking changes, new features or for any other reason that requires a version bump,+manually increment the version in the `package.json` file. Such as from `0.14.15` to `0.15.0`.+This will automatically trigger a prerelase build with the `0.15.0-beta.0` version.++### Existing versions+Travis will not allow you to publish a prerelease of a version that's already published. For example,+if version `0.14.15` already exists on NPM, you can no longer release a `0.14.15-beta.0` version.++## Latest release+To trigger a latest release, go to the GitHub release tool (select `releases` under `<> Code`).+There, draft a new release, fill the form (see below) and hit `Publish release`. Travis CI will automatically+pick up this action and release a latest version of the package.++### How to fill the form+- The version tag should be in the format `v${VERSION}` where `VERSION` is the version from `package.json`.+ Such as `v0.15.0` or `v0.16.17`.+- The target will typically be `master`, but you can also release any previous commit by selecting it or searching+ for it by ID. This is useful when there have been some changes in master from the latest prerelease and you'd+ like to release an older prerelease version as latest. You can find the commit ID easily by searching for the+ prerelease tag.+- The title should be the same as the version tag.+- Typically just adding changelog to the release would be fine, but feel free to add extra information.++## Release process+Similarly to the prerelease, the latest release process:+1. Triggers build, lints, checks docs, runs tests.+2. Publishes new package to NPM with the `latest` tag and the version from package.json. Such as `0.15.1`.+3. Builds Apify docker images that updates the `latest` packages to the newly published package.