Apache Nutch is an extensible and scalable web crawler
Prepare for new development after release of 1.23 - update version number - update changes / release notes
Sebastian Nagel8006e1094e74
diff --git a/CHANGES.md b/CHANGES.mdindex d34b61644..72e7f5f5d 100644--- a/CHANGES.md+++ b/CHANGES.md@@ -1,5 +1,81 @@# Nutch Change Log+## Nutch 1.23 Release 07/08/2026 (dd/mm/yyyy)+Release Report: <https://s.apache.org/c4qjb>++### Breaking Changes++- Nutch requires JDK 17 at build and runtime.+- The minimum required Hadoop version is 3.5.0 if Nutch is run on a Hadoop cluster.+- [NUTCH-1732](https://issues.apache.org/jira/browse/NUTCH-1732) introduced a new CrawlDatum status,+ `db_parse_failed`. A CrawlDb written with Nutch 1.23 might fail to be read with older Nutch versions,+ if it contains records with status `db_parse_failed`. As the name indicates, the status marks+ documents which failed to parse. It is only set if the property `parser.delete.failed.parse` is+ `true` (default: `false`). If your workflow requires that Nutch data can be read with Nutch 1.22+ (or older), please keep the property set to `false`.++### Sub-task++- [NUTCH-2932](https://issues.apache.org/jira/browse/NUTCH-2932) - Create OpenAPI specification for Nutch 1.x REST API+- [NUTCH-3053](https://issues.apache.org/jira/browse/NUTCH-3053) - Upgrade build and CI to JDK17++### Bug++- [NUTCH-1446](https://issues.apache.org/jira/browse/NUTCH-1446) - Port NUTCH-1444 to trunk (Indexing should not create temporary files)+- [NUTCH-1732](https://issues.apache.org/jira/browse/NUTCH-1732) - IndexerMapReduce to delete explicitly not indexable documents+- [NUTCH-3160](https://issues.apache.org/jira/browse/NUTCH-3160) - Leftover System.exit(..) in CommonCrawlDataDumper+- [NUTCH-3162](https://issues.apache.org/jira/browse/NUTCH-3162) - Fetcher and Parser latency metrics to properly merge data from all threads and tasks+- [NUTCH-3164](https://issues.apache.org/jira/browse/NUTCH-3164) - Generic exceptions in catch block may lead to deletion of links from crawldb+- [NUTCH-3168](https://issues.apache.org/jira/browse/NUTCH-3168) - Sandbox Commons JEXL usage in crawl and index pipelines+- [NUTCH-3174](https://issues.apache.org/jira/browse/NUTCH-3174) - protocol-okhttp: request may hang despite http.time.limit is set+- [NUTCH-3177](https://issues.apache.org/jira/browse/NUTCH-3177) - Fetcher to report idle threads not as hung threads+- [NUTCH-3180](https://issues.apache.org/jira/browse/NUTCH-3180) - BasicURLNormalizer missing catching ICUInputTooLongException+- [NUTCH-3186](https://issues.apache.org/jira/browse/NUTCH-3186) - Split SonarQube workflow in unprivileged build and privileged analysis part+- [NUTCH-3187](https://issues.apache.org/jira/browse/NUTCH-3187) - protocol-okhttp: IP address filter fails to compare first byte for /32 resp. /128 CIDRs+- [NUTCH-3188](https://issues.apache.org/jira/browse/NUTCH-3188) - CI builds startup failure because of disallowed workflow actions+- [NUTCH-3191](https://issues.apache.org/jira/browse/NUTCH-3191) - Fix SonarCloud PR analysis: missing JaCoCo XML, test binaries path, and source encoding warnings+- [NUTCH-3193](https://issues.apache.org/jira/browse/NUTCH-3193) - Exclude Jacoco files from source release package+- [NUTCH-3201](https://issues.apache.org/jira/browse/NUTCH-3201) - Fetcher launches only a single reduce task, ignoring the configuration++### New Feature++- [NUTCH-3085](https://issues.apache.org/jira/browse/NUTCH-3085) - Augment CI by adding code coverage and code quality reporting+- [NUTCH-3154](https://issues.apache.org/jira/browse/NUTCH-3154) - Implement integration testing framework for Nutch IndexWriter plugins using Testcontainers+- [NUTCH-3163](https://issues.apache.org/jira/browse/NUTCH-3163) - Integrate Apache Yetus' pre-commit patch testing into Nutch GitHub Continuous Integration+- [NUTCH-3176](https://issues.apache.org/jira/browse/NUTCH-3176) - URLUtil and urlnormalizer-basic: add support for IDNA2008++### Improvement++- [NUTCH-2987](https://issues.apache.org/jira/browse/NUTCH-2987) - Upgrade to Java 17+- [NUTCH-3138](https://issues.apache.org/jira/browse/NUTCH-3138) - Upgrade to Creadur RAT 0.17+- [NUTCH-3145](https://issues.apache.org/jira/browse/NUTCH-3145) - Upgrade to JUnit 6+- [NUTCH-3167](https://issues.apache.org/jira/browse/NUTCH-3167) - Upgrade to Hadoop 3.5.0+- [NUTCH-3173](https://issues.apache.org/jira/browse/NUTCH-3173) - protocol-okhttp: store OkHttp's internal URL in response metadata+- [NUTCH-3175](https://issues.apache.org/jira/browse/NUTCH-3175) - Implement integration testing framework for Nutch Protocol plugins using Testcontainers+- [NUTCH-3178](https://issues.apache.org/jira/browse/NUTCH-3178) - Add AGENTS.md and SECURITY.md to the Nutch repository+- [NUTCH-3182](https://issues.apache.org/jira/browse/NUTCH-3182) - Add GitHub Dependabot configuration to update GitHub workflows+- [NUTCH-3183](https://issues.apache.org/jira/browse/NUTCH-3183) - Upgrade Tika to 3.3.1+- [NUTCH-3184](https://issues.apache.org/jira/browse/NUTCH-3184) - Upgrade OkHttp to 3.4.0+- [NUTCH-3185](https://issues.apache.org/jira/browse/NUTCH-3185) - Upgrade NekoHTML to 1.9.22+- [NUTCH-3189](https://issues.apache.org/jira/browse/NUTCH-3189) - Upgrade core dependencies+- [NUTCH-3192](https://issues.apache.org/jira/browse/NUTCH-3192) - Address Yetus detsecrets/blanks and SonarCloud security findings in config templates and DummySSL++### Test++- [NUTCH-3129](https://issues.apache.org/jira/browse/NUTCH-3129) - Investigate empty TEST-org.apache.nutch.util.TestMimeUtil.xml result++### Wish++- [NUTCH-3166](https://issues.apache.org/jira/browse/NUTCH-3166) - Create a security page++### Task++- [NUTCH-3064](https://issues.apache.org/jira/browse/NUTCH-3064) - Upgrade index-geoip to GeoIP2 5.0.2+- [NUTCH-3104](https://issues.apache.org/jira/browse/NUTCH-3104) - [SECURITY] please replace use of org.codehaus.jackson+- [NUTCH-3157](https://issues.apache.org/jira/browse/NUTCH-3157) - Add indexer plugin integration testing guidance to the Nutch wiki+- [NUTCH-3165](https://issues.apache.org/jira/browse/NUTCH-3165) - Remove the Nutch web service++## Nutch 1.22 Release 12/02/2026 (dd/mm/yyyy)Release Report: https://s.apache.org/1ip89diff --git a/conf/nutch-default.xml b/conf/nutch-default.xmlindex 45d45163a..2ddab8322 100644--- a/conf/nutch-default.xml+++ b/conf/nutch-default.xml@@ -222,7 +222,7 @@<property><name>http.agent.version</name>- <value>Nutch-1.23-SNAPSHOT</value>+ <value>Nutch-1.24-SNAPSHOT</value><description>A version string to advertise in the User-Agentheader.diff --git a/default.properties b/default.propertiesindex d4b8f8062..b1d96aef6 100644--- a/default.properties+++ b/default.properties@@ -14,7 +14,7 @@# limitations under the License.name=apache-nutch-version=1.23-SNAPSHOT+version=1.24-SNAPSHOTfinal.name=${name}-${version}year=2026diff --git a/src/bin/nutch b/src/bin/nutchindex 87018ec32..255a8c43e 100755--- a/src/bin/nutch+++ b/src/bin/nutch@@ -61,7 +61,7 @@ done# if no args specified, show usageif [ $# = 0 ]; then- echo "nutch 1.23-SNAPSHOT"+ echo "nutch 1.24-SNAPSHOT"echo "Usage: nutch COMMAND [-Dproperty=value]... [command-specific args]..."echo "where COMMAND is one of:"echo " (Crawl commands)"