Apache Nutch is an extensible and scalable web crawler
How to Upgrade OkHttp in Nutch
Upgrade OkHttp dependency in src/plugin/protocol-okhttp/ivy.xml
Upgrade OkHttp's own dependencies in src/plugin/protocol-okhttp/plugin.xml
To get the list of dependencies and their versions execute in the Nutch root folder: $ ant clean runtime $ ls build/plugins/protocol-okhttp/
| grep '.jar$'
| grep -vF protocol-okhttp.jar
| sed 's/^/ <library name="/g' | sed 's/$/"/>/g' In the plugin.xml replace all lines betweenand
with the output of the command above.
Build Nutch and run all unit tests:
$ ant clean runtime test
At least, run the protocol-okhttp unit tests:
$ ant test-plugin -Dplugin=protocol-okhttp
(optionally but recommended) Run a test crawl using protocol-okhttp