An open source, non-profit web search engine
Commits on main
Use URL path to store locally so that we can easily get a local path from a URL
a8a6c67 Daoud Clarke committed 4 years ago
Implement a batch cache to store files locally before preprocessing
0d1e7d8 Daoud Clarke committed 4 years ago
Use a more specific exception in case we're discarding ones we shouldn't
a471bc2 Daoud Clarke committed 4 years ago
Merge pull request #66 from mwmbl/fix-unicode-encode-error
3c97fdb Daoud Clarke committed 4 years ago
Merge pull request #61 from milovanderlinden/issue-60-consistent-use-of-env-vars
e1e1b00 Daoud Clarke committed 4 years ago
Merge pull request #58 from mwmbl/improve-ranking-for-root-domains
2e40ae1 Daoud Clarke committed 4 years ago
Merge pull request #57 from mwmbl/clear-indexed-documents
4b5df76 Daoud Clarke committed 4 years ago
Delete documents that have been preprocessed from the database to save space
9482ae5 Daoud Clarke committed 4 years ago
Merge pull request #56 from mwmbl/allow-links-from-unknown-domains
6fa192d Daoud Clarke committed 4 years ago