Skip to content

armgit / mwmbl

Public Imported from GitHub
0

An open source, non-profit web search engine

Commits on main

1d8573f
b2fed3e
69b4abe
e947c9e
Improve exception
Daoud Clarke committed 2 years ago
eeb8ee5
Remove the limit on number of fields allowed
Daoud Clarke committed 2 years ago
86efa8e
Process more batches when updating URLs
Daoud Clarke committed 2 years ago
fbf412e
0d8bd11
Merge pull request #183 from mwmbl/validate-domains
Daoud Clarke committed 2 years ago
96ba3c5
Validate domains
Daoud Clarke committed 2 years ago
3c0caa8
Add domain submission to model admin
Daoud Clarke committed 2 years ago
9c88711
fb426fa
Refactor to avoid using Django models in unit tests
Daoud Clarke committed 2 years ago
3fa4203
Add curated domains to the crawl queue
Daoud Clarke committed 2 years ago
a26fa62
a4cfe4c
List submissions and show on domain page
Daoud Clarke committed 2 years ago
493b58c
Allow domain submissions
Daoud Clarke committed 2 years ago
9968e48
Merge pull request #181 from mwmbl/count-urls-faster
Daoud Clarke committed 2 years ago
261ba37
Use multiprocessing to speed up counting
Daoud Clarke committed 2 years ago
86e91ca
Parse URLs faster using a regex
Daoud Clarke committed 2 years ago
9201c76
Count URLs faster
Daoud Clarke committed 2 years ago
7ecd930
Add rankeval
Daoud Clarke committed 2 years ago
3dcca31
Merge pull request #180 from mwmbl/curations-page
Daoud Clarke committed 2 years ago
d14fabb
Add a view for all curations
Daoud Clarke committed 2 years ago
f83432a
Add missing stats
Daoud Clarke committed 2 years ago
e14ebe5
Fix bug in URL db
Daoud Clarke committed 2 years ago
ba9b4ef
Merge pull request #179 from mwmbl/recrawl-old-pages
Daoud Clarke committed 2 years ago
86047d5
Fix check for whether a URL has been crawled before
Daoud Clarke committed 2 years ago
94d9c1b