Skip to content

armgit / mwmbl

Public Imported from GitHub
0

An open source, non-profit web search engine

Commits on main

Add old index migration
Daoud Clarke committed 2 years ago
52f78cd
Simplify old index
Daoud Clarke committed 2 years ago
f96bd87
Merge branch 'main' into copy-index-app
Daoud Clarke committed 2 years ago
d1d088f
d5a5318
5a54edd
Update dependencies
Daoud Clarke committed 2 years ago
0800f9b
Merge branch 'main' into copy-index-app
Daoud Clarke committed 2 years ago
0f1d846
Delete domain scores when getting batches
Daoud Clarke committed 2 years ago
15bd43e
Update the old index info
Daoud Clarke committed 2 years ago
9719895
Rename background task update_batches
Daoud Clarke committed 2 years ago
3ccd4ec
Remove run background settings config option
Daoud Clarke committed 2 years ago
56e8aa3
Delete old URL queue
Daoud Clarke committed 2 years ago
c0e1f4d
Run URL update as a separate app
Daoud Clarke committed 2 years ago
77a9a03
WIP: implement Redis-based queue
Daoud Clarke committed 2 years ago
5530d03
Merge pull request #159 from mwmbl/copy-index
Daoud Clarke committed 2 years ago
6d1e781
Add a test for copying pages
Daoud Clarke committed 2 years ago
11d388b
Refactor and get pages in the right format
Daoud Clarke committed 2 years ago
bc27518
WIP: index pages from old index in new one
Daoud Clarke committed 2 years ago
393ef7e
Catch exceptions when updating URL queue
Daoud Clarke committed 2 years ago
a585709
684c070
Add a comment explaining what we're doing
Daoud Clarke committed 2 years ago
4ba25a7
Ensure Document title and extract are never None
Daoud Clarke committed 2 years ago
f01ae42
423c808
Handle the case where title or extract are None
Daoud Clarke committed 2 years ago
67c5192
b26576a
Better logging for confusing error
Daoud Clarke committed 2 years ago
0477c27
Merge pull request #152 from diagonalo/main
Daoud Clarke committed 2 years ago
b06927f