Skip to content

armgit / mwmbl

Public Imported from GitHub
0

An open source, non-profit web search engine

Commits on main

Implement new indexing approach
Daoud Clarke committed 4 years ago
1bceeae
Merge pull request #68 from mwmbl/fix-missing-query
Daoud Clarke committed 4 years ago
27a4784
Log at info level
Daoud Clarke committed 4 years ago
5ce333c
a097ec9
Enough preprocessing
Daoud Clarke committed 4 years ago
cfca015
Run preprocessing
Daoud Clarke committed 4 years ago
003cd21
Just index a single page for now
Daoud Clarke committed 4 years ago
bcd3132
Run update
Daoud Clarke committed 4 years ago
ce9f522
Catch corrupt data
Daoud Clarke committed 4 years ago
09a9390
Add util script to send batch; add logging
Daoud Clarke committed 4 years ago
93307ad
3c97fdb
Fix unicode encoding error
Daoud Clarke committed 4 years ago
680fe1c
10x index size
Daoud Clarke committed 4 years ago
fee5cbb
Fix issue #60
milovanderlinden committed 4 years ago
dfd3f39
Don't include web.archive.org as a curated domain
Daoud Clarke committed 4 years ago
dba50b3
Add a URL length penalty
Daoud Clarke committed 4 years ago
43815c7
Score domain and path, weight components
Daoud Clarke committed 4 years ago
a3ff2f5
4b5df76
Record new batches as being local
Daoud Clarke committed 4 years ago
f9fefa0
Allow crawling links from unknown domains
Daoud Clarke committed 4 years ago
e578d55
Merge pull request #55 from mwmbl/index-continuously
Daoud Clarke committed 4 years ago
4967830
Don't require a slash for the search URL
Daoud Clarke committed 4 years ago
db1aa1a
Actually used the passed in timestamp
Daoud Clarke committed 4 years ago
24f82a3