Skip to content

armgit / mwmbl

Public Imported from GitHub
0

An open source, non-profit web search engine

Commits on main

Use API key for crawler
Daoud Clarke committed 2 years ago
6560686
Revert "Remember which pages we indexed"
Daoud Clarke committed 2 years ago
6d1b8f1
Remember which pages we indexed
Daoud Clarke committed 2 years ago
c949cfb
Define schema for response
Daoud Clarke committed 2 years ago
c980b91
Return as Django Ninja expects
Daoud Clarke committed 2 years ago
f07628a
Handle missing API key
Daoud Clarke committed 2 years ago
1339451
Django doesn't support lifespan
Daoud Clarke committed 2 years ago
45b0b03
Add Sentry
Daoud Clarke committed 2 years ago
a5cb43e
Make justext part of mwmbl to make building easier
Daoud Clarke committed 2 years ago
ccd457a
Add local justext
Daoud Clarke committed 2 years ago
84068d0
Merge branch 'main' into new-crawler
Daoud Clarke committed 2 years ago
e2aec51
Merge pull request #199 from mwmbl/load-test
Daoud Clarke committed 2 years ago
02b50c5
Disable LTR for now to save CPU
Daoud Clarke committed 2 years ago
232a67d
Update gitigore, add TODO
Daoud Clarke committed 2 years ago
eab581f
Handle errors from wiki search
Daoud Clarke committed 2 years ago
c727e51
Merge remote-tracking branch 'codeberg/main'
Daoud Clarke committed 2 years ago
3c521b2
Update README.md
Binzy_Boi committed 2 years ago
6d39843
Max query length
Daoud Clarke committed 2 years ago
33e7104
Post results
Daoud Clarke committed 2 years ago
ebfc0ec
Don't try and get curated domains locally
Daoud Clarke committed 2 years ago
119fc9d
Temporarily disable crawling
Daoud Clarke committed 2 years ago
40e5cba
Get top terms currently indexed
Daoud Clarke committed 2 years ago
c865368
Remove some todos
Daoud Clarke committed 2 years ago
15e90a3
Sort documents using heuristic ranking
Daoud Clarke committed 2 years ago
4ce3663
Score documents properly
Daoud Clarke committed 2 years ago
b0eb9b7
Multiprocess
Daoud Clarke committed 2 years ago
2278b26
First implementation of crawler
Daoud Clarke committed 2 years ago
246c90f
Merge pull request #197 from mwmbl/reduce-db-load
Daoud Clarke committed 2 years ago
5870cc9
Manually use cache instead of broken decorator
Daoud Clarke committed 2 years ago
b512b7f