Skip to content
Arm
Git
Search or jump to…
Ctrl K
Categories
Theme: System
Sign in
Get started
armgit
/
mwmbl
Public
Imported from GitHub
Fork
0
Watch
0
Star
0
An open source, non-profit web search engine
Python
non-profit
search-engine
Code
Commits
Issues
Pull requests
Wiki
Builds
Discussions
Projects
Insights
Search
main
73 branches · 0 tags
Go to file
t
Code
Commits on
main
Return updated URLs
Daoud Clarke committed 3 years ago
01f08fd
Copy SHA
Don't try and update an empty list of URLs
Daoud Clarke committed 3 years ago
bd0cc38
Copy SHA
Update URL queue separately from the other background process to speed it up
Daoud Clarke committed 3 years ago
d347a17
Copy SHA
Fix some bugs in URL fetching query
Daoud Clarke committed 3 years ago
7bd12c1
Copy SHA
Fix postgres install
Daoud Clarke committed 3 years ago
a50f1d8
Copy SHA
Install postgres client
Daoud Clarke committed 3 years ago
1ab16b1
Copy SHA
Add core domains
Daoud Clarke committed 3 years ago
dda5a25
Copy SHA
Exclude google plus
Daoud Clarke committed 3 years ago
ab37bbe
Copy SHA
Allow posting extra links with lower score weighting
Daoud Clarke committed 3 years ago
2336ed7
Copy SHA
Check the domain is correct, potential bug in psql
Daoud Clarke committed 3 years ago
6edf486
Copy SHA
Tidy, improve logging
Daoud Clarke committed 3 years ago
b798468
Copy SHA
Update the URL queue earlier
Daoud Clarke committed 3 years ago
7c14cd9
Copy SHA
Merge pull request #86 from mwmbl/improve-crawling
Daoud Clarke committed 3 years ago
0d33b4f
Copy SHA
Reinstate background tasks
Daoud Clarke committed 3 years ago
a86e172
Copy SHA
Get results from other domains
Daoud Clarke committed 3 years ago
d9cd3c5
Copy SHA
Update URL status
Daoud Clarke committed 3 years ago
77f08d8
Copy SHA
Sample domains
Daoud Clarke committed 3 years ago
36af579
Copy SHA
WIP: improve method of getting URLs for crawling
Daoud Clarke committed 3 years ago
ea16e7b
Copy SHA
WIP: improve method of getting URLs for crawling
Daoud Clarke committed 3 years ago
7dae39b
Copy SHA
Don't delete an index if the sizes don't match
Daoud Clarke committed 3 years ago
c69108c
Copy SHA
Number of pages is an int
Daoud Clarke committed 3 years ago
bb8a36a
Copy SHA
Merge branch 'master' of github.com:mwmbl/mwmbl
Daoud Clarke committed 3 years ago
c01129c
Copy SHA
Use the correct storage location in prod
Daoud Clarke committed 3 years ago
26351a1
Copy SHA
Merge pull request #83 from omasanori/spacy-deps-rework
Daoud Clarke committed 3 years ago
f3f3831
Copy SHA
Rework installation of spaCy models for clarity
Masanori Ogino committed 3 years ago
71187a3
Copy SHA
Remove apt command
Daoud Clarke committed 3 years ago
d85067e
Copy SHA
Put install in correct place
Daoud Clarke committed 3 years ago
1ef60e8
Copy SHA
Install psql client
Daoud Clarke committed 3 years ago
8e613dd
Copy SHA
Exclude a domain
Daoud Clarke committed 3 years ago
80282cf
Copy SHA
Format fetched url
Daoud Clarke committed 3 years ago
8676abb
Copy SHA
Newer
Older