Skip to content

web search done right

Commits on main

mapreduce types doesn't have to be sync
Mikkel Denker committed 4 years ago
da66ffa
7db1d92
various webgraph insert + merge optimizations
Mikkel Denker committed 4 years ago
5b98e49
parsing speedup by using tl crate if possible
Mikkel Denker committed 4 years ago
002981d
serialize/deserialize webgraph
Mikkel Denker committed 4 years ago
40a2c2c
speedup webgraph with caching
Mikkel Denker committed 4 years ago
6f66d04
mapreduce success async
Mikkel Denker committed 4 years ago
1c067a2
mapreduce is now way more robust to failures
Mikkel Denker committed 4 years ago
e1a8ff3
forgot a todo item for mapreduce
Mikkel Denker committed 4 years ago
2040134
simple mapreduce without much error handling
Mikkel Denker committed 4 years ago
344dc3f
new mode: webgraph builder
Mikkel Denker committed 4 years ago
718034f
moved warc download into warcfile
Mikkel Denker committed 4 years ago
4433cc8
node iterator refactoring
Mikkel Denker committed 4 years ago
58d2ee4
merge graphs
Mikkel Denker committed 4 years ago
6ce0b10
host-level graph
Mikkel Denker committed 4 years ago
d90888d
tokenizer module
Mikkel Denker committed 4 years ago
86b07e3
navigational ranking only applies to homepage
Mikkel Denker committed 4 years ago
19916da
less confusion between searcher and tantivy::Searcher
Mikkel Denker committed 4 years ago
06fde74
refactoring
Mikkel Denker committed 4 years ago
43fb159
navigational query
Mikkel Denker committed 4 years ago
8f98ebd
simple snippet generation
Mikkel Denker committed 4 years ago
2870c5f
Added license to cargo.toml
Mikkel Denker committed 4 years ago
0d97184
Added license to source code files
Mikkel Denker committed 4 years ago
7eb4ea8
Added LICENSE.md
Mikkel Denker committed 4 years ago
494c6f1