web search done right
Commits on main
rename '@type' to 'type' in api since some openapi generators gets confused
45f213a Mikkel Denker committed 3 years ago
Ability to limit number of words considered in snippet generation. Some search results has a very large amount of text, which results in the vast majority of time being spent in snippet generation compared to search.
68ba133 Mikkel Denker committed 3 years ago
Partial support for compounded words. the query "wishlist" now also matches search results that has the terms "wish list". This is done using the bigram- and trigram fields.
7706dcd Mikkel Denker committed 3 years ago
sharded url state db to deal with domains that has a lot of urls
dc0390b Mikkel Denker committed 3 years ago
huggingface seems to have stealth updated the tokenizer crate
1f2308e Mikkel Denker committed 3 years ago
if there is no cleantext on site, then we probably can't create a good snippet anyway
00bc05c Mikkel Denker committed 3 years ago
reduce thresholds and minimum_clean_words to make it easier to trigger sidebars during development
47c84a0 Mikkel Denker committed 3 years ago
Add `#[serde(tag = "type", content = "value")]` to OpenAPI exposed types (#76)
5030446 Oliver Bøving committed 3 years ago
Fix the byte/char index mixups identified in issue 77 (#78)
d5624ee Mikkel Denker committed 3 years ago
no need to create 'warc_files' folder when downloading anymore
0ac07ab Mikkel Denker committed 3 years ago
indieweb tag extraction bug: should not match substring classes
7f28937 Mikkel Denker committed 3 years ago
optimize optic matching when the pattern consists of a single normal term and nothing else
d84c973 Mikkel Denker committed 3 years ago
Remove 'post' requirement from lemmy urls. It was too slow for some reason. Will need to investigate why, but lets disable it for the time being
11d18a3 Mikkel Denker committed 3 years ago
Support indieweb only optics. Still need to update the quickstart and blogroll optic to include the new match location.
7dbd149 Mikkel Denker committed 3 years ago
cannot use alpine when having strict CSP headers. User security > developer convenience
912dcc5 Mikkel Denker committed 3 years ago