web search done right
Commits on main
move spellcheck into separate api endpoint and only correct non-special terms. we don't ant to correct "site:...", "inurl:..." etc. parts of the query.
c34849c Mikkel Denker committed 2 years ago
actually skip disambiguation pages for entity index. turns out that none of the usual disambiguation elements from the online wiki are present in the .zim dump. instead, disambiguation pages seem to have a "<meta property='mw:PageProp/disambiguation'>" element which we can use.
f4e7d19 Mikkel Denker committed 2 years ago
Modal with long domains moved “do you like results from ...” too far to the right
c91cf3d Mikkel Denker committed 2 years ago
don't show discussions widget if there is a user applied optic
a4a3a8d Mikkel Denker committed 2 years ago
hopefully more robust way to detect disambiguation pages. this is difficult to test as there is no disambiguation pages in the dev .zim file. let's just try on prod
6bb7311 Mikkel Denker committed 2 years ago
if there are no simple terms in the query, we should use prefix of description (or body if there is no description) as snippet
e008e70 Mikkel Denker committed 2 years ago
properly strip ‘https://' and 'http://' prefix from explore
2205849 Mikkel Denker committed 2 years ago
get_webpage only looked at the response from the shard with the quickest response. this obviously caused webpages to not be found seemingly at random, which caused summarizer to fail
88c8cbc Mikkel Denker committed 2 years ago
deduplicate related entities by their images. looks better
b4159d5 Mikkel Denker committed 2 years ago
skip disambiguation pages when constructing entity index
6d3af18 Mikkel Denker committed 2 years ago
split webgraph server into host and page. allows us to host each graph on separate sets of servers.
788f92c Mikkel Denker committed 2 years ago
Move entity index out of normal search index and have dedicated search server for it
cc91935 Mikkel Denker committed 2 years ago
Move widget, discussions etc out from api searcher and into separate api endpoints. This creates a better separation between the frontend and backend, and avoids e.g. 'fetch_discussions' abstraction leak. It also allows for these endpoints to be used in different contexts separately without having to perform a full search.
14e5466 Mikkel Denker committed 2 years ago
[serp] if there aren't many results for a search on mobile, the results would be shown far down on the page. this fixes the css-grid so the first result is placed at the top
eab4612 Mikkel Denker committed 2 years ago
prepare index writers in functions that uses the writer where possible. this removes the annoying footgun of having a panic if we for instance forgot to prepare the writer for a new index after merge...
f501e4a Mikkel Denker committed 2 years ago
key count approximation doesn't help much if we iterate nodes right after...
a5a675f Mikkel Denker committed 2 years ago
use estimate of number of nodes during approx harmonic calculation
fa6c48f Mikkel Denker committed 2 years ago