An open source, non-profit web search engine
Commits on main
Fix logic in found URL logic SQL and allow crawling URLs crawled by one user for now
36b168a Daoud Clarke committed 4 years ago
Temporarily disable startup background processes; add root domains; check for empty batches.
5e1ec9c Daoud Clarke committed 4 years ago
Tokenize documents and store pages to be added to the index
4330551 Daoud Clarke committed 4 years ago
WIP: index continuously. Retrieve batches and store in Postgres
9594915 Daoud Clarke committed 4 years ago