Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Commits on main

c3fd13a
Update removeUnwantedElements.ts
Nicolas committed 1 year ago
07f4b71
Update extraction-service.ts
Nicolas committed 1 year ago
33632d2
Nick: rm vscode settings
Nicolas committed 1 year ago
27cc3db
Update queue-worker.ts
Nicolas committed 1 year ago
bd81b41
Nick: async background index
Nicolas committed 1 year ago
e6da214
Nick: url normalization + max metadata size
Nicolas committed 1 year ago
7a31306
Nick: index exploration
Nicolas committed 1 year ago
bf9d41d
71a8f74
Nick: rm .xml from isFile
Nicolas committed 1 year ago
8ae34a0
fix(scrapeURL/fetch): block loopback and link-local IPs
Gergő Móricz committed 1 year ago
4d1f92f
Update index.ts
Nicolas committed 1 year ago
e255301
c1fa5a4
Update index.ts
Nicolas committed 1 year ago
1eca61b
Update index.ts
Nicolas committed 1 year ago
f9d55ef
Nick: we are using runpod
Nicolas committed 1 year ago
b8d7f9f
Merge branch 'main' into mog/mineru
Nicolas committed 1 year ago
5fcf3fa
Merge pull request #1013 from yujunhui/main
Nicolas committed 1 year ago
65cf4cd
feat: return favicon url when scraping
Ademílson F. Tonato committed 1 year ago
a4cf814
Sitemap fixes (#1010)
Gergő Móricz committed 1 year ago
0421f81
Update __init__.py
Nicolas committed 1 year ago
6851281
Update v1-openapi.json
Nicolas committed 1 year ago
e8f0a22