Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Commits on main

wip
rafaelmmiller committed 2 years ago
807703d
fix(scrapeURL/pdf): retry
Gergő Móricz committed 2 years ago
7081bef
fix(scrapeURL/pdf): fix llamaparse upload
Gergő Móricz committed 2 years ago
9ace2ad
fix(requests.http): default to localhost baseUrl
Gergő Móricz committed 2 years ago
687ea69
feat: improve requests.http using format features
Gergő Móricz committed 2 years ago
3a5eee6
fix(v0): crawl timeout errors
Gergő Móricz committed 2 years ago
f2ecf0c
Merge branch 'main' into nsc/new-extract
Nicolas committed 2 years ago
464b41a
Update extract.ts
Nicolas committed 2 years ago
a23364e
Nick:
Nicolas committed 2 years ago
a4f1526
fbabc77
Update extract.ts
Nicolas committed 2 years ago
d430cfc
Update fireEngine.ts
Nicolas committed 2 years ago
5bbbb52
feat(api): graceful shutdown for less 502 errors
Gergő Móricz committed 2 years ago
740a429
Nick:
Nicolas committed 2 years ago
540d4e5
fix(queue-worker): don't log timeouts
Gergő Móricz committed 2 years ago
c327d68
feat(scrapeURL): log URL for easy searching
Gergő Móricz committed 2 years ago
9f8b8c1
fix(scrapeURL): don't log fetch request
Gergő Móricz committed 2 years ago
e95b665
fix(scrapeURL): don't log engineResult
Gergő Móricz committed 2 years ago
f42740a
fix(scrape): better timeout handling
Móricz Gergő committed 2 years ago
3815d24
fix(queue-worker): logging job on batch scrape error
Móricz Gergő committed 2 years ago
aa9a47b
feat(batchScrape): handle timeout
Móricz Gergő committed 2 years ago
91f5228
fix(crawl-redis): batch scrape lockURL
Móricz Gergő committed 2 years ago
f6db9f1
fix(tests): maxDepth tests
Gergő Móricz committed 2 years ago
d8bb1f6
fix(crawl/maxDepth): fix maxDepth behaviour
Gergő Móricz committed 2 years ago
68c9615
Update package.json
Nicolas committed 2 years ago
7d576d1
fix(html-to-markdown): build error
Gergő Móricz committed 2 years ago
5cb46dc
fix(scrapeURL/llmExtract): fix schema-less LLM extract
Gergő Móricz committed 2 years ago
2ca2265
fix(html-to-markdown): reduce logging frequency
Gergő Móricz committed 2 years ago
56bebc8
fix(scrapeURL): reduce logs
Gergő Móricz committed 2 years ago
d13a2e7