Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Commits on main

Nick:
Nicolas committed 2 years ago
22848af
Nick:
Nicolas committed 2 years ago
ebe9de2
Update index.test.ts
Nicolas committed 2 years ago
5056dcd
tests
rafaelmmiller committed 2 years ago
be5e6da
Update extract.ts
Nicolas committed 2 years ago
796cd07
Update extract.ts
Nicolas committed 2 years ago
1b5f6a0
Nick: refactor and /* glob pattern support
Nicolas committed 2 years ago
d6749c2
sdk allowexternallinks
rafaelmmiller committed 2 years ago
41b45a8
sdks wip
rafaelmmiller committed 2 years ago
80d6cb1
fix(cache): don't cache on failure error code
Gergő Móricz committed 2 years ago
359c30f
feat: cache
Gergő Móricz committed 2 years ago
49ff37a
fix(sitemap): scrape with tlsclient
Gergő Móricz committed 2 years ago
86a78a0
Create README.md
Eric Ciarla committed 2 years ago
62c8b63
fix(scrapeURL): adjust error message for clarity
Móricz Gergő committed 2 years ago
5519f07
fix(html-to-markdown): make error reporting less intrusive
Móricz Gergő committed 2 years ago
0a1c990
Nick: changed email from hello to help
Nicolas committed 2 years ago
bd928b1
Merge branch 'main' into nsc/new-extract
Nicolas committed 2 years ago
a1c018f
wip
rafaelmmiller committed 2 years ago
904c904
fix(crawl): redirect rebase
Gergő Móricz committed 2 years ago
0310cd2
Update package.json
Nicolas committed 2 years ago
0d1c4e4
feat(v1/webhook): complex webhook object w/ headers (#899)
Gergő Móricz committed 2 years ago
32be2cf
mvp done?
rafaelmmiller committed 2 years ago
25f3200
wip
rafaelmmiller committed 2 years ago
a175c15
Update email_notification.ts
Nicolas committed 2 years ago
1a636b4
fix(crawl): initialURL setting is unnecessary
Gergő Móricz committed 2 years ago
5ce4aaf
fix(queue-worker): do not kill crawl on one-page error
Gergő Móricz committed 2 years ago
93ac20f
fix(scrapeURL/pdf,docx): ignore SSL when downloading PDF
Gergő Móricz committed 2 years ago
16e8502