Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
fix(readme): clarify that scrape_options must be a ScrapeOptions instance in crawl_url and typo fixes. (#1647)
117af9f Arkit committed 1 year ago
fix(js-sdk/tests): fix the testing situation (FIR-2253) (#1644)
a329117 Gergő Móricz committed 1 year ago
feat(gcs-jobs): ditch exists check to cut lookup time in half (#1641)
8ef3e84 Gergő Móricz committed 1 year ago
feat(apps/test-suite): add Rafa's index benchmark notebook
6e88737 Gergő Móricz committed 1 year ago
debug(api/scrape): increased logging to diagnose scrape fluke length
0c7f864 Gergő Móricz committed 1 year ago
feat(sdk): Index parameters + other missing parameters (#1638)
4337992 Gergő Móricz committed 1 year ago
feat(webhook): refactor callWebhook and add logWebhook (FIR-2218) (#1629)
78580f6 Gergő Móricz committed 1 year ago
feat(api/index): port queryIndexAtSplitLevel to RPC (FIR-2241) (#1640)
f050b16 Gergő Móricz committed 1 year ago
feat(scrapeURL/index): don't put results by "dumb" engines into the index
a08d52e Gergő Móricz committed 1 year ago
Merge pull request #1635 from mendableai/refactor/api-integration-parameter
b2e0f65 Ademílson Tonato committed 1 year ago
feat(api): propagate integration field in queue worker job processing
6e1f8d6 Ademílson Tonato committed 1 year ago
Merge pull request #1632 from mendableai/feat/api-integration-parameter
71caf8a Ademílson Tonato committed 1 year ago
refactor: remove unnecessary logs and set integration as default to null
4c49bb9 Ademílson Tonato committed 1 year ago
feat(api): add integration field to jobs and update related controllers and types
57a0aed Ademílson Tonato committed 1 year ago
feat(api/tests): add webhook tests + refactor batch scrape lib (#1630)
077c5dd Gergő Móricz committed 1 year ago