Skip to content

The headless browser for AI agents and web scraping

2.8 KB
Raw Blame

--dump formats the page output without writing JavaScript.

obscura fetch https://example.com --dump html
obscura fetch https://example.com --dump text
obscura fetch https://example.com --dump markdown
obscura fetch https://example.com --dump links
obscura fetch https://example.com --dump assets
obscura fetch https://example.com --dump original
obscura fetch https://example.com --dump cookies

html

Rendered HTML after JavaScript runs. Default.

obscura fetch https://news.ycombinator.com --dump html > hn.html

text

Plain text. No markup.

obscura fetch 'https://en.wikipedia.org/wiki/Rust_(programming_language)' --dump text

markdown

Markdown conversion: headings, lists, links, code blocks, images.

obscura fetch https://docs.example.com/page --dump markdown > page.md

links

Links from the page, one per line. Each line contains the resolved URL and, when present, a tab followed by the link text.

obscura fetch https://example.com --dump links

assets

Every external resource (stylesheets, scripts, images, fonts, iframes), plus the URLs the page requested through fetch()/XHR, one JSON object per line.

obscura fetch https://example.com --dump assets

original

The raw HTTP response body, including binary content. This bypasses HTML parsing and JavaScript execution.

obscura fetch https://my-spa.example --dump original > before.html
obscura fetch https://my-spa.example --dump html     > after.html
diff before.html after.html

cookies

Every cookie in the jar as a JSON array, including HttpOnly cookies that document.cookie cannot see. Useful for capturing session tokens set by anti-bot challenges.

obscura fetch https://example.com --dump cookies

With --wait-until

--dump runs after the wait condition:

obscura fetch https://my-spa.example --wait-until load --dump markdown

Pipe and redirect

obscura fetch https://example.com --dump markdown > example.md
obscura fetch https://example.com --dump text --quiet | wc -w

Reader libraries

After loading a reader library such as Mozilla Readability, run it on a document clone to leave the page unchanged:

const article = new Readability(document.cloneNode(true)).parse();

Obscura does not bundle Readability. Supply the library through your automation client before evaluating this expression. Document clones preserve metadata and live form-control state. Attribute clones retain their value and namespace. DOMParser HTML documents preserve the document structure and root attributes, inherit the parser's document URL, and do not execute scripts. XML parsing still uses the existing limited XML implementation.