Scrape URL

Turn a public webpage into clean, structured JSON for downstream applications.

POST/search/v2/scrape-url

Related NOSIBLE resources

research & data

Inspect the Search data dictionary or read NOSIBLE research for implementation context and methods.

Authorizations

apiKey

Api-Keystringrequiredheader

Your NOSIBLE API key.

Example: nos_sk_...

Body

application/json

urlstringbody

The public URL to retrieve and process.

Example: https://example.com/article

htmlstringbody

Optional HTML supplied by the caller. When present, NOSIBLE can process it without fetching the URL body again.

Example: <html>...</html>

renderbooleanbody

Force JavaScript rendering for pages that need a browser to produce their content.

Example: false

recrawlbooleanbody

Request a fresh crawl instead of reusing a cached document.

Example: false

How to use this endpoint

guidance

Use Scrape URL when you need a clean document object rather than search ranking. Pass a public URL; use html when you already have page content and recrawl when you need to bypass a cached copy. The response is the same structured document family used by Search results, so treat fields such as metadata and content as optional when the source does not provide them.

Response body

application/json

These are the fields you can build against. Nested names use dot notation; optional sections are called out in their descriptions.

messagestring

Human-readable retrieval acknowledgement.

Example: Document retrieved

added_to_batchboolean

Whether the document was added to a downstream batch; normally false for this endpoint.

Example: false

responseobject

Structured document record. Metadata availability varies by source; do not assume every optional field is populated.

Example: { "url": "https://example.com/article", "title": "Article title", "netloc": "example.com", "content": "..." }

response.urlstring

Canonical document URL when available.

Example: https://example.com/article

response.titlestring

Extracted document title when available.

Example: Article title

response.netlocstring

Source hostname.

Example: example.com

response.publisheddate|null

Source publication date when detected.

Example: 2026-07-20

response.visiteddatetime|null

Timestamp at which NOSIBLE retrieved or crawled the document.

Example: 2026-07-21T08:14:22Z

response.authorstring|null

Detected author when available.

Example: Jane Doe

response.descriptionstring|null

Source or generated description when available.

Example: A concise article summary.

response.best_chunkstring|null

Most useful extracted passage for retrieval workflows.

Example: The article begins...

response.contentstring

Cleaned page or article text. Preserve it as UTF-8 text; it may be large.

Example: Clean extracted text...

Responses and errors

HTTP

200Request succeeded. The response body is shown in the panel on the right.

400invalid_request — The URL or scrape options are invalid.

401unauthorized — The API key is missing or invalid.

429rate_limited — The scrape quota has been exhausted; retry after the server's backoff window.

502upstream_fetch_failed — The source could not be fetched or parsed.

504upstream_timeout — The source did not respond within the retrieval window.

400scrape_failed — The URL could not be fetched or parsed. Runtime scrape failures are returned as a 400 response with the upstream reason.