String docs
Fetch

Overview

Send a URL and get the page back as JSON, raw bytes or Markdown. The API picks rendering, proxy tier and captcha handling.

POST /fetch sends an HTTP or HTTPS request to a URL and returns the result. Send only the url: that is the recommended request, and the one to start with for every page.

From the url alone, the API decides everything a request needs:

  • JavaScript rendering. Pages that build their content with JavaScript come back rendered.
  • Proxy tier. Each request goes out on the standard or premium tier, whichever the destination needs.
  • Captchas. A captcha or interactive challenge on the page is attempted as part of the request. A solve can fail; see Captcha solving.
curl https://request.usestring.ai/v1/fetch \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com" }'

Add other fields only to change what comes back, such as format, or when a page needs something the url can't express. To always render the page in a browser, set executeJS; see JavaScript rendering.

Methods

Supports GET, POST, PUT, PATCH, DELETE, HEAD, and OPTIONS. Method names are case-insensitive, so get and Post work too; any other method returns a 400 that names it. A request body is allowed on every method except GET and HEAD — sending one with either returns a 400.

A string body is sent as given. An object body is sent as the exact JSON text you wrote, so key order, number formatting such as 1.50 or 1e3, and integers too large for a double reach the destination unchanged.

Google search URLs

Fetching https://www.google.com/search?q=... returns search results as structured JSON, not the results page's HTML. The request is answered by Search and the response is Search's — a results array.

The reroute maps these parts of the URL and request onto Search:

  • q is the query.
  • hl is the results language.
  • num is the number of results, and start the offset of the first one. start=20 returns results from the 21st on, so a deeper start fetches, and bills, more Search pages.
  • The country comes from gl when it's present and from the Google domain otherwise, so google.co.uk/search?q=... and google.com/search?q=...&gl=uk both return UK results. When the URL names no country, countryCode sets it.
  • solveCaptcha: false carries over. A search avoids solving captchas wherever it can and rarely solves one; with false, none is ever solved on your behalf. If the search can't be answered without one, it fails with a 502 (504 if it times out), and retrying may succeed. See Captcha solving.

Parameters a browser adds that don't change the results, such as oq, sourceid, ie, client, ei and ved, are ignored. Anything else is refused rather than dropped: a method other than GET, body, headers, a format other than json, jsonSchema, mainContentOnly, executeJS, requireWSS, screenshot, actions, a countryCode that disagrees with the URL's country, a gl, domain, hl, num or start Search can't use, and any other URL parameter, such as lr, safe or tbs. The request returns a 400 that lists each one:

{
  "error": "google_search_reroute_unsupported_input",
  "reason": "safe, tbs are not supported on a Google search sent to /fetch; use POST /search, which takes structured search options.",
  "unsupported": ["safe", "tbs"],
  "endpoint": "/search"
}

Send those searches to POST /search, which takes structured search options. In place of a refused tbs, use its dateRange and sortBy fields, and use its page to pick a results page by number. In place of a refused format, send format: "raw" to get the results pages as HTML.

A Google AI Mode URL (udm=50) is rerouted the same way and refuses num and start, since AI Mode returns one answer rather than ranked results.

Every rerouted response, a 400 included, carries the x-rerouted-to: /search header.

These requests bill at the Search rate rather than the Fetch rate. Every other Google URL — Maps, News, the image and shopping tabs, everything outside /search — fetches normally.

A request that sets maxRedirects or followRedirects: false is not answered by Search; the URL is fetched as a page.

What you can control

Every field below is optional, and none is needed for the API to render, pick a proxy tier or attempt a captcha.

CapabilityFieldPage
Response shapeformatResponse formats
Extract specific fieldsjsonSchemaStructured extraction
Force a browser renderexecuteJS, requireWSSJavaScript rendering
Choose when the page is readwaitUntilWaiting for page load
Read the page's iframesincludeIframes, captureIframesIframes
Return the page's API callscaptureXHRXHR capture
Forward request headersheadersCustom headers
Geolocate the requestcountryCodeProxies & geolocation
Captcha handlingsolveCaptchaCaptcha solving
Accept a bad TLS certificateignoreCertificateErrorsTLS verification
Stop at a redirectmaxRedirects, followRedirectsRedirects
Capture an image / drive the pagescreenshot, actionsScreenshots, Browser actions

For the complete parameter list and response schema, see the /fetch API reference.

How billing works

Usage is billed per request at the rate for the fetch type the API used, which the x-billed-request-type header reports. See Pricing for rates and for which failed requests are billed.

Access control

Some destinations require approval before use, and some request patterns require KYC. Both return 403 with a reason that says which one applies. See Access control & KYC.