String docs
Site integrations

Wayback Machine

The Internet Archive's Wayback Machine: every snapshot it holds of a URL, host or path prefix, with capture time, status and type, and an archived page as it was captured, with its title, text and links. Useful for pages that have since changed or gone.

Call these actions with your String API key; see Authentication. Every call answers with the envelope described in Responses & errors, whose data holds the rows below.

  • Site: wayback
  • Version: 1.0.0
  • Reads from: archive.org
ActionDescriptionListed price
snapshotsList the Wayback Machine's snapshots of URLs: capture time, the URL as captured, the original server's status, content type and content digest, with links to each capture and its original bytes. Filter by time window and status, fold near-duplicates with collapse, list a whole host or path prefix with matchType, and take the oldest or newest first. The page action reads a capture's text and links.Billed per request at your plan's rates
pageRead pages from the Wayback Machine as they were captured: the capture closest to a given time (or the latest), with its title, description, visible text and every link on it, marked internal or outbound. Reads the capture's original bytes, without the archive's toolbar or rewritten links. The snapshots action lists the captures a URL has.Billed per request at your plan's rates

snapshots

List the Wayback Machine's snapshots of URLs: capture time, the URL as captured, the original server's status, content type and content digest, with links to each capture and its original bytes. Filter by time window and status, fold near-duplicates with collapse, list a whole host or path prefix with matchType, and take the oldest or newest first. The page action reads a capture's text and links.

POST https://request.usestring.ai/v1/integrations/wayback/snapshots

Input

Unknown fields are rejected.

FieldTypeDescription
itemsstring[]Required. The URLs to list snapshots of: each an http(s) URL, a bare host such as paulgraham.com, or a web.archive.org snapshot URL. 1 to 300 entries.
fromstringOnly snapshots captured at or after this time: a Wayback timestamp of 4 to 14 digits, yyyyMMddhhmmss, such as 2010 or 20100315. Matches ^\d{4,14}$.
tostringOnly snapshots captured at or before this time, as from. Matches ^\d{4,14}$.
limitintegerThe most snapshots to return per URL, 1 to 25000 (oldest first) or 1 to 10000 (newest first). Defaults to 100. A listing longer than 5000 reads the index in pages. From 1 to 25000.
newestbooleanReturn the most recent snapshots (newest first) instead of the oldest (oldest first)
matchTypeexact prefix host domainexact (the default) lists that URL; prefix every URL under it; host every URL on its host; domain every URL on its host and subdomains. prefix, host and domain can take 30 to 60 seconds.
collapsestringFold runs of adjacent snapshots sharing a field into one: timestamp:N (the first N digits, such as timestamp:8 for one per day or timestamp:6 per month), digest (unchanged content), urlkey (one per distinct URL, for prefix listings), statuscode or mimetype. Matches `^(timestamp(:([1-9]\
statusCodeinteger | nullOnly snapshots the original server answered with this HTTP status, such as 200. Shorthand for the filter statuscode:<n>, and combines with filters.
filtersstring[] | nullOnly snapshots matching every one of these, each field:regex (or !field:regex to exclude), field one of statuscode, mimetype, original, urlkey, digest or length, such as ["statuscode:200", "mimetype:text/html"] or ["!statuscode:[45].."]. At most 10. At most 10 entries.

Output

data is an array of rows, one per result. Each row has:

FieldTypeDescription
urlstringThe URL looked up, as the caller gave it, normalized.
timestampstringThe capture time as the archive keys it, yyyyMMddhhmmss in UTC.
capturedAtstring | nullThe capture time, RFC 3339 in UTC.
originalstringThe URL as it was captured.
statusCodeinteger | nullThe status the original server answered with; null for a revisit record, which repeats an earlier capture's content.
mimeTypestring | nullThe captured content type, such as text/html; warc/revisit for a revisit record.
digeststring | nullThe SHA-1 (base 32) of the captured body; equal digests mean unchanged content.
lengthinteger | nullThe stored record's size in bytes, compressed.
snapshotUrlstringThe capture on web.archive.org, as a person would open it.
rawUrlstringThe capture's original bytes, without the archive's toolbar or link rewriting; the page action reads it.

Example

curl https://request.usestring.ai/v1/integrations/wayback/snapshots \
  -H "Authorization: Bearer $STRING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "items": ["paulgraham.com"],
    "limit": 5,
    "collapse": "timestamp:4"
  }'

data starts with these rows:

[
  {
    "url": "http://paulgraham.com",
    "timestamp": "20000918084506",
    "capturedAt": "2000-09-18T08:45:06Z",
    "original": "http://www.paulgraham.com:80/",
    "statusCode": 200,
    "mimeType": "text/html",
    "digest": "JXOUBG323XDXHDX35JGL5RJWWL37LFTV",
    "length": 759,
    "snapshotUrl": "https://web.archive.org/web/20000918084506/http://www.paulgraham.com:80/",
    "rawUrl": "https://web.archive.org/web/20000918084506id_/http://www.paulgraham.com:80/"
  },
  {
    "url": "http://paulgraham.com",
    "timestamp": "20010301213038",
    "capturedAt": "2001-03-01T21:30:38Z",
    "original": "http://www.paulgraham.com:80/",
    "statusCode": 200,
    "mimeType": "text/html",
    "digest": "X5SAZ3OWRX3HTUWQ5JGH2TAWX52RMXC3",
    "length": 987,
    "snapshotUrl": "https://web.archive.org/web/20010301213038/http://www.paulgraham.com:80/",
    "rawUrl": "https://web.archive.org/web/20010301213038id_/http://www.paulgraham.com:80/"
  },
  {
    "url": "http://paulgraham.com",
    "timestamp": "20020122023619",
    "capturedAt": "2002-01-22T02:36:19Z",
    "original": "http://paulgraham.com:80/",
    "statusCode": 200,
    "mimeType": "text/html",
    "digest": "6FYT7YCWL35OSVPE3XP3F2JLOWMV63VI",
    "length": 1117,
    "snapshotUrl": "https://web.archive.org/web/20020122023619/http://paulgraham.com:80/",
    "rawUrl": "https://web.archive.org/web/20020122023619id_/http://paulgraham.com:80/"
  },
  {
    "url": "http://paulgraham.com",
    "timestamp": "20030125034420",
    "capturedAt": "2003-01-25T03:44:20Z",
    "original": "http://www.paulgraham.com:80/",
    "statusCode": 200,
    "mimeType": "text/html",
    "digest": "JRETAHOLZX5RHSQU6OZS7WDBJRUB2LPR",
    "length": 1245,
    "snapshotUrl": "https://web.archive.org/web/20030125034420/http://www.paulgraham.com:80/",
    "rawUrl": "https://web.archive.org/web/20030125034420id_/http://www.paulgraham.com:80/"
  },
  {
    "url": "http://paulgraham.com",
    "timestamp": "20040102223155",
    "capturedAt": "2004-01-02T22:31:55Z",
    "original": "http://www.paulgraham.com:80/",
    "statusCode": 200,
    "mimeType": "text/html",
    "digest": "AHQ25LJZL2MN76JAYPIQKCYDEASE46YU",
    "length": 1358,
    "snapshotUrl": "https://web.archive.org/web/20040102223155/http://www.paulgraham.com:80/",
    "rawUrl": "https://web.archive.org/web/20040102223155id_/http://www.paulgraham.com:80/"
  }
]

Requests and billing

  • Requests: 1–15 Web Access requests per entry in items.
  • Billing: passthrough. Each request the action makes bills at its own rate on your plan; there is no separate charge for the action. See Pricing.

page

Read pages from the Wayback Machine as they were captured: the capture closest to a given time (or the latest), with its title, description, visible text and every link on it, marked internal or outbound. Reads the capture's original bytes, without the archive's toolbar or rewritten links. The snapshots action lists the captures a URL has.

POST https://request.usestring.ai/v1/integrations/wayback/page

Input

Unknown fields are rejected.

FieldTypeDescription
itemsstring[]Required. The pages to read: each an http(s) URL, a bare host such as paulgraham.com, or a web.archive.org snapshot URL. 1 to 300 entries.
timestampstringRead the capture closest to this time: a Wayback timestamp of 4 to 14 digits, yyyyMMddhhmmss, such as 2005 or 20050601. Defaults to the most recent capture. A web.archive.org snapshot URL without this reads the capture closest to the URL's own timestamp; this, when set, wins. Matches ^\d{4,14}$.

Output

data is an array of rows, one per result. Each row has:

FieldTypeDescription
urlstringThe URL looked up, as the caller gave it, normalized.
requestedTimestampstring | nullThe timestamp asked for, from timestamp or else the snapshot URL given; null for the most recent capture.
timestampstringThe capture actually read, yyyyMMddhhmmss in UTC.
capturedAtstring | nullThe capture time, RFC 3339 in UTC.
originalstringThe URL as it was captured.
snapshotUrlstringThe capture on web.archive.org, as a person would open it.
statusCodeinteger | nullThe status the original server answered with when captured.
contentTypestring | nullThe captured content type.
titlestring | nullThe page's title.
descriptionstring | nullThe page's meta or og: description.
textstringThe page's visible text, scripts and styles removed, up to 100000 characters; empty for a capture that is not HTML.
linksobject[]Every distinct http(s) and mailto link on the page, in page order, from anchors, image maps and rel=me or alternate link elements.
links[].urlstringThe link's target, resolved against the page's URL as captured.
links[].textstring | nullThe link's visible text, else its title, else its image's alt text.
links[].relstring | nullThe link's rel attribute, such as me or alternate.
links[].internalbooleanWhether the target is on the page's own host (ignoring www.)

Example

curl https://request.usestring.ai/v1/integrations/wayback/page \
  -H "Authorization: Bearer $STRING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "items": ["paulgraham.com"],
    "timestamp": "2005"
  }'

data starts with these rows:

[
  {
    "url": "http://paulgraham.com",
    "requestedTimestamp": "2005",
    "timestamp": "20051231231850",
    "capturedAt": "2005-12-31T23:18:50Z",
    "original": "http://www.paulgraham.com:80/",
    "snapshotUrl": "https://web.archive.org/web/20051231231850/http://www.paulgraham.com:80/",
    "statusCode": 200,
    "contentType": "text/html",
    "title": "Paul Graham",
    "description": null,
    "text": "In\nstores\nnow:\nHackers & Painters.\n\nNew:\nGood and Bad Procrastination |\nHow to Make Wealth.\n\n© mmv pg",
    "links": [
      {
        "url": "http://www.paulgraham.com/articles.html",
        "text": null,
        "rel": null,
        "internal": true
      },
      {
        "url": "http://www.paulgraham.com/books.html",
        "text": null,
        "rel": null,
        "internal": true
      },
      {
        "url": "http://ycombinator.com",
        "text": null,
        "rel": null,
        "internal": false
      },
      {
        "url": "http://startupschool.org",
        "text": null,
        "rel": null,
        "internal": false
      },
      {
        "url": "http://www.paulgraham.com/lisp.html",
        "text": null,
        "rel": null,
        "internal": true
      },
      {
        "url": "http://www.paulgraham.com/arc.html",
        "text": null,
        "rel": null,
        "internal": true
      },
      {
        "url": "http://www.paulgraham.com/antispam.html",
        "text": null,
        "rel": null,
        "internal": true
      },
      {
        "url": "http://www.paulgraham.com/faq.html",
        "text": null,
        "rel": null,
        "internal": true
      },
      {
        "url": "http://www.paulgraham.com/raq.html",
        "text": null,
        "rel": null,
        "internal": true
      },
      {
        "url": "http://www.paulgraham.com/quo.html",
        "text": null,
        "rel": null,
        "internal": true
      }
    ]
  }
]

Requests and billing

  • Requests: 2–5 Web Access requests per entry in items.
  • Billing: passthrough. Each request the action makes bills at its own rate on your plan; there is no separate charge for the action. See Pricing.