Wayback Machine
The Internet Archive's Wayback Machine: every snapshot it holds of a URL, host or path prefix, with capture time, status and type, and an archived page as it was captured, with its title, text and links. Useful for pages that have since changed or gone.
Call these actions with your String API key; see Authentication. Every call answers with the envelope described in Responses & errors, whose data holds the rows below.
- Site:
wayback - Version:
1.0.0 - Reads from:
archive.org
| Action | Description | Listed price |
|---|---|---|
snapshots | List the Wayback Machine's snapshots of URLs: capture time, the URL as captured, the original server's status, content type and content digest, with links to each capture and its original bytes. Filter by time window and status, fold near-duplicates with collapse, list a whole host or path prefix with matchType, and take the oldest or newest first. The page action reads a capture's text and links. | Billed per request at your plan's rates |
page | Read pages from the Wayback Machine as they were captured: the capture closest to a given time (or the latest), with its title, description, visible text and every link on it, marked internal or outbound. Reads the capture's original bytes, without the archive's toolbar or rewritten links. The snapshots action lists the captures a URL has. | Billed per request at your plan's rates |
snapshots
List the Wayback Machine's snapshots of URLs: capture time, the URL as captured, the original server's status, content type and content digest, with links to each capture and its original bytes. Filter by time window and status, fold near-duplicates with collapse, list a whole host or path prefix with matchType, and take the oldest or newest first. The page action reads a capture's text and links.
POST https://request.usestring.ai/v1/integrations/wayback/snapshotsInput
Unknown fields are rejected.
| Field | Type | Description |
|---|---|---|
items | string[] | Required. The URLs to list snapshots of: each an http(s) URL, a bare host such as paulgraham.com, or a web.archive.org snapshot URL. 1 to 300 entries. |
from | string | Only snapshots captured at or after this time: a Wayback timestamp of 4 to 14 digits, yyyyMMddhhmmss, such as 2010 or 20100315. Matches ^\d{4,14}$. |
to | string | Only snapshots captured at or before this time, as from. Matches ^\d{4,14}$. |
limit | integer | The most snapshots to return per URL, 1 to 25000 (oldest first) or 1 to 10000 (newest first). Defaults to 100. A listing longer than 5000 reads the index in pages. From 1 to 25000. |
newest | boolean | Return the most recent snapshots (newest first) instead of the oldest (oldest first) |
matchType | exact prefix host domain | exact (the default) lists that URL; prefix every URL under it; host every URL on its host; domain every URL on its host and subdomains. prefix, host and domain can take 30 to 60 seconds. |
collapse | string | Fold runs of adjacent snapshots sharing a field into one: timestamp:N (the first N digits, such as timestamp:8 for one per day or timestamp:6 per month), digest (unchanged content), urlkey (one per distinct URL, for prefix listings), statuscode or mimetype. Matches `^(timestamp(:([1-9]\ |
statusCode | integer | null | Only snapshots the original server answered with this HTTP status, such as 200. Shorthand for the filter statuscode:<n>, and combines with filters. |
filters | string[] | null | Only snapshots matching every one of these, each field:regex (or !field:regex to exclude), field one of statuscode, mimetype, original, urlkey, digest or length, such as ["statuscode:200", "mimetype:text/html"] or ["!statuscode:[45].."]. At most 10. At most 10 entries. |
Output
data is an array of rows, one per result. Each row has:
| Field | Type | Description |
|---|---|---|
url | string | The URL looked up, as the caller gave it, normalized. |
timestamp | string | The capture time as the archive keys it, yyyyMMddhhmmss in UTC. |
capturedAt | string | null | The capture time, RFC 3339 in UTC. |
original | string | The URL as it was captured. |
statusCode | integer | null | The status the original server answered with; null for a revisit record, which repeats an earlier capture's content. |
mimeType | string | null | The captured content type, such as text/html; warc/revisit for a revisit record. |
digest | string | null | The SHA-1 (base 32) of the captured body; equal digests mean unchanged content. |
length | integer | null | The stored record's size in bytes, compressed. |
snapshotUrl | string | The capture on web.archive.org, as a person would open it. |
rawUrl | string | The capture's original bytes, without the archive's toolbar or link rewriting; the page action reads it. |
Example
curl https://request.usestring.ai/v1/integrations/wayback/snapshots \
-H "Authorization: Bearer $STRING_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"items": ["paulgraham.com"],
"limit": 5,
"collapse": "timestamp:4"
}'data starts with these rows:
[
{
"url": "http://paulgraham.com",
"timestamp": "20000918084506",
"capturedAt": "2000-09-18T08:45:06Z",
"original": "http://www.paulgraham.com:80/",
"statusCode": 200,
"mimeType": "text/html",
"digest": "JXOUBG323XDXHDX35JGL5RJWWL37LFTV",
"length": 759,
"snapshotUrl": "https://web.archive.org/web/20000918084506/http://www.paulgraham.com:80/",
"rawUrl": "https://web.archive.org/web/20000918084506id_/http://www.paulgraham.com:80/"
},
{
"url": "http://paulgraham.com",
"timestamp": "20010301213038",
"capturedAt": "2001-03-01T21:30:38Z",
"original": "http://www.paulgraham.com:80/",
"statusCode": 200,
"mimeType": "text/html",
"digest": "X5SAZ3OWRX3HTUWQ5JGH2TAWX52RMXC3",
"length": 987,
"snapshotUrl": "https://web.archive.org/web/20010301213038/http://www.paulgraham.com:80/",
"rawUrl": "https://web.archive.org/web/20010301213038id_/http://www.paulgraham.com:80/"
},
{
"url": "http://paulgraham.com",
"timestamp": "20020122023619",
"capturedAt": "2002-01-22T02:36:19Z",
"original": "http://paulgraham.com:80/",
"statusCode": 200,
"mimeType": "text/html",
"digest": "6FYT7YCWL35OSVPE3XP3F2JLOWMV63VI",
"length": 1117,
"snapshotUrl": "https://web.archive.org/web/20020122023619/http://paulgraham.com:80/",
"rawUrl": "https://web.archive.org/web/20020122023619id_/http://paulgraham.com:80/"
},
{
"url": "http://paulgraham.com",
"timestamp": "20030125034420",
"capturedAt": "2003-01-25T03:44:20Z",
"original": "http://www.paulgraham.com:80/",
"statusCode": 200,
"mimeType": "text/html",
"digest": "JRETAHOLZX5RHSQU6OZS7WDBJRUB2LPR",
"length": 1245,
"snapshotUrl": "https://web.archive.org/web/20030125034420/http://www.paulgraham.com:80/",
"rawUrl": "https://web.archive.org/web/20030125034420id_/http://www.paulgraham.com:80/"
},
{
"url": "http://paulgraham.com",
"timestamp": "20040102223155",
"capturedAt": "2004-01-02T22:31:55Z",
"original": "http://www.paulgraham.com:80/",
"statusCode": 200,
"mimeType": "text/html",
"digest": "AHQ25LJZL2MN76JAYPIQKCYDEASE46YU",
"length": 1358,
"snapshotUrl": "https://web.archive.org/web/20040102223155/http://www.paulgraham.com:80/",
"rawUrl": "https://web.archive.org/web/20040102223155id_/http://www.paulgraham.com:80/"
}
]Requests and billing
- Requests: 1–15 Web Access requests per entry in
items. - Billing:
passthrough. Each request the action makes bills at its own rate on your plan; there is no separate charge for the action. See Pricing.
page
Read pages from the Wayback Machine as they were captured: the capture closest to a given time (or the latest), with its title, description, visible text and every link on it, marked internal or outbound. Reads the capture's original bytes, without the archive's toolbar or rewritten links. The snapshots action lists the captures a URL has.
POST https://request.usestring.ai/v1/integrations/wayback/pageInput
Unknown fields are rejected.
| Field | Type | Description |
|---|---|---|
items | string[] | Required. The pages to read: each an http(s) URL, a bare host such as paulgraham.com, or a web.archive.org snapshot URL. 1 to 300 entries. |
timestamp | string | Read the capture closest to this time: a Wayback timestamp of 4 to 14 digits, yyyyMMddhhmmss, such as 2005 or 20050601. Defaults to the most recent capture. A web.archive.org snapshot URL without this reads the capture closest to the URL's own timestamp; this, when set, wins. Matches ^\d{4,14}$. |
Output
data is an array of rows, one per result. Each row has:
| Field | Type | Description |
|---|---|---|
url | string | The URL looked up, as the caller gave it, normalized. |
requestedTimestamp | string | null | The timestamp asked for, from timestamp or else the snapshot URL given; null for the most recent capture. |
timestamp | string | The capture actually read, yyyyMMddhhmmss in UTC. |
capturedAt | string | null | The capture time, RFC 3339 in UTC. |
original | string | The URL as it was captured. |
snapshotUrl | string | The capture on web.archive.org, as a person would open it. |
statusCode | integer | null | The status the original server answered with when captured. |
contentType | string | null | The captured content type. |
title | string | null | The page's title. |
description | string | null | The page's meta or og: description. |
text | string | The page's visible text, scripts and styles removed, up to 100000 characters; empty for a capture that is not HTML. |
links | object[] | Every distinct http(s) and mailto link on the page, in page order, from anchors, image maps and rel=me or alternate link elements. |
links[].url | string | The link's target, resolved against the page's URL as captured. |
links[].text | string | null | The link's visible text, else its title, else its image's alt text. |
links[].rel | string | null | The link's rel attribute, such as me or alternate. |
links[].internal | boolean | Whether the target is on the page's own host (ignoring www.) |
Example
curl https://request.usestring.ai/v1/integrations/wayback/page \
-H "Authorization: Bearer $STRING_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"items": ["paulgraham.com"],
"timestamp": "2005"
}'data starts with these rows:
[
{
"url": "http://paulgraham.com",
"requestedTimestamp": "2005",
"timestamp": "20051231231850",
"capturedAt": "2005-12-31T23:18:50Z",
"original": "http://www.paulgraham.com:80/",
"snapshotUrl": "https://web.archive.org/web/20051231231850/http://www.paulgraham.com:80/",
"statusCode": 200,
"contentType": "text/html",
"title": "Paul Graham",
"description": null,
"text": "In\nstores\nnow:\nHackers & Painters.\n\nNew:\nGood and Bad Procrastination |\nHow to Make Wealth.\n\n© mmv pg",
"links": [
{
"url": "http://www.paulgraham.com/articles.html",
"text": null,
"rel": null,
"internal": true
},
{
"url": "http://www.paulgraham.com/books.html",
"text": null,
"rel": null,
"internal": true
},
{
"url": "http://ycombinator.com",
"text": null,
"rel": null,
"internal": false
},
{
"url": "http://startupschool.org",
"text": null,
"rel": null,
"internal": false
},
{
"url": "http://www.paulgraham.com/lisp.html",
"text": null,
"rel": null,
"internal": true
},
{
"url": "http://www.paulgraham.com/arc.html",
"text": null,
"rel": null,
"internal": true
},
{
"url": "http://www.paulgraham.com/antispam.html",
"text": null,
"rel": null,
"internal": true
},
{
"url": "http://www.paulgraham.com/faq.html",
"text": null,
"rel": null,
"internal": true
},
{
"url": "http://www.paulgraham.com/raq.html",
"text": null,
"rel": null,
"internal": true
},
{
"url": "http://www.paulgraham.com/quo.html",
"text": null,
"rel": null,
"internal": true
}
]
}
]Requests and billing
- Requests: 2–5 Web Access requests per entry in
items. - Billing:
passthrough. Each request the action makes bills at its own rate on your plan; there is no separate charge for the action. See Pricing.
Substack
Substack writers and publications: a writer's public profile with bio, typed outbound links (X and others), the publications they write and the newsletters they publicly subscribe to; a publication's newest posts; and a writer's public activity feed of posts, restacks and notes.
Discogs
Discogs, the music database and record marketplace: a member's public profile (location, home page, profile text and the links in it, join date, collection and wantlist sizes) and the releases in their collection and wantlist, newest first, when the member made them public.