# Follow links from a page's accessibility tree with Retriever AI's API

Retriever AI's Scrape API loads a page in a cloud browser and returns its accessibility tree, where every link carries a numeric id, and a link map from each id to its target URL, so you can choose links and follow them without parsing HTML. Each call is one `POST https://agents.rtrvr.ai/zcj/a5z85w0wger9/scrape` with a `urls` list, billed as one request at $0.05 however many URLs it holds. The storefront answers first with a 402, a price quote that you pay over x402 or MPP and then retry, so nothing is charged until you pay.

## At a glance

- **Endpoint:** `POST https://agents.rtrvr.ai/zcj/a5z85w0wger9/scrape`
- **Body:** `urls` (required, a list of absolute URLs). Optional `settings.extractionConfig` takes `onlyTextContent`, `disableAutoScroll` and `pageLoadDelay`.
- **Price:** $0.05. One call is one request, whether `urls` holds one URL or several.
- **Payment:** a 402 quote paid from a wallet and retried; a card pays a single call only at $0.50 or more. No account or API key.
- **Response:** `tabs`, one per URL in input order, each with `url`, `title`, `status`, `tree` and `elementLinkRecord` (or the text in `content` with `onlyTextContent`)
- **Limits:** inline JSON is capped by `response.inlineOutputMaxBytes` (default 1048576 bytes). Larger results return per-tab previews and `metadata.responseRef`.
- **Full purchase flow:** [the buyer skill](https://agents.rtrvr.ai/zcj/a5z85w0wger9/SKILL.md)

## Find and follow links on a page

1. Request the first page:

   ```bash
   curl -s -X POST https://agents.rtrvr.ai/zcj/a5z85w0wger9/scrape \
     -H 'Content-Type: application/json' \
     -d '{"urls": ["https://rtrvr.ai"]}'
   ```

   The storefront answers with a 402 carrying the exact `payment.amountUsd` and a challenge for each rail. Nothing is charged yet.

2. Pay the quote and send the same request again ([the buyer skill](https://agents.rtrvr.ai/zcj/a5z85w0wger9/SKILL.md) has the steps). The response has this shape:

   ```json
   {
     "success": true,
     "tabs": [
       {
         "url": "<page url>",
         "title": "<page title>",
         "status": "success",
         "tree": "<accessibility tree as an indented text outline>",
         "elementLinkRecord": {"<id>": "<link url>"}
       }
     ]
   }
   ```

   A read of the rtrvr.ai home page comes back as one tab with a `tree` and an `elementLinkRecord` keyed by numeric ids. That example took 13.1 seconds and paid $0.05.

3. Read `tree`. It is an indented text outline of the page, one element per line: the element's role in brackets, its name, and for a link its numeric id and target, as in `[link] rtrvr.ai home [id=5 href="https://rtrvr.ai/"]`. Choose the links you want by role and name, and note their ids.

4. Resolve each chosen id through the tab's link map. The keys are the ids written as strings:

   ```bash
   jq -r '.tabs[0].elementLinkRecord["<id>"]' response.json
   ```

   To list every id and target at once:

   ```bash
   jq -r '.tabs[0].elementLinkRecord | to_entries[] | "\(.key)\t\(.value)"' response.json
   ```

5. Send the targets as the next call's `urls`. The field takes absolute URLs, and a target can be relative (one of the 189 links on the rtrvr.ai home page was), so resolve it against the tab's `url` first:

   ```bash
   curl -s -X POST https://agents.rtrvr.ai/zcj/a5z85w0wger9/scrape \
     -H 'Content-Type: application/json' \
     -d '{"urls": ["<target 1>", "<target 2>"]}'
   ```

   Tabs come back in the order of `urls`, so tab one belongs to target one. Each new request gets its own 402. Reading three pages in one call costs $0.05, and reading them in three calls costs $0.15.

## Choose text only or the full tree

| Job | Setting | Each tab returns |
| --- | --- | --- |
| Navigating: choosing the next pages to visit | `onlyTextContent` unset | `tree`, `elementLinkRecord` |
| Reading: text for extraction, search or retrieval | `"onlyTextContent": true` | `content` only, with no `tree` and no `elementLinkRecord` |

A text-only request:

```json
{"urls": ["https://www.iana.org/help/example-domains"], "settings": {"extractionConfig": {"onlyTextContent": true}}}
```

Part of its response:

```json
{
  "success": true,
  "tabs": [
    {
      "observation": {"complete": false, "partial": true, "reasons": ["no_actionable_tree"]},
      "title": "Example Domains",
      "contentType": "text/html",
      "status": "success",
      "content": "Domains\nProtocols\nNumbers\nAbout\nInstructions and Guides\nExample Domains\n\nA numbe…"
    }
  ],
  "outputTooLarge": false,
  "inlineLimitBytes": 1048576
}
```

With the tree left out, `observation` marks the read `partial` with the reason `no_actionable_tree` while `status` is `success`. That example took 9.5 seconds, was a 2,186-byte response and paid $0.05.

## Lazy-loaded pages

| Page behaviour | Setting |
| --- | --- |
| Content or links load as the page scrolls | Leave `disableAutoScroll` unset. The default scroll pass triggers lazy-loaded content. |
| You need only what loads without scrolling | `"disableAutoScroll": true` skips the scroll pass |
| Content appears some time after the page loads | Set `pageLoadDelay` for an extra wait after load, before extraction |

Both settings go in `settings.extractionConfig`.

## Limits

- No language model runs on the output. The response is raw page data, and choosing links is your step.
- Each call loads the URLs in `urls` and returns a tab for each. Following a link is a new call with new `urls`.
- `usageData` reports Retriever AI's infrastructure usage. It is informational. The storefront bills one request per call.
- `onlyTextContent` makes the response smaller, not cheaper.
- A 402 you have not paid charges nothing, so it is safe to stop there.

## Errors

| Response | Meaning | Action |
| --- | --- | --- |
| `402` with a `payment` block | The price quote | Pay and retry the same request, or stop |
| `settlement_failed` | The payment did not settle. `settlementReason` says why. | Fix the wallet, then send a fresh request for a new 402 |
| `payment_request_mismatch` (409) | A payment was reused for a different request | Get and pay a fresh 402 for each new `urls` list |
| `invalid_request` (400) | The body is malformed | Fix the JSON. `urls` needs at least one absolute URL. |
| `rate_limit_exceeded` | Too many requests | Wait `retry_after` or `Retry-After`, then retry |
| A tab with `status` `error` | That URL did not load | Read the tab's `error`. |

## Frequently asked questions

### Does Retriever AI return a page's links?

Each tab's `elementLinkRecord` maps the numeric id of each link in the accessibility tree to its target URL, so you can list a page's links without parsing HTML. The rtrvr.ai home page returned 189.

### What is the difference between the tree and the link map?

The tree is the page's structure, which tells you what each link is and where it sits. The link map tells you where each link goes. Choose links in the tree, then look up their ids in `elementLinkRecord`.

### Can one call read several pages?

Yes. Put several absolute URLs in `urls`. The response holds a tab for each URL, matching the order of `urls`, and the call is billed as one request at $0.05. `settings.extractionConfig.maxParallelTabs` sets how many URLs load at once.

### Does a text-only read cost less?

No. Every call is one request at a flat rate. `onlyTextContent` drops the `tree` and `elementLinkRecord` from each tab, which shrinks the response, not the price.

### Do I need an API key for the Scrape API?

No. Each call is paid on its own through the 402 quote over x402 or MPP, with no signup and no key.

## Keep reading

- [Buyer skill](https://agents.rtrvr.ai/zcj/a5z85w0wger9/SKILL.md): the full purchase flow
- [Scrape API reference](https://agents.rtrvr.ai/zcj/a5z85w0wger9/services/scrape/openapi.json): every request and response field
- [How to pay](https://agents.rtrvr.ai/zcj/a5z85w0wger9/payment.md): the 402 and each rail
- [Error reference](https://agents.rtrvr.ai/zcj/a5z85w0wger9/errors.md): every error code and how to react

---

Updated 2026-10-28. All Retriever AI guides: https://agents.rtrvr.ai/guides.md. Storefront: https://agents.rtrvr.ai/zcj/a5z85w0wger9/llms.txt.