When Your Browser Agent Gets Stuck, Hand the Browser to a Person

The short answer: keep the agent and the person in the same browser. When the agent hits something only a human can do — a texted code, a press-and-hold challenge, a judgement call — send a person a live link to that exact session, wait until the page shows the step is done, and let the agent continue. Do not restart, and do not ask the person to do it in their own browser: the agent needs the result in its session, with its cookies and its IP.

This post is the pattern in working code, and the details that make it reliable.

The walls agents cannot climb

Every browser agent, however good its model, eventually meets a page built to stop it:

• Two-factor codes sent to a phone the agent does not have.
• Press-and-hold challenges (PerimeterX/HUMAN), which no CAPTCHA-solving service handles.
• "Is this you?" prompts after a sign-in from a new place.
• Native dialogs — a confirm() that blocks the tab until someone answers it, which looks to the agent exactly like a frozen page.
• Decisions — "Submit this order for $4,200?" — that you do not want an agent to make alone.

The common responses are to fail the run, or to retry until the account is locked. The better one is to make a human one tap away.

What "take over" has to mean

For a handoff to work, three things have to be true:

1. The person acts in the agent's browser, not a copy. Same cookies, same tabs, same exit IP. A code typed into a different browser signs in a different browser.
2. The person can be anywhere. The account owner with the phone that receives the code is usually not at a desk. A viewer that only works with a mouse turns a one-minute interruption into an afternoon.
3. The agent knows when to continue. "The human clicked a button in Slack" is not a reliable signal. "The dashboard is on screen" is.

The pattern in code

The examples use Scrappy's REST API from Python. The same calls exist as MCP tools (browser_*) if your agent speaks MCP.

import os, time, requests

API = "https://scrappy.gg/api/browser"
H = {"Authorization": f"Bearer {os.environ['SCRAPPY_API_KEY']}"}

def call(method, path, **kw):
    r = requests.request(method, f"{API}{path}", headers=H, timeout=90, **kw)
    r.raise_for_status()
    body = r.json()
    return body.get("data", body)

1. Detect the wall

Check what is actually on screen. Many sign-in pages load a CAPTCHA vendor's script whether or not a challenge is shown, so "the script is present" proves nothing; rendered is the field to act on.

def blocked_reason(sid):
    cap = call("POST", f"/sessions/{sid}/captcha", json={"action": "detect"})
    if cap.get("rendered"):
        return f"{cap.get('vendor') or cap.get('type')} challenge"
    url = call("POST", f"/sessions/{sid}/evaluate", json={"expression": "location.href"})["result"]
    if any(p in url for p in ("/verify", "/challenge", "/2fa", "/mfa")):
        return "verification step"
    return None

Your own agent will have better signals for its own sites — an element that only appears on the code-entry form, say. Use them.

2. Send a person a live link

Mint a viewer link for the session and send it wherever the person will see it: SMS, Slack, email, a push notification in your own app. The link is short-lived (about five minutes) and carries full control of the session, so mint it when you send it and only send it to the person who should have it.

def ask_a_human(sid, reason, notify):
    viewer = call("POST", f"/sessions/{sid}/viewer-token")
    notify(f"The browser needs you ({reason}). Open: {viewer['viewerUrl']}")

On a phone, the viewer is driven with a finger: one finger is the mouse (including press-and-hold for as long as the finger stays down), two fingers pan and zoom, and a keyboard button brings up the phone's keyboard for codes and passwords. The remote page still receives desktop mouse and keyboard events, so the browser's fingerprint does not change halfway through the session — which matters on exactly the pages that are checking it.

3. Wait for the page, not for a message

Wait until the page itself shows that the step is done. wait-for-function polls in the browser and returns as soon as the expression is true, up to a minute per call; loop it, and keep the session alive while you wait.

def wait_for_human(sid, done_expression, give_up_after=900):
    deadline = time.time() + give_up_after
    while time.time() < deadline:
        res = call("POST", f"/sessions/{sid}/wait-for-function",
                   json={"expression": done_expression, "timeout": 60000})
        if res.get("satisfied"):
            return True
        call("POST", f"/sessions/{sid}/keepalive")   # an idle session is reaped
    return False

done_expression is synchronous JavaScript that is truthy once the person has finished, for example !!document.querySelector('[data-testid="dashboard"]') or !location.pathname.startsWith('/login'). An expression that throws counts as not yet true, so it is safe to reference globals the next page will define.

4. Carry on

reason = blocked_reason(sid)
if reason:
    ask_a_human(sid, reason, notify=send_sms)
    if not wait_for_human(sid, "!location.pathname.startsWith('/login')"):
        raise RuntimeError("nobody cleared the wall in 15 minutes")
# the agent continues in the same session, now past the wall

If the wall was a sign-in, capture the session into a saved login afterwards (POST /api/browser/profiles/{id}/capture), so the person is not needed again next run. The login scraping walkthrough covers that half.

Approve instead of take over

Sometimes the person does not need to do anything, only to say yes. When Claude drives the session through its browser tool, Scrappy can hold chosen kinds of action for a person to approve before they happen:

{ "content": [...], "options": { "approval": { "members": ["input", "navigation"] }, "allowedHosts": ["portal.example.com"] } }

Calls run in order until the first gated one, which comes back as pending; your app shows it to a person and posts the turn again with resume: { from, approved: [tool_use_id] } (or denied). allowedHosts keeps the agent on the sites you named, whatever a page tells it to do.

Two traps

A frozen tab is often a dialog. A native alert/confirm opened by the page blocks every command on that tab, and a fresh connection to the browser cannot see it. Before calling a person for a "hung" page, ask: POST /sessions/{id}/dialog with {"action": "status"}. If one is open, dismiss or accept it — often that is the whole fix. Scrappy's failing actions also say so themselves, with an error.dialogOpen field. We wrote up how this fooled us once in The Dialog That Looks Like a Frozen Tab.

Do not leave the session idle while you wait. A session that nothing touches is reaped (30 minutes idle; 10 for an API-key session that has never been used), and a person who opens a link to an ended session sees nothing. keepalive resets the idle clock without touching the page.

What it costs

The session keeps billing per minute while it waits — 1 credit a minute on Scrappy — so put a ceiling on the wait (give_up_after above) that matches what the step is worth. Fifteen minutes for a person to clear a sign-in is 15 credits; a failed run that has to start over usually costs more.

Summary

• Is a wall really on screen? POST /sessions/{id}/captcha (rendered), or your own selector
• Reach a person: POST /sessions/{id}/viewer-token → send viewerUrl
• Wait for them: POST /sessions/{id}/wait-for-function in a loop, with keepalive
• Only need a yes: options.approval on /claude-tools
• Tab frozen? POST /sessions/{id}/dialog {"action": "status"} first

The cloud browser guide documents every call above, and Human-in-the-loop browser automation has the short version.