Question
How do I scrape a website that requires a login?
Have a person sign in once in a real browser, save that signed-in session, and replay it on every later run from the same residential IP. Scripting the sign-in form on each run is what gets accounts challenged: a fresh login from a new address every time is the clearest automation signal a site sees.
Last updated
Sign in once, not every run
Logging in is the step sites watch most closely — rate limits, risk scores, two-factor prompts. Do it once, with a person at the keyboard if the site asks for anything unusual, then reuse the session it produced.
Keep the session where the site expects it
Save cookies and local storage together, restore them before the first page loads, and send every run through the same exit IP with a timezone that matches it. A session cookie that arrives from a different address on each run reads as a stolen session.
Notice when the login ends
Sites expire sessions on their own schedule. Check for a sign-in page or a password field before trusting a run's results, so an expired login does not quietly return empty pages.
With Scrappy
A person signs in through the live viewer and you capture the session into a saved login. Later sessions start already signed in, leave through the residential IP the login was captured on, save rotated cookies back when they end, and fire a BROWSER_LOGIN_LOST webhook if the site signs them out.
Questions
- Is it allowed to scrape behind a login?
- That depends on the site's terms and on whose account it is. Use accounts you or your users control, get the user's consent before saving their login, and delete the saved login when they disconnect.
- Can I try it before paying?
- Yes. A new account gets 100 credits, enough for a few sessions of real use. A session costs 5 credits to start and 1 per minute after that.
Start
Try it on the site you need this week. The first 100 credits are free.
Enough for ten scrapes or a first hour in the cloud browser. No card, no call, no minimum.