# FAQ

**Is this a replacement for Playwright?**
No. Playwright is the engine inside CrawlGenie. CrawlGenie adds what a team needs to run an *agent* on it unattended: a token-cheap observation format, policy and approvals the model can't bypass, profiles, recording with rewind, routines, live takeover, and APIs for every model and framework. See [COMPARISON.md](/docs/comparison).

**How much does it really save in tokens?**
Measured on six live sites: the default snapshot is ~16× smaller than raw HTML and ~4× smaller than a full accessibility tree; delta mode makes re-checks cost tens of tokens; routines cost zero. Your own number is on every session (`usage.tokenSavingsPct`) and on the Overview chart. The multiplier depends on the site and on how often your agent re-observes.

**Does it bypass Cloudflare / CAPTCHAs / bot detection?**
No, and we won't claim to. CrawlGenie normalises the user agent and removes automation flags, and lets you plug in your own proxy pools (datacenter or residential providers via username templates). Sites with commercial bot defence may still challenge headless Chromium from a datacenter IP; that is an IP-reputation and challenge-solving problem for every vendor.

**Which models/frameworks work?**
Any. MCP for Claude Code, Cursor, Codex and other MCP clients; `/v1/tools?format=openai|anthropic|gemini` for function calling in those SDKs and in LangChain/CrewAI/AutoGen; REST + TypeScript/Python SDKs for custom loops; raw CDP for existing Playwright/Puppeteer scripts.

**Where does data live? Do I need a database?**
No external database. Metadata is in SQLite (`/data/vessel.db`), recordings and profiles are files under `/data`. Back up that volume. Postgres/S3 only matter when you run several API nodes (see DEPLOY → Scaling out).

**How many sessions per server?**
Shared tier ≈ 60–130 MB per session; 4 vCPU/8 GB handles 25–35 agent-paced sessions, fewer on JS-heavy sites. `VESSEL_MAX_SESSIONS` caps it; the Overview shows warm slots and capacity.

**What happens when an agent tries something dangerous?**
With `approvals.mode: "destructive"`, clicks/Enter on targets that look like pay/send/delete/confirm (or your `patterns`) block until a human approves on the dashboard, via API, or via your webhook flow. The agent receives `approval_denied` if refused or the timeout passes. Domain rules and the SSRF guard apply to every request regardless.

**Can two agents share one login?**
Only if you ask for it (`profileLock: "shared"`). By default a profile is exclusive while a session holds it, so sessions don't invalidate each other's cookies.

**Why did a routine fail with "target not found"?**
The page no longer has an element with that role and name (a redesign, a different language, or you're on the wrong page). Open the run's session (Routines → Recent runs → Replay) to see what the agent saw, then edit the routine (reorder/remove/insert `assert` checkpoints) or re-record.

**How do I upgrade?**
`docker compose pull && docker compose up -d` (or `npm update vessel-browser`). SQLite migrations are additive and automatic; the data volume carries over.

**How do I get help / report a security issue?**
Open an issue with your version, tier, policy JSON and the session's events export; for security, see [SECURITY.md](/docs/security).

**Routine or extractor: which one?**
An extractor is a ready-made reader for a kind of site (jobs, reviews, prices, filings); it returns rows and needs no setup. A routine is a recording of *your own* clicks on a site nobody wrote an extractor for (a supplier portal, an internal tool). Use an extractor when one exists; record a routine when the task is specific to you. Both run on a schedule and cost zero model tokens.

**What does "needs proxy" mean on an extractor?**
That site blocks requests from datacenter IP addresses (Yelp, eBay, G2, LinkedIn jobs). CrawlGenie cannot change that; run the extractor with a residential proxy (`session.proxy`) or pick an API-backed extractor for similar data. The catalogue marks these honestly instead of returning silent empty results.

**Does CrawlGenie bypass CAPTCHAs or bot walls?**
No. It detects the challenge, stops the loop so no tokens are wasted on retries, and lets a person take over in the live view or use a saved login profile. The constructive answers are a residential proxy and the signed agent identity, which lets sites recognise your agent instead of challenging it.

**Does the knowledge base use embeddings or a vector database?**
No. It uses SQLite full-text search (FTS5) with BM25 ranking and porter stemming, on your own disk. It is exact-word retrieval: strong for "where does it say", weaker for paraphrase. Nothing is sent to a model to index or to search.

**What is a signed agent / Web Bot Auth?**
A cryptographic identity for your agent's HTTP requests (RFC 9421 signatures with an Ed25519 key). Sites that support Web Bot Auth, such as those behind Cloudflare's signed-agents programme, can verify who is browsing and allow you specifically. Turn it on in Settings → Signed agent identity or per session with `policy.signedAgent`.

**What does maxTokens do?**
It cuts a snapshot, text or scrape result at an exact token count using the real tokenizer, so a tool result never overflows the budget you give your model. Pair it with `q` (question focus) to keep only the passages that matter before cutting.

**Can I run an extractor or a crawl on a schedule, for example every morning?**
Yes. Any extractor or monitor becomes a *watch* with a cron expression (`POST /v1/watches { kind, input, cron }`, or "Turn into a watch" on a run); routines and crawls are scheduled under **Schedules**. Each run is stored with "what changed", can post to a webhook, and costs zero model tokens. Hosts that sleep can trigger due runs with `POST /v1/tick`.

**Can the agent log in to sites that ask for a one-time code (2FA / authenticator app)?**
Yes. Store the authenticator seed (the base32 secret or `otpauth://` URL you would normally scan into Google Authenticator or 1Password) in the vault, and let the agent type `{{totp:NAME}}` into the code field. The server generates the current 6-digit code at that moment (RFC 6238); the seed never reaches the model, the recording shows only the placeholder, and domain scoping limits where it can be used. Passwords work the same way with `{{secret:NAME}}`. SMS codes are not supported (nothing to generate from); saved **profiles** keep you logged in so the code is rarely asked twice.

**Can people sign in with Google?**
Yes. Set `VESSEL_GOOGLE_CLIENT_ID` and `VESSEL_GOOGLE_CLIENT_SECRET` (a free OAuth client from Google Cloud Console, redirect URI `<your public URL>/auth/google/callback`) and a **Continue with Google** button appears on the sign-in page and in the dashboard. A Google sign-in with the same e-mail as an existing account links to that account; a new e-mail creates a workspace when sign-up is enabled. No third-party auth platform is involved: accounts, sessions and API keys live in CrawlGenie's own database, passwords are scrypt-hashed, and Google is only asked "who is this person?".

**Do I need a separate database or auth service?**
No. CrawlGenie stores accounts, API keys, sessions, recordings, watches and memory in an embedded SQLite database on its data volume. For off-site backups, point `VESSEL_BACKUP_URL` at any S3-compatible bucket (Cloudflare R2, Oracle Object Storage, Backblaze B2) and hourly encrypted snapshots are kept automatically.

**What is a session?**
A real Chromium browser that an agent (or you) drives step by step. Each step is recorded with a screenshot, so you can replay, rewind, fork or take over live. Agents read a tidy snapshot of the page instead of raw HTML, which is why a look costs 3–8k tokens instead of 50–180k.

**What is a scrape?**
One page (or PDF, Word, Excel, slide deck, Google Doc) turned into clean markdown, text or links, or, with a question, only the passages that answer it. No browser session is kept open and no model tokens are used to produce it.

**What is a crawl?**
A whole site turned into markdown per page, with "ask across all pages" and one JSON answer from many pages. Use it to load a docs site into memory or compare many product pages at once.

**What is an extractor (ready-made agent)?**
A code-only job that returns a table for a known kind of site: jobs, reviews, places, products, posts, filings, papers, pricing pages, SEO reports. There are 61 of them and they use 0 model tokens. Run one from Agents, the Playground, the API or MCP.

**What is a watch (monitor)?**
Any extractor or page on a schedule with memory: CrawlGenie keeps the last result, tells you only what changed, keeps a trend series and a weekly report, and can post to a webhook, Slack or email. 0 model tokens per check.

**What is a routine?**
A session flow you taught once (log in, click, download) that replays by name with self-healing targets and zero model tokens per replay. Save one from any session with "Save as routine"; schedule it under Schedules.

**What is a schedule?**
Cron for routines and crawls (watches carry their own schedule). Timezone-aware, with a webhook when a run finishes.

**What is a profile?**
A saved login that new sessions resume, so the agent stays signed in to a portal without typing a password each time. Profiles take an exclusive lock so two agents never collide in one account.

**What is a secret, and what is an approval?**
A secret is a credential the model never sees: the agent types `{{secret:name}}` (or `{{totp:name}}` for a one-time code) and the server substitutes the real value. An approval is a pause before a risky click (pay, send, delete, confirm) until a person says yes in the dashboard or on a phone.

**What is memory (knowledge)?**
Everything your agents read, saved once into collections and searchable later with sources. The next question costs a few hundred tokens and never re-fetches the page.

**Do I still need Browserbase, Firecrawl, TinyFish or Apify alongside CrawlGenie?**
No. CrawlGenie covers what those four sell separately: browser sessions with approvals and saved logins (what Browserbase sells), scrape, crawl and search returned as passages (Firecrawl), goal agents that run in batches (TinyFish) and 61 ready-made agents at zero model tokens (Apify), plus monitoring and memory. One key, one flat plan, and a token receipt on every call that none of them offer. The homepage comparison and the `/compare/*` pages list the differences feature by feature.

**How do I contact support or ask a question?**
Email **hello@crawlgenie.com** (sales, support, privacy and security all go there) or use the chat bubble at the bottom right of the website: "Ask a question" answers from these docs instantly, "Message us" sends your email address and message to hello@crawlgenie.com and we reply within one working day. Self-hosters can read website messages at `GET /v1/contact` (admin key).

**Where does the website chat bubble get its answers?**
From these documentation pages, with retrieval only: no model is called, so the answer is the same for everyone and shows how many tokens of docs it read. If a question is not covered, use "Message us" and we will answer and add it here.

**What is a receipt?**
The line on every call that shows the tokens your model actually read and the tokens it avoided. The Token savings page adds them up per day, per member and per connector.

**Why would I upgrade if the Free plan has every product?**
Free is sized for one person testing: 2 browsers at a time, 120 browser-minutes, 1,000 reads, 300 searches, 200 agent runs, 2 watches, 100 memory documents, 3-day retention, no teammates. Pro multiplies the capacity and unlocks alerts to Slack and webhooks, scheduled routines, isolated browsers with raw CDP and proxy pools; Team adds unlimited members, priority support and invoicing. When a limit is reached the API answers `402 plan_limit` (or `402 plan_feature`) with the upgrade link, and **Usage & billing** shows every meter.