# How CrawlGenie works — a walkthrough for the owner

You run **one server** (Docker image). It owns a pool of Chromium browsers. Two kinds of users talk to it:

* **Humans** use the dashboard (`https://your-domain/`) to watch, approve, replay, teach routines and
  manage keys/secrets.
* **Agents / code** use the API (`/v1/...`), MCP (`/mcp`) or the function-calling tools (`/v1/tools`).
  They never see the dashboard; they get compact snapshots and act by `ref`.

## Who can use it: two modes

| | Self-hosted (default) | Cloud mode (`VESSEL_SIGNUP=true`) |
|---|---|---|
| Who gets in | Keys you put in `VESSEL_API_KEYS` | Anyone who signs up in the dashboard (email + password) |
| Where accounts live | nowhere — keys are config | `users`, `orgs`, `api_keys` tables in `/data/vessel.db` |
| Isolation | all keys see everything (optionally quotas per key) | each organisation sees only its own sessions, profiles, routines, secrets, keys; env keys are admins |
| Keys | fixed | minted/revoked in **Setup → API keys**, shown once, stored hashed |

So: *yes, the database is in-house.* Sign-ups are rows in SQLite on your VM; nothing goes to a third party.

## A session, end to end

1. **Create** — `POST /v1/sessions` (or the **New session** button). Everything is optional:
   * `url` – where to start.
   * `tier` – `shared` (pooled Chromium, fastest, default) or `isolated` (own Chromium + raw CDP).
   * `profile` – resume a saved login; `profileLock` exclusive by default.
   * `proxy` – `{ pool, country, sticky }` or your own `{ server, username, password }`.
   * `policy` – the guardrails: `allowDomains`, `denyDomains`, `blockResources` (images/media/fonts), `maxActions`,
     `maxDurationSec`, `idleTimeoutSec`, `maxBytes`, `approvals: { mode: off|destructive|all, patterns }`, `record`.
   * `snapshotDefaults` – `{ mode: interactive|full, maxChars, delta }`.
   * `viewport`, `userAgent`, `locale`, `timezoneId`, `extensions` (isolated), `metadata` (labels for the dashboard).
   You get back `SessionInfo`: `id`, `liveUrl` (dashboard page), `cdpUrl` (isolated), `expiresAt`, `usage`.
2. **Observe** — `GET /v1/sessions/:id/snapshot` returns the page as a short accessibility tree with
   `[ref=eN]` markers (2–8k chars instead of 50–500k of HTML). `delta=true` returns only what changed.
3. **Act** — `POST /v1/sessions/:id/act { actions: [...] }` with `click/fill/type/press/select/check/scroll/
   upload/goto/wait/assert/evaluate/tab/dialog` by `ref`. The response includes the next snapshot, so one
   round trip per step. Credentials go in as `{{secret:name}}`.
4. **Guardrails run server-side** — a navigation outside `allowDomains` is blocked; a click that looks
   destructive (pay/send/delete, or your `patterns`) is **held** until a human approves it in the dashboard
   (or via `POST .../approvals/:aid`, or your webhook).
5. **Everything is recorded** — each step with a screenshot and storage state. The dashboard shows the live
   screen (WebSocket screencast), the timeline, the exact snapshot the model saw, usage and the network log.
   You can **take over** with mouse/keyboard, **fork** a new session from any step, or **save as routine**.
6. **Close** — `DELETE /v1/sessions/:id`, or idle/max timers do it. `usage` reports actions, bandwidth,
   observation tokens read vs raw, and the estimated dollars saved.

## Routines, profiles, secrets, one-shots

* **Routine** = a recorded flow with `{{params}}`; replays re-find elements by role + name (self-healing),
  cost zero model tokens, and can carry `assert` steps so they double as monitors. Run from the dashboard,
  `POST /v1/routines/:name/run`, or the `browser_run_routine` tool.
* **Profile** = cookies + localStorage saved from a session; pass `profile` on create to resume logged in.
* **Secret** = a credential stored server-side; the model types the placeholder, the server types the value.
* **One-shot** = `POST /v1/scrape`, `GET /v1/screenshot`, `GET /v1/pdf` when you only want the page.

## How to test it end to end right now

On your own machine (already running at `http://localhost:7070` during development) or on the VM after
`deploy/install.sh`:

1. Open the dashboard → **Get started**. Click **New session**, URL `https://news.ycombinator.com`, approvals
   `destructive`. Watch the live view; open **Snapshot** and note the token count vs raw.
2. Click **Take over**, log in to any site you own, then **Save profile**. Create another session with that
   profile: it starts logged in.
3. Scrub the **Timeline**, click **Save as routine**, then run it from **Routines** — zero model tokens.
4. **Secrets** → add `demo_pw`; in a session, `fill` a password field with `{{secret:demo_pw}}` (via API or MCP)
   and confirm the recording shows only the placeholder.
5. Hand it to a model: `claude mcp add --transport http vessel http://localhost:7070/mcp --header "Authorization: Bearer <key>"`
   and ask Claude to "open example.com and tell me the heading". Or `pip install vessel-browser` and run
   `packages/python/tests/smoke.py`.
6. Cloud mode: start with `VESSEL_SIGNUP=true`, open the dashboard, sign up, mint a key under **API keys**,
   and repeat step 5 with that key. Sign up a second account and confirm it cannot see the first one's sessions.