# Strategy: why CrawlGenie, why it lasts, and what it honestly costs

## 1. The layer we own

Every AI agent that touches the web needs three things Playwright does not provide: **a cheap way for a
model to see**, **guardrails the model cannot bypass**, and **operational memory** (logins, flows,
recordings). CrawlGenie is that layer. The model is a commodity that changes every quarter; the browser
engine is a commodity (Chromium); the layer between them is where the operational knowledge of a
business accumulates, and that is what people pay for.

## 2. Why it will not become redundant

We bet on standards and on things that compound, not on a model or a vendor:

| Bet | Why it survives model/vendor churn |
|---|---|
| **Accessibility tree (ARIA) as the observation format** | A W3C standard every browser maintains; works for text models today, remains the cheapest grounding for vision models tomorrow. Refs make actions deterministic regardless of model. |
| **CDP / WebDriver BiDi for control** | Browser-vendor standards; Playwright is our current adapter, not our dependency. The `Session` interface hides the engine, so Firefox/WebKit via BiDi, or a lightweight non-Chromium engine for read-only work, can slot in. |
| **MCP + plain JSON-Schema tools** | Whatever protocol an agent framework speaks, the tool surface is defined once (`tools/registry.ts`) and emitted in each shape. New protocol = new emitter, not a rewrite. |
| **Model-agnostic by design** | CrawlGenie never calls an LLM. A better model makes routines easier to teach and approvals rarer; it never makes governance, recording or profiles less necessary. |
| **Governance demand rises with autonomy** | The more agents act unattended, the more required "who approved what, with what evidence" becomes (EU AI Act human-oversight and logging duties; SOC 2 change control). Approval gates + flight recorder map to that directly. |
| **Customer data compounds** | Routines, profiles, recordings and policies are the customer's operational memory. They get more valuable with use and are painful to leave. |
| **OCI container deployment** | Runs anywhere for the next decade; no proprietary runtime. |

Threats we watch: browsers shipping native agent APIs (WebMCP) → CrawlGenie should expose a site's declared
tools through the same tool surface (roadmap); OS-level computer-use agents → the web remains the
universal interface for business software, and the governance layer applies to any action surface.

## 3. Token economics (measured, `scripts/tokens.mjs`)

| Observation style | Avg tokens / step (6 sites) | vs CrawlGenie |
|---|---|---|
| Raw HTML | 57,528 | 15.7× more |
| Full accessibility tree (Playwright MCP default) | 14,168 | 3.9× more |
| Screenshot (vision) | ~1,500 + extra steps for coordinates | comparable per step, more steps |
| **CrawlGenie interactive** | **3,670** | — |
| CrawlGenie delta (same page, small change) | 80–300 | 12–45× less than interactive |
| CrawlGenie routine replay | 0 | no model call |

Claims we make: "4–16× fewer observation tokens than DOM/a11y agents", "delta steps cut repeat
observations by 60–95%", "routines cost zero tokens per step", and every session shows its own
`tokenSavingsPct` and `estimatedSavingsUsd`. We do not claim a universal "10×": it depends on the site
and the agent's loop, which is why the meter is in the product.

## 4. Cost: what "$10/month" really means

The VM is $7–10/month (Hetzner CX23/CX33). Total cost of ownership for a small team is more like
**$10–40/month**: add residential proxy bandwidth if you need it ($1–8/GB on the open market, bought
direct), recording storage, and an hour or two of operator time. Density depends on workload:
25–35 concurrent shared sessions on 4 vCPU/8 GB holds for agent-paced browsing; sustained JS-heavy pages
drop that to 8–12. The hosted alternatives land at $100–300/month for the same usage, so the honest
multiple is 5–10×, not 30×. A future hosted CrawlGenie tier would need ~$0.02–0.04/browser-hour to cover
operations and margin, still at or below the cheapest incumbent.

## 5. Adoption without a sales network

1. **Zero-friction start**: `docker run` or `npx`, dashboard in 60 seconds, MCP in one command.
2. **The number**: every session shows tokens and dollars saved; screenshots of that are the marketing.
3. **Distribution surfaces that need no network**: GitHub, Docker Hub, npm, PyPI, MCP directories
   (Claude, Cursor, Smithery, PulseMCP), Show HN, the Playwright/MCP communities.
4. **Framework reach**: MCP + OpenAI/Anthropic/Gemini tool schemas + TS/Python SDKs means every tutorial
   can include CrawlGenie in five lines.
5. **Honesty as positioning**: published benchmarks with code, limits stated up front (no anti-bot
   promises), commercial licence. The category's top complaints are about vendors hiding exactly these things.

## 6. Definition of "best" we hold ourselves to

* Fastest honest observe→act loop per token (measured, published).
* The only open platform with fork/rewind, approval gates and self-healing routines.
* Secure by default: auth on, SSRF guard on, non-root container, approvals available.
* Works with any model, any framework, any language, on any Linux box.
* No feature exists without a test; no claim exists without a script that reproduces it.