ripunjay← All work

[ 02 ] Agentic testing · Live project

MIMIC

Synthetic usability testing: six AI personas with different behaviours take the same goal through your site in parallel, and MIMIC shows where they got stuck, with the evidence.

In the recorded demo run, 2 of 6 personas reached the goal and 3 gave up on the same OTP step; the 11 findings covered 6 of the 12 detectable seeded defects. MIMIC simulates controlled behavioural assumptions. It does not predict real human behaviour, and it is for authorized targets only.

MIMIC — LIVE RUN — JOURNEY MAP ACROSS SIX PERSONAS
LIVE RUN — JOURNEY MAP ACROSS SIX PERSONAS
MIMIC — RUN REPORT — METRICS AND EVIDENCE-BACKED FINDINGS
RUN REPORT — METRICS AND EVIDENCE-BACKED FINDINGS

Inside the build

Six personas, one goal

Impatient, Low digital literacy, Power user, Cautious, Explorer, and Chaos each get explicit traits, an action budget, and a frustration threshold at which they give up. Their behaviour is enforced in code: the low-literacy persona only sees controls with visible text, the cautious persona must check fees and terms before payment or consent, and the chaos persona types edge-case inputs and double-clicks. Decisions come from Groq (qwen3.8-27b, gpt-oss-120b) and Gemini 3.5 Flash-Lite.

The AI can only pick from what’s on the page

Each persona is a LangGraph loop: observe the page as compact text with a numbered element menu, pick one action from that menu as JSON, apply the persona policy, act through Playwright, then update state with deterministic rules. The model cannot invent selectors or end its own journey by claiming success; counters, frustration, budgets, and success criteria are plain Python. A short journey memory stops personas looping on dead ends.

Findings come with evidence

Thirteen deterministic friction detectors run over the stored events and cluster repeated friction across personas. Each finding is split into Observed, Evidence, Interpretation, and Suggested investigation, and links to the events and screenshots behind it. The LLM only rewrites interpretation text for findings that already exist and is labelled as AI-written; it cannot add findings or change evidence.

Measured against seeded ground truth

SurakshaSetu, a fictional health-insurance onboarding site, ships with 13 documented usability defects, 12 of them detectable from text and behaviour. An evaluation script scores each run against that ground truth. The recorded run found 6 of 12 in 2 min 54 s; across six scored runs during development, MIMIC found 42–83%. Results vary because the agents are LLM-driven, which is why every finding links back to evidence.

Architecture and rate limits

A Next.js UI on Vercel talks to FastAPI on Modal over HTTPS and server-sent events. One Chromium runs six isolated browser contexts with no shared cookies or storage. A model router applies per-model token buckets with spillover; screenshots go to Gemini only when the text view is not enough, at most four per run and shared across personas. Runs, events, findings, and screenshots persist in Supabase.

Replay, tracing, and safety

Any persona can be replayed step by step: what it saw, what it decided and why, and how its state changed. Every persona run is a LangSmith trace with keys and screenshot bytes filtered out. Runs require an authorization checkbox; private and loopback targets are rejected, navigation is fenced to the target host, irreversible actions are blocked outside the demo site, and CAPTCHA challenges end the persona with no bypass.

LANGSMITH — PUBLIC TRACE OF THE CAUTIOUS PERSONA
LANGSMITH — PUBLIC TRACE OF THE CAUTIOUS PERSONA