Real cases

Real cases, real findings, from AI agents.

We have run UX Surgery on a range of real websites and apps, from service websites to web apps behind a login and an Android app. These are real cases, anonymised, with the problems the AI agents found and what they said while using them.

Real sessions

What an AI agent does in a real session.

Short clips from recorded sessions on our own products: ProfileAlign on the web and Fotorie on Android and iPhone. Each one comes with the report finding it belongs to.

  • ProfileAlign Web

    Agent Alex, 26, customer service rep who has never heard of the service: looking for a phone number or email for the company.

    Finding Contact searches ended without a visible company address or phone number (8 sessions). Blocked

    AI agent session (synthetic participant) on our own product; recorded by UX Surgery.

  • ProfileAlign Web · phone

    Agent Riley, 35, marketing specialist with a €35/month budget, on a phone: does Job Hunter cover four applications?

    Finding The mobile comparison does not initially expose all plan columns. Doubt

    AI agent session (synthetic participant) on our own product; recorded by UX Surgery.

  • Fotorie Android

    Agent Mother of two with a full-time job: a quick way to understand her meals, starting from a breakfast photo.

    Finding The meal total was not found before saving. Doubt

    After an English sign-in the app switched to Spanish: a separate finding, seen in all 3 sessions.

    AI agent session (synthetic participant) on our own product; recorded by UX Surgery.

  • Fotorie Android

    Agent The same participant, breakfast logged: wants the coach’s advice on maintaining her weight.

    Finding Coach continuation opens with repeated safety text. Doubt

    AI agent session (synthetic participant) on our own product; recorded by UX Surgery.

  • Fotorie iPhone

    Agent Retired man whose doctor wants him to lose weight, who finds small text hard: signing in with email.

    Finding The keyboard partially obscures the next action (“Send code”). Doubt

    The session stopped here because the test does not type passwords at this step: a test limit, not an app failure.

    AI agent session (synthetic participant) on our own product; recorded by UX Surgery.

Clips are muted, from English-language sessions, with no personal data: the test account email is blurred. Findings are hypotheses backed by the video, not opinions of real people.

Real cases

Anonymised studies

Studies run with AI agents on real products. We removed names and any detail that could identify the client.

  • AI agents · anonymised

    Clinical-trial back office · web app behind a login

    • 69 sessions
    • 14 roles
    • 28 tasks

    What was tested

    Study coordinators, data entry, monitors and pharmacy staff, each with a test account, on tasks like registering a patient, logging samples or dispensing medication.

    Top 3 problems

    1. Identifiers already on screen had to be typed again (site, screening number, batch).
    2. After pressing Save, the empty required field was far from the button, so people scrolled the whole form to find it.
    3. A visit showed as “Done” while several of its forms still said “Fill in”.
  • AI agents · anonymised

    Real-estate agency website · desktop, tablet and mobile

    • 36 sessions
    • 6 user profiles
    • 3 formats

    What was tested

    A first-home buyer, a buyer abroad, an owner who wants to sell, a long-term renter, a small investor and someone looking for a warehouse.

    Top 3 problems

    1. Rental conditions (length, availability) were not next to the price.
    2. Owners who wanted to sell couldn’t find the agency’s fees.
    3. With “Rent” selected, the price filter still used sale prices.

These are hypotheses backed by session videos, not opinions of real customers.

Case in depth

uxsurgery.com: found, fixed, re-tested

This is our own website, so this case is not anonymised: the 3 problems the AI agents found, what we changed and what the re-test showed.

#1 Severity: Low Real finding

The offer is understandable, but who applies the fixes is unclear

Observed: After finding the price, the participant browsed the sample report and method pages to work out who makes the changes, and ended up inferring it was on them. Morgan, 38 · online shop owner, arrives with no context · desktop and mobile

Proposed fix: Say who applies the fixes right next to the fix prompt.

#2 Severity: Low Real finding

Supporting text was hard to read with simulated reduced vision

Observed: Repeatedly reported blurry small text and said it slowed down reviewing the sample report. Prices stayed readable. Elliot, 54 · founder of a small SaaS; simulated reduced vision, in a hurry · desktop and mobile

Proposed fix: Enlarge only the supporting text that makes reading harder.

#3 Severity: Low Real finding

Confirming the currency meant opening a pricing FAQ

Observed: Needed an extra click to confirm the prices were in US dollars and excluded taxes before finishing the cost check. Casey, 62 · digital operations director; imprecise taps · desktop

Proposed fix: Show currency and taxes next to the prices.

What we changed

  • Next to the fix prompt we now say who applies the fixes: you or your team, we never touch your site.
  • Currency and taxes next to the prices (before, only in the FAQ).
  • A direct link to the credit rules at the start of the monthly plans.
  • Larger supporting text for people who see less well.

What the report kept apart

Some failures came from the test setup, not the site: the agents had no website of their own to submit to the free first UX test form, and the automation rejected some imprecise taps. The report files these as test problems, not site problems.

Re-test: iteration 2

Same profiles and tasks on the changed site:

  • Fixed: currency and taxes next to the prices.
  • Fixed: the link to the credit rules.
  • Still there: who applies the fixes (1 session, down from 2).
  • Still unclear: supporting text with reduced vision.
  • New: the mobile video packs in too much small text; sample fixes take exploring to find; the free offer names the video two ways.

Task completion fell from 42% to 33%, mostly from limits of the test itself (agents did not submit the free form and some automated clicks failed), which the report files apart.

See the fix prompt it generated (Markdown)

AI agents, not real people: their findings are hypotheses backed by evidence, not opinions of real customers.

In their own words

What the AI agents said

Verbatim lines the agents “thought out loud” while using real websites and apps in our studies. They are not customers or reviews: they are the evidence behind each finding, and you can watch them on video.

  • “That is a clear fix that could also help my shop, but I still do not know whether they make the changes.”

    Shop owner, arriving with no context · desktop

    This website (uxsurgery.com)

    → Became finding #1. We fixed the wording.

    Verbatim

  • “I can read “Free. No card,” but the smaller details are blurry.”

    Founder in a hurry, simulated reduced vision · mobile

    This website (uxsurgery.com)

    → Became finding #2. We enlarged the supporting text.

    Verbatim

  • “I can see the home page and the “Shall we call you?” section, but there’s still no phone number or email.”

    Customer planning a new kitchen · tablet

    Kitchen showroom website

    Translated from Spanish

  • “I see the agency’s page and a cookie notice covering part of the content. I can’t see a phone or an email.”

    Events business owner · tablet

    Marketing agency website

    Translated from Spanish

  • “I see “Menu”, “Fridge” and “Shopping”, but I can’t find where to add my meal. There’s a big green button at the bottom right, with no words to guide me.”

    Retiree with tired eyesight · mobile

    Nutrition app (our pilot)

    Translated from Spanish

  • “Severity is “Grade 1 · mild”, the “Is it serious (SAE)?” box is unticked and the status is “Open”. I still can’t see the button to save.”

    Clinical research associate, logged in · desktop

    Clinical-trial back office

    Translated from Spanish

  • “The case form covers consent, history and medical tests; that’s clear to me. I don’t know what ECOG is and I don’t see any explanation.”

    Visitor with no context · desktop

    Clinical-trial back office

    Translated from Spanish

  • “I’ve read the coach’s limits, but I haven’t reached Privacy or “Where is my data?”. I’m leaving without deciding on the app.”

    User worried about health-data privacy · desktop

    Nutrition app website

    Translated from Spanish

Anonymised studies: no brands or domains, except for our own site.

An honest note

AI agents are not your customers. They are quick, cheap and tireless, and they find the problems a first-time visitor would hit. They do not know your market, your prices or your competitors the way people do. Every report includes a short script so you can check the findings with three to five real people.

What AI agents are and are not

See where people get lost on your site.

Paste your URL. In a few minutes you get the 3 worst problems and one session video.

3 AI agent sessions on your site, the 3 worst problems and a video.