Stop Letting Browser Agents Improvise Every Click. Use AI to Heal Playwright Instead.

The browser agent demo is always the same.

You give it a sentence. It opens a site, reads the page, finds a button, fills a form, and finishes the task while somebody narrates how the era of selectors is over. It looks magic right up until you run the same job 10,000 times and discover that the agent has spent real money deciding where the same button is, clicked the wrong "continue" three times, and has no useful explanation for why it stopped at 2:17 AM.

The web is messy. That is not a reason to ask a model to improvise every click.

For workflows you understand, I now prefer a hybrid shape: Playwright executes the normal path deterministically. The agent wakes up only when the workflow breaks, collects the evidence, diagnoses the failure, proposes a repair, and proves the repaired path in an isolated run. The next ordinary execution goes back to being a boring script.

It is less impressive in a demo. It is much better in production.

The Wrong Choice Is Usually Framed as Playwright Versus an Agent

That framing makes the decision sound ideological. Selectors are brittle, agents are adaptive, pick a side.

Both statements are true and neither tells you what to build.

A selector-based workflow is fast, cheap, and inspectable. When it fails, it normally fails loudly. You get a missing locator, a timeout, a screenshot, and a stack trace. The maintenance burden is that sites change, A/B tests move elements, and someone replaces the label you depended on.

An AI browser agent handles some of that variation well because it can reason about text and visual structure. It may find the checkout button after the class name changes or recover from a slightly different route. But each action is a decision with latency, token cost, and a non-zero chance of being wrong. The failure can look successful: the agent clicks a plausible button, extracts a plausible number, and stores a bad result.

I went through the full comparison of Stagehand, Browser Use, and Playwright earlier this year. The conclusion still holds: use an agent for ambiguity, not for the parts you can write down.

The Architecture: Deterministic by Default, AI on Failure

The pattern has four parts.

Scheduled job
  -> Playwright happy path
  -> success: validate result and store it
  -> known failure: retry once with normal recovery
  -> unknown failure: capture evidence for an AI repair run
  -> human approval or sandbox validation
  -> update the deterministic workflow

The agent is not the runner. It is the mechanic.

That changes what you ask the model to do. Instead of "go buy the cheapest plan," you ask: "This known Playwright workflow failed at step 4. Here is the URL, sanitized DOM snapshot, accessible tree, screenshot, previous locator, expected semantic target, and error. Identify whether the page changed. Propose the smallest repair. Do not submit the form or make an irreversible action."

The task is bounded. The evidence is concrete. The agent has a stop condition. Those are the same properties that make a coding-agent task safe to delegate in the first place.

Build the Deterministic Path Like You Expect It To Break

Self-healing does not rescue a bad script. Start with good Playwright discipline.

Use semantic locators first

Prefer getByRole, getByLabel, getByText, and stable data-testid attributes you control. A CSS selector that points at the third child of a div is a future incident ticket.

If you own the site, add test IDs for important, non-semantic interactions. If you do not own it, build locators around visible intent and record what the target is supposed to mean, not only what the DOM looked like last Tuesday.

Split actions from assertions

Every significant action should have a corresponding check. After a login, verify the account area. After a filter, verify the active state. After extracting a price, verify that the currency and plan name are present.

This is how you catch silent wrongness. A click returning no error is not proof that the click did the right thing. The assertion is the contract.

Set a real step budget

An agent and a script can both get trapped in retries. Put a ceiling on navigation attempts, clicks, and total time. A price-monitoring task that has not finished in two minutes should create an incident, not burn an hour retrying a page that is blocked by a consent wall.

Budget failures are useful signals. They tell you to investigate rather than letting an automation quietly convert a product change into a cloud bill.

Record the state that lets you debug tomorrow

On failure, save the URL, error, timestamp, page title, sanitized DOM or accessibility tree, screenshot, network status for relevant requests, and the last successful run. Do not give an agent a giant raw browser profile or every cookie it can find. Give it the smallest useful record.

The same caution applies to agent permissions. A browser session is not just a test tool. It can contain active accounts and irreversible actions. Keep repair runs in a separate profile with the minimum access the workflow needs.

What the AI Repair Step Should and Should Not Do

The repair step earns its place when the failure is ambiguous.

Maybe a site changed "Continue" to "Next step." Maybe a new modal blocks the form. Maybe an A/B test moved the price card. Maybe the DOM no longer exposes the text your locator needed. An agent can compare the expected target with the current accessible tree and propose a better locator or an alternate branch.

It should not get permission to finish a purchase, send a message, delete data, or publish a change because the original script timed out. Diagnosis and irreversible action belong in separate stages.

Here is the repair contract I would use:

• Read the failure evidence and identify the most likely cause.
• Propose no more than three repair candidates.
• Explain the observable target for each candidate.
• Run the candidate only in a sandbox or dry-run state.
• Return before-and-after evidence.
• Escalate if the page requests new authentication, a CAPTCHA, a payment confirmation, or a permission change.

The explicit escalation list matters. Browser agents are especially tempted to treat a new security control as an obstacle to work around. It is not. It is a boundary the automation has reached.

Cache Successful Decisions, Not Just Scripts

One useful extension is action caching. When the agent successfully heals a workflow, record the page state and the repaired semantic target. A page with the same normalized URL, role, accessible name, and nearby labels can use the repaired path next time without asking the model to rediscover it.

Do not cache a brittle coordinate or a single DOM position. Cache the reason you believe it is the right element. "Button with role button, name Continue to payment, inside checkout form" is a usable memory. "div:nth-child(4)" is a delayed failure.

This also gives you a diffable history of the site's changes. If the repair agent has made three changes to the checkout flow this month, that is information about the target system. It may be time to add monitoring, contact the vendor, or accept that the workflow is too unstable to automate cheaply.

The Cost Math Is Better Than It Looks

Say your workflow runs 10,000 times each month. A deterministic run takes three seconds and almost no marginal model cost. An agent-driven run takes 20 seconds and reasons through several observations. Even if the agent path is only a few cents per run, that is a real bill for a task whose normal state was predictable.

Now say the site changes on 1% of runs. Let the repair agent wake up for the failures, and perhaps only a small fraction need human review. You pay the expensive reasoning cost where it has a chance to add value. The other 99% remain cheap, fast, and easy to inspect.

There is a second benefit: an outage is visible. A fully agentic workflow can route around a change in ways that hide the fact the page changed. The hybrid workflow tells you that a deterministic contract broke. That makes it easier to audit data quality and easier to decide whether the new path is actually allowed.

Where This Pattern Fits

Use it for repetitive workflows with a known outcome and occasional UI drift:

• monitoring a public pricing page
• collecting reports from a vendor dashboard
• checking inventory or appointment availability
• regression testing a critical customer flow
• navigating an internal tool that changes too often for a completely static script

Do not use it as a way to automate around CAPTCHAs, rate limits, authentication challenges, or a site's terms. Those failures are not selector problems. They are signals that the workflow needs a human, a documented API, or permission from the service.

Also do not start with the repair agent if the path itself is not known. For a new, exploratory task, an interactive agent can help discover the flow. Once you understand it, turn the repeated part into a deterministic script. Exploration and execution are different jobs.

The Production Checklist

Before I put a hybrid workflow on a schedule, I want these answers:

• What exact result proves success?
• What happens when the result is missing or malformed?
• Which actions are read-only and which are irreversible?
• What is the maximum step and time budget?
• Which domain, account, and browser profile does it need?
• What evidence is saved on failure?
• Can the repair run affect the real system, or only a sandbox?
• Who approves a new action path before it becomes the default?

This looks like operational overhead until the first time a site redesign changes the meaning of a button. Then it is the difference between a failed job you can fix and a successful job that did the wrong thing for three days.

AI browser automation is useful. It is just most useful as a layer of judgment around deterministic software, not a replacement for deterministic software.

Frequently Asked

Is Playwright better than an AI browser agent?

For a known, repeated workflow, Playwright is usually faster, cheaper, and easier to debug. An AI agent is valuable as a fallback for ambiguous UI changes or exploratory navigation.

Can an AI agent automatically fix a broken Playwright script?

It can diagnose the failure and propose or test a repair in an isolated environment. Keep irreversible actions and production rollout behind a separate approval step.

How do I prevent a browser agent from taking unsafe actions?

Use a separate browser profile, minimum required credentials, domain and action allowlists, step budgets, and an explicit escalation list for authentication, payment, deletion, or permission changes.