Modern UI automation has one enemy: broken locators. 😤

Can LLMs watch the DOM at runtime, understand the element, and generate or repair the locator automatically?

Current Status: Locator breaks -> Test fails ->Human fixes

Future: Locator breaks -> LLM analyzes DOM -> Finds the right element -> Repairs locator -> Test continues

Self-healing automation powered by LLMs.

A developer changes: <button id=”checkout-btn”>Checkout</button> to <button data-testid=”checkout”>Checkout</button>

Your test fails even though the application behavior hasn’t changed. What if an AI agent could detect the failure, find the new element, verify it, and repair the test? That’s the idea behind a Self-Healing Test Automation Agent.

1. End-to-End Flow Diagram

2. Technology Stack

• Playwright — browser automation
• OpenAI — locator reasoning and candidate generation
• Claude — independent validation
• Node.js + TypeScript — orchestration
• SQLite/PostgreSQL — locator memoryGitHub Actions — CI execution

Note: Same can be replicated with selenium + java or any framework

3. Create a Normal Playwright Test

4. Simulate a Locator Failure

Now change the application:

<button data-testid="checkout">
  Checkout
</button>

The test fails:

Error: locator("#checkout-btn")
was not found

This failure becomes the input to our AI agent.

5. Capture Failure Evidence

Don’t send the entire application to the LLM.

Collect only useful information:

{
  "url": "/checkout",
  "action": "click",
  "originalLocator": "#checkout-btn",
  "error": "element not found",
  "target": {
    "tag": "button",
    "text": "Checkout",
    "role": "button"
  }
}

Also collect relevant DOM candidates.

For example:

[
  {
    "tag": "button",
    "text": "Checkout",
    "role": "button",
    "testId": "checkout"
  }
]

This keeps the AI input small and focused.

6. Generate Candidate Locators

Before asking AI anything, generate possible locator strategies.

For example:

getByTestId(“checkout”)
getByRole(“button”, { name: “Checkout” })
getByText(“Checkout”)

Prefer stable, semantic locators.

Avoid generating complicated CSS/XPath unless there is no better option.

7. Ask OpenAI to Reason About the Candidates

Send OpenAI:

Original locator:
#checkout-btn

Action:
click

Original element:
button
Checkout

Candidates:
1. getByTestId("checkout")
2. getByRole("button", { name: "Checkout" })
3. getByText("Checkout")

Ask it to return structured data:

{
  "strategy": "testId",
  "value": "checkout",
  "confidence": 96,
  "reason": "Same semantic button and checkout action."
}

The model isn’t executing anything.

8. Ask Claude to Validate the Proposal

Claude acts as an independent reviewer.

Example response:

{
  "valid": true,
  "confidence": 94,
  "reason": "The candidate uniquely identifies the same checkout action."
}

This second opinion is useful because we don’t want one LLM making both the decision and the validation.

9. Calculate Confidence

For example:

OpenAI = 96%
Claude = 94%

Final confidence = 95%

Set simple rules:

95–100 → Auto-heal
85–94  → Heal + log
70–84  → Suggest only
<70    → Don't heal

10. Let Playwright Verify the Repair

The agent proposes:

getByTestId("checkout")

Your application resolves it:

const locator = page.getByTestId("checkout");

Then verify:

const count = await locator.count();

if (count !== 1) {
  throw new Error("Locator is not unique");
}

await locator.click();

If the click succeeds, the repair is verified.

11. Save the Successful Repair

Once verified, store it.

Example:

{
  "test": "checkout.spec.ts",
  "original": "#checkout-btn",

  "healed": {
    "strategy": "testId",
    "value": "checkout"
  },

  "confidence": 95,

  "verified": true
}

This becomes the agent’s memory.

12. Create a Pull Request

The next step is GitHub integration.

The agent can create:

Branch: ai/heal-checkout-locator

Commit:AI: Heal checkout locator

Changed:checkout.spec.ts

And create a PR:

AI Self-Healing Test Repair

Original:#checkout-btn

New:getByTestId("checkout")

OpenAI confidence: 96%
Claude confidence: 94%
Runtime verification: PASS

For the first version, keep the PR human-approved rather than automatically merging it.

13. The Complete Architecture

14. MVP Scope

V1

• Playwright
• Element-not-found detection
• DOM candidate extraction
• OpenAI locator proposal
• Claude validation
• Playwright verification
• Locator memory

V2

• Screenshot analysis
• Flaky-test detection
• Failure classification
• Automatic Git diff
• GitHub PR

V3

• Root-cause analysis
• Test impact analysis
• Automatic test maintenance
• CI quality gates
• Autonomous QA agent

That’s where Agentic AI + Test Automation becomes genuinely useful by simply integrating LLM.

AgenticAI #AIinTesting #AITesting #TestAutomation #SelfHealingTesting #QA #SoftwareTesting #SDET #Playwright #OpenAI #ClaudeAI #GenerativeAI #AIAgents #AutomationTesting #QualityEngineering #DevOps #ContinuousTesting #TestEngineering #SoftwareQuality #TechInnovation