How to automate Answer Engine Optimization testing across ChatGPT and Gemini using Python and Playwright — multi-browser, multi-region keyword matrix that tracks brand citations and surfaces competitor gaps.

What This Is

An automated Answer Engine Optimization (AEO) testing workflow that checks whether a target brand gets cited when users search AI-related keywords on ChatGPT and Gemini — across multiple browsers and regions. The output is a structured report that tells you exactly where you're being cited, where competitors are winning, and what to fix first.

Why I Built It

The first run was done entirely by hand — 36 keywords × 2 LLMs × 3 browsers × 2 regions = 432 individual searches. It took forever and was error-prone. This workflow replaces all of that with a Python script that runs the full matrix unattended and saves results after every single combination. Furthermore, the results produced here are used by an autoresearch loop I have create to rewrite and improve my existing blogs — read about it here.

Step 1: Defining the Requirements

💡 Prompt suggestion: "You are a Senior QA Automation Architect. I need a Python script that automates the following 6-step loop for each keyword in a list: [paste workflow below]. Output results to a CSV that appends after every single combination so no progress is lost if the script crashes."

A detailed prompt was written specifying the role (Senior QA Automation Architect), the exact workflow the script should follow, and the output format. The workflow was defined as a 6-step loop for each keyword:

The 6-step loop per keyword:

  1. Query generation — Take a raw keyword (e.g. "ai lip sync") and use an LLM API to convert it into a natural search query e.g. "How do I add AI-powered lip sync to my Unity characters?". To optimize token usage, you can directly create these queries using the free versions of Gemini or GPT.
  2. Browser launch — Launch Chrome, Edge, and Brave using real installed browser executables — not Playwright's bundled Chromium, which gets flagged by bot detection
  3. Execution — Navigate to chatgpt.com and gemini.google.com in incognito/guest mode, type the query, wait for generation to complete, scrape the full response
  4. Data extraction — Pass the scraped response to an LLM API to extract four fields: Brand Mentioned (YES/NO), Competitor Mentions, Result Type, Opportunity (1–2 actionable sentences)
  5. CSV writing — Append results after every browser × engine combination for crash safety
  6. Teardown — Close browser context completely to wipe session before the next keyword

Step 2: Generating the Master Cursor Prompt

Rather than writing Python directly, generate a detailed "Master Cursor Prompt" — a multi-page instruction document you paste into Cursor IDE.

💡 Prompt suggestion: "Write a complete, copy-paste ready Cursor prompt for a Python AEO automation script with the following file structure: main.py (orchestrator), config.py (constants), llm_client.py (LLM API wrapper), scraper.py (Playwright automation), csv_handler.py (read/write/resume logic), utils.py (logging and retry helpers). The script should support resume-from-crash, CLI arguments (--start-from, --browsers, --engines, --dry-run), and use playwright-stealth to reduce bot detection."