How to test your Google AI Mode visibility for 50 queries in an afternoon

Key takeaways

• You can run a meaningful AI Mode visibility audit manually in one afternoon: 50 queries, incognito browser, a spreadsheet, and about 3-4 hours.
• AI Mode runs query fan-out, expanding each prompt into multiple sub-queries, so test conversational prompts, not just head keywords.
• Record five things per query: AI response present, your domain cited, competitors cited, content type cited, and sentiment.
• Your headline metric is the AI Mode citation rate: queries where you're cited divided by queries where AI Mode responded.
• Cross-check with Search Console's AI Mode filter, and scale later with a tracking tool once you know which queries matter.

Google AI Mode passed 1 billion monthly users at Google I/O 2026, and query volume has more than doubled every quarter since launch. If you haven't checked whether your brand shows up in AI Mode answers, you're flying blind on a channel that's eating clicks from traditional search.

The good news: you don't need an enterprise platform to get a baseline. A disciplined manual audit of 50 queries gives you ground truth about your visibility, and it takes one afternoon. This guide walks through the whole thing, from picking queries to calculating your citation rate.

Why a manual audit is worth your time

Automated AI visibility tools are useful, but they sample responses on a schedule and can miss nuances: whether an answer expanded, whether your brand was mentioned without a link, whether a competitor dominated the follow-up questions. A manual run gives you ground truth that automated trackers sometimes miss, and it's the best way to validate any automated setup you adopt later.

There's also a practical reason: Google's own data has a blind spot. Search Console has natively supported an AI Mode filter since June 2025 (Performance → Search Results → "+ New" filter → Search Appearance → "AI Mode"), but an impression only registers once a user scrolls the citation into view. Citations that never get clicked or scrolled are invisible. You could be cited for hundreds of queries with zero evidence in the dashboard. Manual testing fills that gap.

Google Search Console

Free tool to monitor Google search performance

View more

What you need before you start

Three things, none of which cost money:

• A fresh incognito or private browser window, with location set to your target market. AI Mode can pull on a signed-in user's Search and YouTube history through Personal Intelligence, so results vary meaningfully between testers. Incognito doesn't remove all bias (location, language, and device still matter), but it removes the worst of it. Standardize everything you can.
• A spreadsheet with one row per query and columns for the data points below.
• Access to AI Mode: open google.com, run a query, and tap the "AI Mode" tab below the search bar, or go directly to google.com/ai.

Block out 3-4 hours. Fifty queries at roughly 4-5 minutes each, including note-taking, gets you done before dinner.

Step 1: Build your list of 50 queries (45 minutes)

This is the step that determines whether the audit is worth anything. A bad query list tells you nothing about your actual market.

Where to pull queries from

Start with Search Console. Export queries from the last 90 days and sort by clicks, then by impressions. Your top traffic-driving queries are the ones where losing visibility hurts most.

Then apply a regex filter to surface conversational queries, the kind that trigger AI Mode: ^(?:\S+\s+){5,}\S+$ catches queries of six words or more. Run it again with {9,} for ten-plus-word queries. These long, question-shaped queries are where AI Mode lives.

Fill the remaining slots from:

• Commercial-intent prompts: "best [your category] for [use case]", "top [category] tools 2026", "alternatives to [competitor]"
• Problem-aware prompts: "how to fix [problem your product solves]", "why does [problem] happen"
• Brand prompts: your brand name alone, "[brand] vs [competitor]", "is [brand] worth it"
• Competitor prompts: run the same commercial prompts but where a competitor is the likely answer, to map who's winning where you're not

Mix the query types

Aim for roughly this distribution across your 50:

One more thing: write the queries as prompts, not keywords. AI Mode is conversational. "project management software" is a keyword; "what's the best project management software for a small marketing agency" is a prompt. Test both forms for your top terms if you have room.

Step 2: Run the test (2 hours)

For each query, in your incognito window with location locked:

1. Enter the query in AI Mode and let the full response generate.
2. Record five data points before moving on:

• AI response present (Y/N): did AI Mode generate a substantive answer? Almost always yes, but note it.
• Your domain cited (Y/N): check the citation chips or links panel carefully. Also note if your brand is named in the text without a link, which is visibility even without the citation.
• Competitors cited: list them. This builds your competitive map.
• Content type cited: product page, listicle, comparison, how-to, documentation, review site, forum, video. This tells you what format AI Mode wants for each query.
• Sentiment: when your brand appears, is it positive, neutral, or negative? A citation that misrepresents you is a problem worth knowing about.

3. Expand the follow-ups. AI Mode suggests related questions and often runs additional retrieval passes. Check whether the expanded answer cites sources the first pass didn't. A "show more" expansion can hide additional sources, so open it.

4. Note the fan-out. AI Mode typically decomposes each prompt into multiple sub-queries before answering, so a single test query can surface content you'd never see from the head term alone. If you notice a sub-topic appearing across several queries where competitors get cited and you don't, flag it. That's a content gap.

Work in batches of 10 and take a short break between batches. Fatigue at query 35 is real, and sloppy notes at that point poison the data.

Step 3: Calculate your numbers (30 minutes)

Two metrics matter, and both are simple:

AI Mode citation rate = (queries where you're cited ÷ queries where AI Mode gave a substantive answer) × 100. This is your headline number. If AI Mode answered 46 of your 50 queries and cited you in 9, your citation rate is about 20%.

Brand mention rate = (queries where you're cited OR named ÷ total queries) × 100. This catches unlinked mentions, which matter for how AI Mode describes your category even if no one clicks.

Then break both down by query type. A 35% citation rate on brand queries and 0% on commercial head terms tells a very specific story: you control your narrative but you're invisible at the moment of category evaluation. That's the pattern that should worry most brands, because it means buyers researching the category never see you.

Step 4: Cross-check with Search Console (20 minutes)

Open Search Console and apply the AI Mode search appearance filter. Compare:

• Queries where GSC shows AI Mode impressions but your manual test found no citation. Personalization or timing differences, worth re-testing.
• Queries where your manual test found citations but GSC shows nothing. Expected, given the scroll-into-view requirement, but confirms the dashboard undercounts.
• Watch for position anomalies: a keyword whose average position suddenly jumps to #1 with no on-page or link changes often signals an AI Mode citation credit rather than a real ranking change.

Also set up GA4 to catch AI referral traffic: Admin → Data Display → Channel Groups → new channel "AI Search" with the source matching regex like google.*ai|chatgpt\.com|perplexity\.ai|gemini\.google\.com|copilot\.microsoft\.com, placed above Organic Search in priority. One caveat: Google AI Mode and AI Overview clicks still arrive as standard Google organic in GA4, so this channel mainly catches non-Google AI platforms. Still worth having.

What the data tells you, and what to do next

Patterns worth acting on:

• Competitors cited, you're not, on commercial queries. AI Mode is choosing their pages as sources. Look at what got cited: often it's a comparison page, a pricing page with structured data, or a listicle that directly answers the prompt. Build the equivalent.
• You're cited but sentiment is off. The AI is summarizing a stale review, a Reddit thread, or an outdated comparison. You can't edit the internet, but you can publish current, unambiguous information about the topic so fresher sources outrank the stale ones.
• One content type dominates your category. If AI Mode cites product pages for your queries and you only have blog posts, that's a format problem, not an authority problem. Promptwatch's data on AI Mode citation share shows Google-owned properties (google.com, YouTube) capture over 10% of AI Mode citations, and every domain outside the top three sits under 1% share, so the open-web opportunity lives in the long tail. Match the format AI Mode actually cites.
• Sub-topics from fan-out that you don't cover. These are your next content briefs. Fan-out means AI Mode evaluates you on the sub-questions too, not just the head term.

When to move from manual to automated

A manual audit every month or quarter is fine for a small site. But if you're tracking 50+ queries across multiple locations, or you want daily data, automation pays for itself. Options at different price points:

If you go the tool route, re-run your 50-query manual audit once after setup to validate what the tool reports. Discrepancies between tool data and manual ground truth are common, and you want to know which one you trust.

Promptwatch

Track and optimize your brand visibility in AI search engines

View more

Common mistakes

• Testing signed in. Your search history personalizes AI Mode answers. Always incognito, always same location.
• Testing keywords instead of prompts. You'll undercount visibility on the conversational queries where AI Mode actually triggers.
• Ignoring the expansion. The follow-up questions and "show more" panels contain citations the first pass hides.
• Recording only your own citations. Competitor and content-type data is where the actionable insights live.
• Doing it once and never again. Visibility in AI Mode fluctuates as Google updates the underlying models. A single afternoon gives you a baseline; a quarterly repeat gives you a trend.

The whole thing, on one page

1. Build 50 queries: 10 head, 15 long-tail commercial, 10 informational, 10 brand, 5 competitor. Mix keywords and conversational prompts.
2. Incognito, location locked, google.com/ai.
3. Per query: AI response? Cited? Competitors? Content type? Sentiment? Expand follow-ups.
4. Calculate citation rate and mention rate, broken down by query type.
5. Cross-check GSC's AI Mode filter, set up GA4 AI channel groups.
6. Turn gaps into content briefs, then re-test in 60-90 days.

That's the afternoon. The first run is the slowest; by the second or third you'll cut the time in half and start seeing patterns you can act on immediately.