Key takeaways
- A VPN changes your IP address, but ChatGPT's answer also depends on your account, your prompt history, and the searches it runs behind the scenes. A VPN alone doesn't settle what a local user sees.
- Manual checks show you a single answer from a single moment. ChatGPT's behavior has changed overnight before, so two checks on different dates may not be comparable.
- The searches ChatGPT fires behind your prompt (query fan-outs) drive what it retrieves, and a manual check never shows them.
- For local or multi-location work, you need repeatable, location-pinned monitoring that runs daily, not a one-time screenshot.
- Manual checks are still fine for a quick sanity check or for generating prompt ideas. Don't build reports on them.
The tempting shortcut
Here's the workflow I see all the time. Someone wants to know whether their dental clinic, hotel, or franchise shows up when people in Manchester or Austin ask ChatGPT for a recommendation. So they switch on a VPN, set it to the target city, open a fresh ChatGPT window, type the prompt, and screenshot the answer.
It feels scientific. It isn't. It feels like you've stood in the customer's shoes, but what you've really done is take one sample from a system that varies by time, session, and account.
That doesn't mean VPNs are useless. It means they answer a narrower question than people think they do.
What a VPN actually does (and doesn't)
A VPN routes your traffic through a server in another place, so websites see that server's IP address instead of yours. Providers say this masks your real IP from ChatGPT. Proton's write-up on the topic puts the limit well: a VPN can hide your IP from ChatGPT, but not your prompts, your account, or ads.
That last part is the problem for anyone trying to test local results. The IP is one signal out of several that could shape a response. A few others to keep in mind:
- Your account. If you're logged in, ChatGPT knows things about you that no VPN touches. Settings, saved preferences, and past conversations can all colour an answer.
- Your session. Earlier messages in the same chat influence later ones. A prompt typed into a chat where you've been discussing your own company is not a clean test.
- Your device and browser. Language settings and device location permissions can differ from what the VPN reports.
There's also a visible user frustration in the wild. A community thread on OpenAI's forum asks how ChatGPT determined someone's location when they hadn't told it. Other users post in Facebook groups that ChatGPT misbehaves with or without a VPN. I can't tell you from those threads exactly how OpenAI infers location, and I'd be wary of anyone who claims to know the full recipe. OpenAI's own help documentation is the place to verify how it handles location, and it can change.
The practical takeaway: when you test through a VPN, you don't know which signal produced the answer. Was it the IP? The account? The wording? You can't isolate the variable, and that's the whole point of a test.
Why manual searches are a sample of one
Let's say you ignore the VPN question and just focus on the manual part. Even then, a single check has three weaknesses.
AI answers aren't stable
Run the same prompt twice and you often get different answers, with different brands and different sources. One run tells you what was possible. It doesn't tell you how often it happens. For a local business, the useful number is how frequently you appear across many runs, not whether you appeared once on a Tuesday.
You only see the final answer
When you type a prompt, ChatGPT may run several web searches of its own before writing anything. Promptwatch Data's ChatGPT query fan-out report tracks exactly this. The report says a single prompt can trigger multiple separate searches, though the measured averages were lower than that headline: about 2.15 fan-outs per response in early December, roughly 1.84 by early March, and 1.0 when data resumed in April 2026.
Two things follow from that. First, what ChatGPT retrieves depends on its own rewritten queries, not just your typed prompt, and a manual check never shows those. Second, the numbers moved. Average fan-out query length dropped from about 117 characters in December to the high 80s in February and March, and to around 53 characters in April. The report ties those shifts to changes in ChatGPT's behaviour. So a prompt phrasing that matched one quarter's queries may be stale the next.
The platform changes under you
This is the one that surprises people. According to Promptwatch's data on the site: operator, on August 8, 2026 the share of fan-out queries using the site: operator jumped from 0.37% to 16.8%. Average fan-outs per response rose from about 1.08 to about 1.83 on the same day, and stayed there. Promptwatch concluded it was a platform-wide rollout on OpenAI's side that landed in a single day.
Think about what that does to a manual comparison. Say you checked your visibility in London on August 5 and in Leeds on August 12. The two answers might differ because of location. Or because ChatGPT now searches differently. You can't tell which. Platform-controlled search behaviour can change overnight, which is why continuous monitoring beats one-off audits.
What this looks like in practice
Here's a realistic scenario. A regional estate agency checks ChatGPT through a VPN set to Bristol and finds they're recommended third. Great. A week later a colleague checks from Bath, via a different VPN provider, on a logged-in personal account with a long history, and the agency isn't mentioned at all.
What happened? Possibly any of these: different IP geolocation accuracy between providers, different account context, a different phrasing, random variation between runs, or a change in how ChatGPT searches. You'd have no way to tell. And if you present either screenshot to a client as the truth, you're presenting noise.
Manual vs VPN vs dedicated monitoring
| Approach | Location control | Repeatability | Shows fan-out queries | Tracks over time | Best for |
|---|---|---|---|---|---|
| Manual search, no VPN | Your own location only | Low | No | No | Quick sanity check |
| Manual search with VPN | Approximate, via IP only | Low | No | No | Spot-checking a hunch |
| Spreadsheet of manual checks | Approximate | Medium if disciplined, but slow | No | Sort of | Very small prompt lists |
| Dedicated AI visibility tracker | Varies by tool | High | Varies by tool | Yes | Ongoing reporting |
| Tracker with location pinning | Exact coordinates or city-level | High | Varies by tool | Yes | Local and multi-location brands |
When manual checks are still useful
I don't want to be dogmatic. Manual checking has real uses:
- Brainstorming prompts. Seeing how ChatGPT answers a handful of questions helps you decide what to track.
- Sanity-checking a tool. If your monitoring data says you're never mentioned, a manual look can confirm the answer is at least plausible.
- Understanding the user experience. Reading actual responses teaches you what format the answer takes, how many brands get named, and what sources get cited.
- Privacy. If your goal really is hiding your IP while you use ChatGPT, a VPN does that. Just don't confuse that goal with measurement.
The rule I'd follow: use manual checks to form a question, and use systematic monitoring to answer it.
What a better local check looks like
If you're measuring local AI visibility seriously, a good setup has a few traits.
Run many prompts, many times
Track a defined set of prompts that reflect how locals actually ask: the "near me" queries, the service-plus-city queries, the comparison queries. Run them daily so you see a rate, not an anecdote.
Pin the location deliberately
A tool that lets you set the location for a monitor beats a VPN, because the location becomes a documented setting rather than a side effect of a network route. Promptwatch does this by letting you pin a monitor to exact coordinates (with an accuracy radius from 20 m to 5 km) so ChatGPT, Google AI Overviews, and Google AI Mode answer as if asked from that spot. That's on its Business, Agency Scale, and Enterprise plans.
Track the real interface
User-facing answers and citations can differ from what you'd get through an API. Monitoring the actual interfaces gets you closer to what a customer sees. Promptwatch monitors the real UIs of ChatGPT, Google AI Overviews, AI Mode, Gemini, Perplexity, and others, by country and language.
Look at sources, not just mentions
Local visibility often comes down to which pages AI cites: your site, a directory listing, a review platform, a Reddit thread. Knowing where the answer came from tells you what to fix. Citation tracking and AI crawler logs help here, since they show whether AI systems can even reach your pages.
Keep a change log
When the platform shifts, you want to know. Because behaviour like the August 8 site: change happens platform-wide and overnight, annotate your reports with dates, so a sudden visibility swing isn't blamed on the wrong cause.
Tools to consider
There's a long list of AI visibility trackers, and they vary a lot in how they handle location. Before you commit, check three things: can you set location explicitly (and at what granularity), does it track the real interface rather than just an API, and does it do anything beyond reporting.
A few names from our catalog that track AI answers:
Otterly.AI
Profound
I haven't verified each one's location features, so confirm on their own sites before buying. If your business is genuinely multi-location, also look at local-focused platforms, which track classic local rankings and listings, and treat AI tracking as a separate layer.
For a wider comparison of trackers, the directory at ai-rank-tools.com and the GEO software listing at bestgeosoftware.com are good places to browse.
A simple workflow you can use this week
- Write down 20 to 40 prompts locals would realistically ask. Include service-plus-city, near-me, and comparison phrasings.
- Do a handful of manual checks (VPN or not) purely to sanity-check wording and see what answers look like. Don't record them as data.
- Set up a tracker that runs those prompts daily with an explicit location setting.
- Review rates over two to four weeks: how often you're mentioned, in what position, and which sources get cited.
- Note platform changes on your calendar. If a metric jumps on a single day across all your prompts, suspect the platform before suspecting your content.
- Fix the gaps: missing pages, unreachable content, thin local listings. Then re-measure.
The short version
A VPN tells ChatGPT you're somewhere else. It doesn't give you a controlled experiment. Add in answer variance, hidden fan-out queries, and a platform that can change its search behaviour in a day, and a manual spot-check is a hint, not evidence.
Use it to get ideas. When you need numbers you can put in front of a client or a board, use something that runs the same prompts from a fixed location every day and keeps the history.
If you want broader context on how AI answers get built, Promptwatch's research hub collects its citation and fan-out reports in one place.