Key takeaways
- UI tracking and API polling answer different questions. One captures what users see in the app, the other captures what a model returns through a developer endpoint.
- Most vendors do not publish their collection method. Treat any "real UI" claim as unverified until the vendor states it in docs, an FAQ, or an about page.
- I could not verify a method for all 22 platforms, so the table below labels each tool as disclosed, reported by a third party, or unknown.
- Google AI Overviews and AI Mode have no official API, so any tool tracking them is scraping search results in some form.
- Ask five questions before you buy: which surface, which country, which logged-in state, how often, and what happens when the UI changes.
Why this question matters
Two GEO tools can report different "ChatGPT visibility" for the same brand, and neither has to be wrong. As Martin Kulawik put it in an August 2026 write-up, they may simply be measuring different things. One vendor drives a browser against the chat interface. Another sends the same question to a documented API. A third pulls from purchased clickstream panels.
All three produce plausible answers. As measurements they are not the same, and your dashboard inherits that difference.
The article is worth reading in full. It is built on Achtung.app's own dataset, 105,028 AI runs since March 16, 2026, collected through documented APIs. Kulawik is upfront that he has no head-to-head data against browser scraping, so his point is methodological, not a vendor ranking. That honesty is rarer than it should be in this category.
What each method actually captures
API polling
You send a prompt to a documented endpoint and get a structured response. It is stable, cheap to run at volume, and compliant with the provider's terms. You can often control settings like temperature or whether web search is on.
The catch is that an API answer is not the answer a customer sees in the app. The consumer product wraps the model in its own system prompt, retrieval layer, personalization, and UI features like shopping carousels. Kulawik's data does show APIs returning source URLs reliably: OpenAI at 99.7% of responses, Perplexity at 99.5% before August 6, 2026 and 100% after, and Gemini rising from 84.9% to 100%. So APIs are good at structured sources. They are weaker at reproducing the user-facing surface.
Real UI tracking
A browser or automation layer opens the actual interface, submits the prompt, and captures the rendered answer, citations, and any product cards. This gets you what users see, including features that only exist in the app. The cost is fragility (UIs change), higher expense per response, and, as a Reddit thread on the topic puts it, possible terms-of-service friction. Vendors who scrape also have to deal with IP rotation and geography.
Clickstream and panel data
A third approach does not query models at all. It ingests prompts and responses from user panels. Semrush describes a 289-million prompt-and-response dataset sourced from AI search clickstream data, and Profound's Index methodology describes panels of millions of users. This is real behavior, but it measures what panel members did, not a controlled prompt set you chose.
The honest counterargument
Conductor, which sells an API-based approach, argues the opposite: that UI scraping over-represents a narrow user segment and under-represents how AI platforms behave for most users. There is something to that. A scraped session is one synthetic user in one location with one history. If a vendor does not control for that, its "real UI" data is a sample of one.
So neither method is automatically more truthful. The useful question is which surface you need to measure, and whether the vendor tells you how it measures it.
How the platforms line up
Here is what I could verify, with the source type labeled. "Disclosed" means the vendor states it. "Third party" means someone else says so. "Unknown" means I found nothing.
| Platform | Collection method | Evidence level |
|---|---|---|
| Promptwatch | Real UI of ChatGPT, Google AI Overviews, AI Mode, Gemini, Perplexity, Copilot, Alexa, Meta AI, plus coding agents; API models tracked alongside | Disclosed |
| Scrunch | Mix of browser automation and official APIs | Disclosed (own FAQ) |
| Semrush (incl. Otterly) | In-house clickstream dataset via owned supplier Datos; Otterly acquired | Disclosed (clickstream) |
| Profound | Panel-based prompt ingestion plus own response logging | Disclosed, interpreted by third party as purchased clickstream |
| SE Ranking | Crawls results pages directly; offers an AI Search API | Disclosed |
| Ahrefs Brand Radar | Large prompt index plus scheduled checks across ChatGPT, AI Overviews, AI Mode | Third party, verify attribution |
| Peec AI | Not disclosed; LinkedIn commentary says it scrapes chat UIs | Third party |
| Otterly | Listed by the same commentary as a UI scraper | Third party |
| AthenaHQ | Docs do not state how answers are collected | Unknown |
| Others (Qwairy, LLM Pulse, Rankshift, Brandlight, Bluefish, Searchable, etc.) | No method found in my research | Unknown |
Two things stand out. First, only a handful of vendors say anything specific. Second, Growth Method found that Peec, AthenaHQ, Scrunch, and KIME do not disclose how they collect responses in the pages it reviewed, and Peec's privacy policy lists 22 processors, none of which is a scraping vendor. That does not prove anything. It just means you cannot tell from the outside.
I did not find a verified list covering all 22 platforms, and I will not pretend to give you one. If a vendor is not in the table, assume you need to ask.
Where Promptwatch fits
Promptwatch is one of the few platforms that states its method plainly: it monitors the actual user interfaces of ChatGPT, Google AI Overviews, Google AI Mode, Microsoft Copilot, Gemini, Perplexity, Amazon Alexa, and Meta AI, in each country and language, and tracks API models like Claude, DeepSeek, Mistral, and Grok next to them. Its reasoning is the one this guide keeps returning to: user-facing answers, citations, and shopping recommendations can differ from API outputs.
The pairing matters. UI data tells you what customers see. API data tells you what a model believes from training. Having both, labeled separately, is more useful than either alone. Promptwatch's average sources per response data shows why the surface matters: ChatGPT cites about 5 sources per answer when web search triggers, Google AI Overviews about 10, Perplexity almost exactly 10, and Copilot swung from under 2 to nearly 17 within a few weeks before settling. Those are properties of the interface, and you only see them if you measure the interface. I did not find a Promptwatch report comparing UI-scraped and API-returned answers head to head, so treat the point as an argument for surface-level measurement, not a quantified gap.
Where the choice changes your data
Google AI Overviews and AI Mode
There is no official API for either. Any tool that reports them is reading search results pages. That makes this the clearest case where "API polling" cannot be the whole answer. SE Ranking says so openly. If a vendor claims API-only collection and also reports AI Overviews, ask how.
ChatGPT
ChatGPT is where the gap is widest, because the app adds retrieval behavior and shopping features. Promptwatch's data shows ChatGPT started using the site: operator at scale on August 8, 2026, jumping from about 0.4% to about 17% of fanout queries overnight. Behavior like that is a product-layer change. It is the kind of thing you want your tracker to notice, and the kind of thing a raw API call may not reflect.
Citation sources
The app and the API can also cite differently. Promptwatch's Reddit data shows reddit.com falling from roughly 4% to 0.5% of ChatGPT citations on August 14, 2026. If your tracker only samples through a channel that did not change, you would miss a shift your customers experienced.
Other tools worth checking
The catalog on this site includes dozens of trackers. A few worth asking about their method directly:
Profound
Otterly.AI
For a wider list, the GEO software directory and AI rank tracking directory cover far more than I can compare here.
Five questions to ask any vendor
- Which surface do you query: the consumer app, a documented API, or panel data? Get the answer per engine.
- Which country and language, and how do you set location? A scraped session from a datacenter IP in Virginia is not a user in Lyon.
- Is the session logged in, with memory or personalization on or off?
- How often do you run each prompt, and how do you handle run-to-run variance? AI answers are not deterministic, so one run is an anecdote.
- What happens when the interface changes? Ask how fast they repair collectors, and whether gaps show up in your history.
If the vendor cannot answer these, that tells you something. If the answer is a thoughtful "we use both, here is where each applies," that tells you more.
Reading vendor comparisons with care
A lot of the public material here is written by vendors with a stake. ContentMonk ranks tools by whether they scrape the UI, and it argues API answers differ from what users see. AthenaHQ published a 30-day pilot with AthenaHQ at +45% answer share, Peec at +8%, and Profound at -1%. Both are vendor-authored. Conductor's piece is too. I would use all of them as arguments to test, not as verdicts.
A practical way to decide
- If you need to know what customers see, including shopping cards, ads, and geography, favor UI-based collection and ask for proof of location control.
- If you need cheap, repeatable trend lines on model behavior, API polling is a reasonable base layer, as long as you accept it is not the app.
- If you want market-wide behavior rather than a prompt set you picked, clickstream data answers that, with the limits of panels.
- Best case, run a two-week pilot. Pick 20 prompts you can check by hand in the real apps. Compare each vendor's output to what you see. The tool that matches reality on your prompts wins, whatever its marketing says.
Bottom line
The label "real UI" is not a quality guarantee, and "API" is not a flaw. What separates good tools from bad ones is disclosure and control: a vendor that says what it collects, from where, and how often. Right now most do not. Among the platforms I could verify, Promptwatch and Scrunch state their methods, SE Ranking and Semrush document theirs, and the rest need to be asked. Do the asking before you sign a contract.