API or Browser Scraping: How AI Visibility Data Is Collected

Every AI visibility tool has to get its answers from somewhere. Before a dashboard can tell you that Gemini named you in an answer about payroll software in German, something has to actually ask Gemini that question and read what comes back. There are only two ways to do that, and hardly any buyer asks which one they are paying for.

It is not a trivia question. The collection method decides what your data means, how stable it is over time, and whether the score you are staring at describes the system your customer is talking to or a close relative of it.

Two doors into the same model

The official API

Every major model provider exposes a programmatic interface. You send a request containing a prompt, you name a model, you get text back. It is the same pipe developers use to build applications on top of ChatGPT, Claude, Gemini or Perplexity. A tool that collects this way pushes your prompt set through that pipe on a schedule and stores the responses.

The browser

The other approach is to drive the consumer product the way a person does. Open a browser, log into an account, type the prompt into the chat box, wait for the answer to finish streaming, scrape the text off the page. In practice this means headless browser automation, often routed through residential proxies so the traffic does not look automated, sometimes with artificial delays between keystrokes to survive bot detection.

Both methods end with a block of text that either mentions your brand or does not. That is where the similarity stops.

Why the two do not return the same answer

The consumer chat interface is not a thin skin over the API. It is a product, and products have opinions. Several layers sit between the box you type into and the model that answers:

• A system prompt you never see. The chat product ships with its own instructions about tone, safety, formatting and when to search the web. The API, by default, does not carry those instructions. Same model weights, different briefing.
• Retrieval that may or may not fire. The consumer app can decide, on its own, to go and search the live web before answering. An API call answers from what the model absorbed during training unless you explicitly turn a search tool on. This single difference can flip a brand from absent to present, because a young company that the training data barely knows can still be found by a live search.
• Memory and personalization. A logged-in account accumulates context: past conversations, stated preferences, custom instructions. The API call is a stranger every time.
• Routing. Consumer products increasingly decide for you which model handles a request, and swap that logic without telling anyone. The API makes you name a model.

The net effect is that the same question, asked through both doors on the same afternoon, can return two different lists of recommended vendors. Neither is wrong. They are answers to slightly different questions.

What each method gets right

The API is reproducible. You know which model you queried, you can query it again with the same parameters, and if the answer changes you can reasonably suspect the web changed rather than the measurement apparatus. It is versioned, so a vendor can tell you exactly what was asked and of whom. It scales without a fight, and it is what the provider built for programmatic access, which means nobody is pretending to be a human being to get it.

The browser is closer to the lived experience. If you want to know what your actual buyer sees, on the actual surface they use, with the search toggle in whatever state it defaults to today, the browser is the only thing that shows you. That is a real argument and it deserves to be taken seriously.

What each method gets wrong

The honest weakness of the API is that it is a proxy. It is not literally the screen your customer looks at. If a tool queries a model without retrieval enabled and reports that you are invisible, it has told you something true about what the model knows, and something incomplete about what the consumer app would have said. A vendor that does not disclose this is letting you draw the wrong conclusion.

The weaknesses of browser scraping are more numerous and less discussed:

• It breaks. A layout change on the provider's side can silently corrupt the parsing. The dangerous failure is not the loud one where the tool returns nothing. It is the quiet one where it keeps returning something, and the something is now wrong.
• It is adversarial. Providers actively detect and block automation, and their terms of service generally prohibit it. Building a data product on a collection method the counterparty is trying to shut down is a fragile business model, and the fragility is passed to you.
• It is hard to reproduce. Which account, on which plan, with which memory, from which network, with which feature flags rolled out to that account this week. Change any of these and you have changed the experiment.
• The account itself drifts. Run your prompts from the same logged-in account for months and it accumulates history. The tool is then measuring what a very strange user, one who asks nothing but competitive research questions, gets told.

The trade-off nobody wants to state out loud

There is no correct answer here, only a choice between fidelity to the consumer surface and reproducibility of the measurement. You can be close to what the user sees, or you can be sure that a change in your score means a change in the world rather than a change in your scraper. Optimizing for both at once is marketing, not engineering.

What you are entitled to is not a perfect method. It is a stated one. A vendor that will not tell you how it collects its data is asking you to trust a number whose provenance it considers a secret, which is an odd thing to ask of a customer buying a measurement product.

What to ask before you buy

• Do you query official APIs, or do you automate the web interface? A vague answer here is itself an answer.
• If APIs: is web search or retrieval enabled during the query, and can I see that setting?
• If APIs: which model version, and do you tell me when it changes underneath me?
• If browser: from what kind of account, and what happens to my historical data when the interface changes?
• Either way: how many times do you run each prompt before you report a result? A single run of a probabilistic system is an anecdote.
• Can I see the raw response text behind any mention you claim, or only the aggregated score?

Our method, stated plainly

PSentry collects through the official APIs of the four platforms it covers: ChatGPT, Claude, Gemini and Perplexity. That is a declaration of method, not a claim of superiority. We took reproducibility over fidelity to the consumer interface, because a scraped answer carries variance we cannot separate from the signal we are trying to measure, and because a comparison across languages and markets is only worth running if the apparatus is the same in every one of them.

The cost of that choice is real and we are not going to hide it: we are measuring what the models will say through the door that providers built for machines, not the door your customer walks through. Twice a month is the rhythm chosen over continuous polling, on purpose, and every answer behind that rhythm sits there for you to read yourself, not compressed into a single score you have to take on faith. If a tool will not show you the text, the score is a rumour.

Frequently Asked Questions

Is browser scraping illegal?

Legality and terms of service are different questions, and we are not lawyers. What is uncontroversial is that the major providers prohibit automated access to their consumer interfaces in their terms, and invest in detecting it. That is a business risk sitting inside your data pipeline, and you should know whether it is there.

If APIs answer from training data, does that make them useless for measuring visibility?

No, but it makes them a different lens. What a model says without searching tells you what it has internalized about your brand, which is the slower and more durable signal. A live search result can change tomorrow. Both are worth knowing; a tool should tell you which one it is showing you.

Can a tool do both?

In principle. In practice you would then have two data sets that disagree, and someone has to decide which one becomes the score in the dashboard. Ask how that reconciliation is done, because it is where the interesting assumptions hide.

How would I catch a vendor that quietly changed its collection method?

Look for a discontinuity in your own history: a step change in the score that lines up with nothing you did and nothing that happened in your market. Then ask for the raw responses on both sides of the step. If they cannot produce them, you have learned something.

Does the collection method matter more for some businesses than others?

It matters most if you sell across several languages. The differences between markets are often subtle, and a noisy or drifting collection method will drown them. If you only ever look at one language, you can tolerate more noise, because the signal you are looking for is coarser.