Build or Buy: Scripting Your Own AI Visibility Checks

The objection usually comes from whoever in the room writes code, and it is a reasonable one. All four assistants have public APIs, a loop over a list of questions is not difficult, and a company that already employs engineers may fairly wonder why it should pay a subscription for what looks like an afternoon's work.

The honest answer is that a minimal version really is an afternoon's work. What follows is how to build it, in the order it ought to be built, and then an account of the point at which the afternoon quietly turns into a standing commitment.

1. Write a small prompt set in your buyer's words

Twenty questions is plenty to start with. They should be the questions somebody asks before they know your name: who supplies a given component to a given standard, which software handles a given process for a firm of a given size, what the alternatives are to the obvious market leader. A prompt containing your brand is useless for this purpose, because a model will always find something to say about a name it has been handed.

Write them in the language the buyer actually uses, not in English with a translation pasted underneath. A German purchasing engineer does not phrase the question the way your English brochure does, and the model draws on a different body of text when it answers. Then freeze the list. Rewording a prompt between runs changes the instrument, and every later comparison inherits the change.

2. Call each assistant through its official API, with search switched on

The detail that matters here is retrieval. A bare API call answers from what the model absorbed in training, which is a different thing from an answer assembled after reading the live web. Wherever the API offers a search tool, enable it. OpenAI's Responses API has a web_search tool. Anthropic's API has a web search tool of its own. The Gemini API offers grounding with Google Search through a google_search tool. Perplexity's Sonar models are web-grounded by design, so there is nothing to switch on.

In the first three cases the model still decides, prompt by prompt, whether a search is worth running. Enabling the tool grants permission rather than issuing an instruction.

3. Store the raw answer, not the verdict

The tempting design writes a yes or a no to a spreadsheet and discards the text. Resist it. For every call, keep the full response, the date and time, the prompt exactly as sent, the language, the model identifier the API reports back, and whatever the provider returned about sources: citation annotations, lists of URLs consulted, search results. In six months you will want to run a better brand detector over old answers.

4. Detect the brand, which is the hard part

This is where the afternoon starts to run long. A plain substring search for your company name will miss more than you expect and catch things it should not. A model may drop the legal suffix, or use a different one; it may name the product rather than the company; it may shorten a two-word name to its first word, which also happens to be an ordinary noun; it may transliterate, decline the name in a language with grammatical cases, or hyphenate it. A common word in your name produces false positives. A larger group with a similar name produces worse ones.

Build a list of variants per brand and per language, and apply the same treatment to your competitors, because who gets named instead of you is at least half the value. Some teams hand the classification to a second model call. That works, up to a point, but it adds a second probabilistic step to a measurement that already has one, so a sample of its verdicts wants checking by a person.

5. Repeat each prompt, then repeat the whole run

The same prompt sent twice can return two different lists of suppliers with nothing having changed. A single call is an anecdote; what carries information is how often, across several calls, you were named. Send each prompt several times per run. Then schedule the run and leave it alone, because the useful output is that same proportion, measured again a fortnight later under the same conditions.

6. Work out where the afternoon ends

Everything above fits in a few hundred lines. The real work is arithmetic and upkeep.

The calls multiply. Twenty prompts, across 4 assistants, in 5 languages, with 3 repeats each, is 20 × 4 × 5 × 3 = 1,200 calls per run. Twice a month, that is 2,400 calls a month, before anyone adds a market. Every call is billed in tokens, and searching is billed on top in at least two cases: Anthropic charges per search, and Google charges per search query the model executes on its Gemini 3 models. Current rates live on each provider's pricing page, and are better multiplied out before the first invoice than after it.

The models retire. Providers withdraw models on published schedules; OpenAI keeps a deprecations page listing snapshots and their removal dates. When a model you call is removed, the script stops working, and the replacement you move to answers differently. That is a break in the series. It has to be marked, or later movement will be credited to the wrong cause.

The sources arrive in four shapes. Each provider returns citations in its own structure. Normalising them into one table of domains, per prompt and per language, is a small project in itself, and it is the part that tells you which pages the answers were built from.

The languages drift apart. Keeping the German and the Spanish version of a prompt asking genuinely the same question, as the set grows, needs someone who reads both and minds.

Someone has to own it. Scripts of this sort rarely fail loudly. They go stale: a key expires, a parser breaks on a changed response, the engineer who wrote it moves to another project, and the chart carries on showing the last good number for a quarter.

Build it, or pay someone to keep it running

For a company with an engineer who has the time, one market and one language, building is a perfectly sound decision. The set is small, the brand variants are manageable, and the knowledge of how the numbers were produced stays in the building, which is worth something.

The balance shifts as the matrix grows: more languages, more markets, competitors in each, and a history that has to survive model retirements. That upkeep is what a tool such as PSentry is there to absorb. It queries the four assistants through their official APIs, twice a month, per language and market, and moves to newer models as providers retire old ones. Whether that is worth paying for is a question about your engineers' time, not about the difficulty of the first script.

What the engineer asks next

Which assistant is the least work to start with?

Perplexity, because its Sonar models are grounded in web search by design and return a list of citations and structured search results with each answer. There is no tool to configure.

Is asking a model whether the brand was mentioned good enough?

As a second pass, yes. As the only pass, it swaps a detection rule you can read for a judgement you cannot, and that judgement varies from call to call like everything else here. Match the variant list first and let a model deal with the ambiguous cases.

How many runs before the numbers mean anything?

There is no universal threshold. A workable test is whether adding repeats still changes the proportion for a prompt. Prompts where you are named only sometimes need more repeats than prompts where you are always or never named, and a trend needs several runs, not two.

Can we keep the history when a model is replaced?

Keep it, but do not draw a line straight through the change. Record the date the model identifier changed and read the runs either side as two series. If nearly every prompt moves at once on one assistant, the new model is the likeliest explanation.