How to report AI search performance to executives without overclaiming

Report AI search as one headline number per period. Beside it, put the size and shape of the sample it came from, the competitor set it is measured against, and the caveat that week over week movement can reflect a model update rather than a real change in the market. Then spend the rest of the page on two or three verbatim answers and the single decision you want from the room.

Executive reporting on AI search usually fails for one reason. The number is presented with the confidence of a rank tracker, and the first sharp question takes the whole report down with it. Someone opens ChatGPT on their phone, asks the same question you sampled, gets a different answer, and says so out loud. If your slide claimed a rank, you have nothing to say. If your slide claimed a sampled share with its method stated, you have already answered them.

So the job is two things at once. Say something definite, because a report that hedges every sentence reads as no signal at all. And be specific in advance about what the measurement cannot prove, so the caveat arrives from you rather than from the most sceptical person at the table.

The question that starts most of these projects is not a metric request. It is a version of "how does ChatGPT describe us?", often prompted by a competitor showing up in an answer where the brand did not. That matters, because it tells you what the report has to resolve.

Underneath it there are three decisions an executive is trying to make:

• Are we present, or are we absent? Absence is the finding that changes budgets. Presence at a middling level rarely does.
• Is a rival being named instead of us, and by name which one? Share language only means something against a named set. Without rivals in the denominator you are reporting a count, not a share.
• Is this getting better or worse, and did we cause it? This is where overclaiming happens, because causation in AI answers is weak and the temptation to take credit is strong.

Write the report to those three, in that order. Anything that does not serve one of them is decoration.

The one thing not to promise

Answer engines do not hand you a clickstream. You cannot trace a sentence in an AI answer to a pipeline number with presence data, and if you imply you can, you will be asked to prove it next quarter. Report presence, sentiment and position. Let the demand-generation report own revenue.

Ten years of search reporting trained everyone in the room to expect a rank, a volume and a click. AI search gives you none of those cleanly, and the report has to be built around the differences rather than papering over them.

There is no single result page. There is a set of questions, an answer to each, and your brand's position within each answer. The unit of measurement is the prompt set, not the keyword.

The output is non-deterministic. Ask the same question twice and the wording, the order and sometimes the brands named will differ. This is the standard objection to buying anything in this category, and it is a fair one. The answer is not to pretend the variance is absent. It is to sample enough prompts, on a fixed set, and report the level rather than the decimal.

The measuring instrument changes underneath you. Providers ship model updates without asking. A score that moves the week a model version changes may be telling you about the model, not about your market. Tagging every stored answer with the model version behind it is what lets you tell those two apart, and saying which happened is the difference between a credible report and a guess.

Coverage is partial by definition. No tool reads every answer engine, and any that implies otherwise is overclaiming on your behalf. State which providers you sampled. Seenby samples two at present, OpenAI and Perplexity, and the report should say so plainly rather than say "AI search" and let the room assume everything.

"But the answers change every time I ask"

Agree immediately, then explain the design. A single ask is one draw from a distribution. You sample dozens of prompts on a fixed set on the same cadence, so the level is stable enough to compare even though any individual answer is not. Then give them the honest limit: this method detects large and sustained changes well, and small weekly movements badly. Anyone who promises precision here is selling you something.

"What is a good score?"

There is no published benchmark worth quoting, and inventing one is the worst thing you could do in this meeting. The useful frame is relative and internal: your level against your three named competitors, and your level against your own previous readings on the same prompt set. Say that plainly. "Good" is being named in the prompts where a buyer is choosing, and that is a prompt-level answer, not a score-level one.

"So what do we do about it?"

Answer from the prompt list, not from the headline. The prompts where you are never named tell you which questions the market does not associate with you. Some of those are worth contesting with content, product messaging or a comparison page. Some belong to a rival for structural reasons and are not worth the quarter. Bringing that judgement is the reason a person presents the report rather than emailing a dashboard link.

The reporting rhythm matters as much as the wording, because it decides how much variance reaches the executive.

Weekly, for you. Read the score, the sentiment split and the prompts that changed hands. Check the model versions before you interpret anything. Most weeks the answer is that nothing happened, and knowing that quickly is the point.

Monthly, for the executive. Four readings smooth most sampling noise, and a month is long enough for a content or messaging change to show up in answers if it is going to. This is where the six-line slide goes.

Quarterly, for strategy. Look at the prompt clusters rather than the headline. Which parts of the category name you consistently, which name a rival consistently, and what changed in the answers themselves. This is the reading that informs positioning, and it is worth writing as prose rather than a chart.

One warning about escalation. If you send an alert every time the number moves, the alerts get filtered. Reserve them for sustained movement and for cases where you can point at a model version change as the likely cause. Fewer, better-explained alerts keep the metric trusted.

A short list to run through while the report is still editable.

• Is the prompt set identical to last period? If not, say so and do not draw a trend line across the change.
• Are competitors named, and are they the ones leadership argues about? A share against an empty set is not a share.
• Does every number on the slide carry its sample and its caveat in readable type?
• Have you checked model versions for the window, and can you say whether a move is likely drift from an update or a real shift?
• Is there a verbatim answer on the page, dated?
• Have you removed every decimal point the data does not support?
• Is there exactly one decision, with a name and a date against it?
• Have you said out loud what this data cannot tell you, before someone else does?

If you are producing this report for clients rather than for one employer, the constraint to plan around is different: Seenby tracks one brand per subscription, which shapes how an agency runs several accounts. The agency page in this pillar covers that arrangement. Otherwise the discipline is the same for everyone in this category right now. State the method, report the level, show the evidence, name the uncertainty, and leave the editorial call where it belongs.