Model-version tagging: telling real drift from a model update
Every answer Seenby stores is tagged with the exact model version that produced it, so drift in your Share of AI Voice can be traced to the market or to the model. That tag is what turns a moving number into something you can explain to the people who ask about it.
Seenby is not open yet, so anything on this page about what it does is what it is being built to do.
The week your number moves and nobody can say why
Your Share of AI Voice falls. Before you brief anyone, you need to know which of two things happened: the market moved, or the model did.
LLM answers are not fixed text. Providers version their models, and the version answering in March is not the version answering in June. A brand named third in a list of recommendations can vanish from that list because the new version answers in prose rather than a list, with nothing about your brand, your content or your competitors having changed. The reverse happens too. A version that starts naming more vendors per answer can lift everyone's mentions at once, including yours.
This is the objection every buyer in this category raises, and it is the correct objection: the results change every time you ask. Any tool that reports a weekly number without recording which model produced it is asking you to trust an average over a moving foundation.
The cost of guessing is specific. A content push gets credited with a rise it did not cause, so you do more of something that did not work. A real competitive loss gets written off as noise, so you do nothing for a quarter. Both mistakes look identical on a trend line.
Model-version tagging is the small piece of plumbing that separates them. Seenby records the version on every stored response, which is what makes the question answerable at all.
How model-version tagging works in Seenby
- The prompt set is frozen at setup
Your brand name, description and up to three named competitors produce a set of 50 to 100 prompts, and the same set runs every week. Holding the questions still is the first condition for reading drift: if the prompts changed too, nothing would be comparable.
- Every answer is stored with its model version
Each response from OpenAI and Perplexity is written down with the model version that produced it attached to the record. Not a provider name alone, the version. That tag is the evidence, and it is stored on every response rather than sampled.
- A version change becomes a labelled event, not a mystery
When the version tag on this week's answers differs from last week's, that difference is visible in the record instead of being buried in the score. The comparison you care about, this week against last week, now carries a note about which model answered each time.
- The weekly digest raises a drift alert when a version changed
Seenby is built to send a weekly email with the top insights and, when applicable, a drift alert tied to the model-version change behind the movement. The alert exists so the first person to see the number is not the last person to learn why it moved.
- The methodology caveat travels with the alert
Every digest includes the methodology caveat, in the email itself. Week-over-week shifts can reflect a model update rather than a real competitive change, and Seenby says so in the same place it reports the figure.
- The tag stays on the answer log
The full answer log is kept scrubbed and tagged with its model version, so months later you can see which version produced a given answer instead of relying on memory of what the models were doing that month.
Reading the same drop two ways
| What the week shows | Consistent with a real change | Consistent with a model update |
|---|---|---|
| Mentions fall across most prompts at once | Less likely: competitive movement tends to show up prompt by prompt | More likely, if the version tag changed in the same week |
| One named competitor gains roughly what you lost | Consistent with a shift in how the model ranks you against them | Possible, if the new version reorders or shortens its lists |
| The version tag is identical to last week, on both providers | The change sits in the answers, not in the model | Ruled out for that week |
| One provider moved, the other did not | Points to something provider specific in how you are described | Points to a version change on that provider alone |
What the tag buys you
An attribution, or an honest 'we cannot tell yet'
Sometimes the tag confirms a real move. Sometimes it shows the model changed underneath the measurement. Both answers are more useful than a confident number with no provenance, and Seenby is built to give you whichever one the record supports.
Drift alerts that arrive with the reason attached
The weekly digest is made to name the model-version change alongside the movement, so you read the explanation and the figure in the same minute rather than reconstructing it later.
A number you can defend in a monthly review
When someone asks whether a four point drop is real, you have the version history behind it and the methodology caveat in writing. That is a different conversation from defending a dashboard you cannot interrogate.
Two providers kept separate
OpenAI and Perplexity version independently. Because the tag sits on each stored response rather than on the weekly total, a change on one provider does not silently present itself as a change in the market.
An answer log that still makes sense in six months
Answers are retained scrubbed and version tagged for as long as the account exists, because the week-over-week trend is the product. Old weeks stay readable in the context of the model that produced them.
What the tag cannot tell you
We would rather be specific about the limits than sell certainty we cannot deliver.
- It cannot explain why a new version answers differently. Providers do not publish the internals of a model update. The tag tells you the version changed and when; the interpretation is yours.
- It cannot make LLM output deterministic. The same prompt, on the same version, can come back worded differently. Version tagging separates one source of variance from the others; it does not eliminate variance.
- It cannot see before you start. Tags begin with the first week Seenby reads your prompt set. There is no backfill of how the models described you last year.
- It cannot cover providers we do not read. Seenby reads OpenAI and Perplexity. Movement on a model outside those two is outside what the tag can account for.
If you want the computation itself rather than the drift plumbing, the methodology page in this pillar sets out how Share of AI Voice is calculated and why the caveat is attached to every score display rather than kept in a footnote.
Seenby is in build and open for the waitlist.
Questions
What is model-version drift in LLM tracking?
It is a change in what an AI answer says about your brand that comes from the model being updated rather than from anything changing in your market. Because the wording, length and structure of answers can shift with a new version, mention counts and ranking positions can move on their own.
How do I tell a model update from a real change in my visibility?
You compare weeks that were produced by the same model version, and you treat a week where the version changed as a break in the series rather than a clean comparison. Seenby tags every stored response with its model version so that comparison is possible, and raises a drift alert in the weekly digest when a version change is behind the movement.
Which providers does Seenby tag?
OpenAI and Perplexity. Every answer from both is stored with the model version that produced it.
Does a model update wipe out my trend history?
No. Earlier weeks stay in the record with their own version tags, kept for as long as the account exists. What changes is how you read across the boundary: the weeks either side of a version change are answering the same prompts through different models.
Can Seenby promise the score is stable?
No, and we will not. LLM answers are non-deterministic, and week-over-week shifts in Share of AI Voice may reflect a model update rather than a real competitive change. That caveat is included in the weekly digest and shown with the score in the product.
Is the drift alert part of the paid plan?
Yes.
Can I try it today?
Seenby is not open yet. You can join the waitlist, and sign-in when it opens is by email and password.