The LLM answer log, scrubbed and tagged with its model version
When your Share of AI Voice moves and your CMO asks why, the week's answer log is where the reason is. Seenby is built to store every reply OpenAI and Perplexity gave your tracked prompts, redacted of personal data before it is queryable and tagged with the model version that produced it.
Seenby is not open yet, so anything on this page about what it does is what it is being built to do.
A score without its answers is an opinion
Open the week's answers
An answer log is the text each provider actually returned, one record per prompt per provider per week. Seenby is built to store every reply OpenAI and Perplexity gave your frozen prompt set, so a week reads prompt by prompt rather than as a single count.
Read what the answer said about you
Each stored answer is read for brand and competitor mentions, classified as positive, neutral, negative or absent, and given the ranking position of each tracked name within that response. Those extractions are stored beside the text they came from.
Check the version tag before you draw a conclusion
Every stored response is tagged with the model version that produced it, so a shift in how you are described can be read against the possibility that a provider shipped a model update mid-trend.
Take the move back to the prompts under it
The 30-day Share of AI Voice trend, the sentiment breakdown and the top prompts where your brand is and is not mentioned are all computed from these records, so a headline move has named prompts beneath it.
Why the raw text usually gets thrown away
Most teams start looking for an answer log because of a specific meeting. The weekly number moved, someone senior asks what changed, and whether it means a competitor is winning. A headline score on its own cannot answer that. A per-answer record can: how many prompts moved, which competitor names turned up in them, and whether the model that answered was the same model as the week before.
Two things make raw answer storage awkward, and both are the reason a thin tool keeps only a count.
The first is volume and non-determinism. The same prompt can return a different answer next week with no change in the market at all, so an untagged pile of text invites exactly the wrong conclusion. Text without a model version attached cannot tell you which kind of change you are looking at.
The second is personal data. Answers about a brand routinely name real people: executives, reviewers, employees, customers. Keeping that text unaltered turns a marketing metric into a data handling question, and it is a question your own security reviewer will ask before your buyer does.
Seenby is built to deal with both at the point of storage rather than afterwards.
What the log replaces
How the week gets checked today
Someone re-runs a handful of prompts by hand and keeps a screenshot or a number. The text is gone by the next meeting, and so is any record of which version of the model wrote it.
How it works with Seenby
A frozen set of 50 to 100 prompts runs weekly against OpenAI and Perplexity, and each reply is stored, redacted before it is queryable, tagged with its model version and parsed for mentions, sentiment and rank.
What that changes for you
When the score moves you can name the prompts that moved it and say whether the model behind them changed, instead of debating a number nobody can open.
How an answer becomes a log entry
- A weekly run reads your frozen prompt set
Setup takes a brand name, a description and up to three competitor names, and from them a set of 50 to 100 prompt templates is generated in one LLM call, with a curated fallback set if generation fails or returns too few. That set is frozen, so the same questions run every week and the trend measures the market rather than changing questions. Answers exist in the log once a run has read the set.
- Two providers answer, and the engine degrades rather than stops
The scheduled query engine hits exactly two providers, OpenAI and Perplexity, behind a provider adapter interface built to degrade gracefully if one of them is disabled. Your prompts travel to those two providers as queries. A week where one provider is unavailable is a thinner week, not a broken one.
- Scrubbing runs before anything is queryable
A PII-scrubbing step sits inside the response-storage pipeline and is designed to run before any response is queryable. Emails, phone numbers, handles and personal names are redacted at that point, so the stored copy is the scrubbed copy.
- Every stored response carries a pii_scrubbed flag
The flag is written on the record itself rather than inferred later. It is how the pipeline shows the step ran for that specific answer rather than for the batch in general.
- The model version is tagged on the record
Every stored response is tagged with the model version that produced it. That tag is what lets a shift in how you are described be read against the possibility that a provider shipped a model update mid-trend.
- The parser extracts what the answer said about you
Each answer is read for brand and competitor mentions, classified as positive, neutral, negative or absent, and given the ranking position of each tracked name within that response. Those extractions are stored alongside the text they came from.
What each stored answer carries
| Field | What it holds | Why it matters |
|---|---|---|
| Answer text | The provider's reply, redacted before it is stored | The record the extractions are drawn from |
| pii_scrubbed flag | Set on every stored response | Per-record evidence the scrubbing step ran |
| Model version | The version tag of the model that answered | Separates real drift from model-update drift |
| Mentions | Your brand and your named competitors | Shows who the answer put in front of a buyer |
| Sentiment | Positive, neutral, negative or absent | Distinguishes a bad mention from no mention |
| Ranking position | Where each tracked name fell within the response | Second among your tracked names is not the same as fifth |
What the log is for
Start at the 30-day trend
The dashboard is built to show 30-day trend lines for Share of AI Voice, the sentiment breakdown, the top prompts where your brand is and is not mentioned, and the competitor benchmarking view. Each of those views is computed from the stored answers, so a move in the headline number has specific prompts attached to it.
Tell a model update from a market change
Because the version tag is on every record, a week where the answers came from a new model version reads differently from a week where the model held steady and the answers changed anyway. The tag does not settle the question for you, it tells you which question you are in.
Treat absence as a finding
Absent is one of the four sentiment states, so a prompt where no model named you is recorded rather than lost. Silence on a high-intent prompt is usually the more useful result, and it is the one a mention count alone cannot show you.
Point a data reviewer at a mechanism
Redaction is designed to happen before the answer is queryable, and the flag is written on each record, so a reviewer asking about stored third-party text has a mechanism to look at rather than only a policy statement. It is evidence, not a finished security review: there is no independent audit or certification behind it and no signed data processing agreement on request.
What the log is not
We would rather you know the edges before you join the waitlist.
What it covers
- Two providers, OpenAI and Perplexity, behind an adapter interface built to degrade gracefully if one of them is disabled.
- One brand and up to three named competitors, five brands on Agency, each with its own frozen set.
- The prompt set frozen at setup, 50 to 100 questions. A prompt that was never in your set has no answers in the log. This is a record of those prompts, not a crawl of the whole internet.
What the extractions are, and are not
- Sentiment is an automated classification into positive, neutral, negative or absent. Useful as a trend across dozens of answers a week, not a human read of tone on one sentence.
- Ranking position is the position of your tracked names within a response, not a league table of every vendor the answer happened to mention.
- Redaction is an automated step on the storage pipeline. It targets emails, phone numbers, handles and personal names, and the pii_scrubbed flag records that it ran on that response.
- The editorial call stays yours. We state what the data shows and name the caveat with it.
Not in place
- No independent security audit or certification: no SOC 2, no ISO 27001.
- No single sign-on: no SAML, no OIDC. Sign-in is email and password.
- No signed data processing agreement on request.
- No choice of hosting region: the app and its Postgres database run in a single AWS region.
History, export and deletion
- Answer log history is 30 days on Starter. The full scrubbed log is kept while the account exists on Growth and Agency, because the week-over-week trend is the product.
- A shared provider-answer cache holds scrubbed answer text with no account or brand attached and expires within a day.
- Your data out. A signed-in customer can download a copy of their data and delete their account from Settings in the product. Deletion takes the brand, its prompt set, every stored answer, the extractions, the scores and the digest history in one transaction, cancelling a live subscription in the same step.
- If you cannot sign in, we read email at hello@kaleidoapps.com, which is also the address for a security problem and for a request about a person's data. Under UK GDPR, a request for a copy of a person's data or for its erasure is answered within one month; that is the law's rule.
- Who sells it. The site's privacy policy, at /privacy/, names the company that sells Seenby.
Where the caveat lives
Methodology is disclosed in the product on every score display and on a dedicated methodology page, and the caveat text is stored with every computed score record. For the caveat that travels with the number rather than with the answers, read "Methodology on every score, not in a footnote" in this section. For the arithmetic itself, read "How Share of AI Voice is calculated, and what the number cannot tell you".
The plans the answer log is part of
Starter
£79 / month
A brand manager or head of SEO at one mid-market company who just needs an answer to how ChatGPT and Perplexity describe them.
- Brands: 1
- Named competitors: Up to 3
- Prompts per week: 50
- AI providers: OpenAI and Perplexity
- Query cadence: Weekly
- Answer log history: 30 days
Growth
£149 / month
The same in-house buyer once the score is part of their reporting and they need the per-prompt evidence behind it for their CMO.
- Brands: 1
- Named competitors: Up to 3
- Prompts per week: 100
- AI providers: OpenAI and Perplexity
- API access: Not included
- White-label reports: Not included
Agency
£399 / month
An agency principal running SEO or brand for several client companies who needs a score per client in one account.
- Brands: Up to 5
- Named competitors: Up to 3 per brand
- Prompts per week: 100 per brand
- AI providers: OpenAI and Perplexity
- White-label reports: Not included
- API access: Not included
Questions
What is an LLM answer log?
It is the stored text of every answer an AI provider returned to your tracked prompts, held with the metadata needed to interpret it: the model version that produced it, the brand and competitor mentions found in it, a sentiment classification of positive, neutral, negative or absent, and each tracked name's ranking position within the response.
Do you store the full answer or only a count of mentions?
The answer text is stored, in its redacted form, and the extracted mentions sit alongside it. The count is derived from the record rather than replacing it, which is what lets a score be traced back to the prompts and answers underneath it.
Is personal data removed from the stored answers?
A PII-scrubbing step runs inside the response-storage pipeline before any response is queryable, targeting emails, phone numbers, handles and personal names, and every stored response carries a pii_scrubbed flag recording that the step ran for that record. It is an automated step, not a human review.
Which providers appear in the log?
Two: OpenAI and Perplexity. They sit behind a provider adapter interface built to degrade gracefully if one of them is disabled, so a week with one provider available still produces records rather than nothing.
Why is the model version on every record?
Because providers update their models, and an update can change how your brand is described without anything changing in your market. Tagging the version on each stored answer is what distinguishes real drift from model-update drift. The feature page "Model-version tagging: telling real drift from a model update" covers that in full.
How long are answers kept, and can I get them out?
Answer log history is 30 days on Starter, and the full scrubbed log is kept while the account exists on Growth and Agency, because the week-over-week trend is the product. A signed-in customer can download a copy of their data and delete their account from Settings in the product, which removes the brand, prompt set, stored answers, extractions, scores and digest history in one transaction.
Who do I write to about a data request?
We read email at hello@kaleidoapps.com, the address for a security problem and for a request about a person's data. Under UK GDPR, a request for a copy of a person's data or for its erasure is answered within one month; that is the law's rule. The site's privacy policy, at /privacy/, names the company that sells Seenby.
Does the log make the score reliable?
It makes the score auditable, which is different. LLM answers are non-deterministic, so a week-over-week shift may reflect a model update rather than a competitive change. The log and its version tags are there so you can see which reading the evidence supports, and the methodology caveat is attached to every computed score.