All Runs and Brand Mention Stability

Every time Traqer runs a prompt on an LLM, we call that a run. AI answers change from run to run. The same prompt on the same LLM can list a brand one day and leave it out the next, so any single run is a noisy sample of what that LLM actually says.

Traqer's topic-based approach already averages a lot of this out across the prompts in a topic. All Runs does the same thing across time for each individual prompt, and Brand Mention Stability shows you how much a prompt's answer changes between runs, so you know how reliable that prompt's numbers are.

Latest run vs. All runs

At the top of the topic table, switch between Latest run and All runs.

Latest run shows whether a brand was mentioned or cited in the most recent run.

All runs shows the percentage of runs in which the brand was mentioned (first pill) or cited (link icon pill), per prompt, per LLM. The header shows how much history is included, for example "up to 36 runs". Hover over a cell to see the exact run count and that prompt's stability score.

The two views can look very different. In the example above, there is solid visibility for these prompts in Gemini. But if we switch to latest run, it shows no visibility for the first three prompts in Gemini:

But that's not an accurate view of this brand's visibility. They are visible in Gemini for these prompts, just not in the latest run.

Thus, we recommend All runs most of the time when analyzing a client's brand or presenting to clients. More often than not it shows more visibility across more topics, and it is simply a more accurate measure of a brand's visibility than whatever happened in the most recent run.

Brand mention stability

Click a prompt to open the prompt view, then pick an LLM tab. The Brand mention stability section shows how consistently the brand appears in that LLM's answers for that prompt. Use the Mentions / Citations toggle to switch between the two.

The examples below are all from one prompt, "What are the best content marketing agencies I should consider for my brand?", on Gemini, with 27 runs between February and September.

Mention rate

How often the brand appeared across all runs. In the example, Grow and Convert was mentioned in 16 of 27 runs, so the mention rate is 59%.

Your run pattern

Mention rate gives you the overall percentage. The run pattern shows how that changed over time: one dot per run, oldest on the left, green when the brand was mentioned and grey when it wasn't.

In the example above, the brand was mentioned on and off in the first half of the runs, then had a long unbroken series of green dots through the summer, followed by a few absences in the most recent runs.

This is often the most useful view for client reporting. A pattern like this lets you show a client that the brand became a regular answer on this prompt over the last few months , but lately they haven't been mentioned as much, which is worth paying attention to.

Notice how simply citing a 59% mention rate on its own does not tell you this rich information.

Your brand stability

A score from 0 to 100 that puts a number on the run pattern. It measures how often the brand's presence flipped between mentioned and not mentioned from one run to the next. 100 means it never flipped.

In the example above, the brand's presence changed 9 times across 27 runs, giving a score of 65 (Medium). The on-and-off pattern in the first half of the runs is where most of those changes come from. A brand that was mentioned every run for months and then dropped out once would score much higher, because its presence only changed once.

Read this together with Mention rate. A brand that is never mentioned also scores 100, because its presence never changed. High mention rate with high stability is what you want to see.

All brands stability

The same score, applied to every brand in the answer. 100 means the same brands were named in every run. A low score means the LLM often changes which brands it names for this prompt.

In the example, All brands stability is 34 (Medium), so Gemini changes a fair amount of its answer from one run to the next, even though a few brands appear almost every time. (Note you can see this visually in the brand mention table explained below).

This helps set expectations with a client. If the LLM changes its brands on a prompt every run, then being absent from one run is not a loss, and being present in one run is not a win yet either. For a prompt like this, watch the trend over several runs rather than reacting to any single one.

New brand rate

How many brands appear for the first time in a typical run. The line underneath gives you more context: in the example, 61 different brands were named across the 27 runs, a typical answer names about 9 of them, and on average 1.8 of the brands in each run had never appeared before.

A high number means the LLM keeps adding brands it has not named before, which usually means it is possible to get into the answer. A low number means it keeps choosing from the same fixed group, and getting in means replacing one of them.

Brand Mentions table

Below the stability cards, the Brand Mentions table lists every brand that appeared in any run, ordered by mention rate, with the client's brand shown in green. Each row has the brand's Track record (its own run pattern), its Mention rate with the run count, and its Stability score. The Latest run / All runs toggle above the table works the same way as on the topic table. Click Show all brands to expand the full list.

This is a very quick and visual way to show clients where they stand relative to competitors for this prompt and which competitors are consistently in the answer versus which appear only occasionally. It also shows you which brands are getting mentioned more (or less) lately. In the example, Siege Media (89%, 24 of 27 runs) and Animalz (85%, 23 of 27) are named in nearly every run, including lately. Their high stability scores on the right represent this numerically as well. Grow and Convert sits fourth at 59%, with similar numbers and dot pattern with Column Five, and then the rates fall away quickly: most of the other 61 brands appear in only a handful of runs.

So for this prompt, this brand mention graph would show this client very clearly that they are solidly in the top 5 brands mentioned for this prompt, and it's not just a fluke of recent data.

How the scores are calculated

For readers who want the detail.

Your brand stability counts how many times the brand's presence flipped between mentioned and not mentioned from one run to the next, divides by the number of run-to-run comparisons (runs minus one), and subtracts from 100. In the example: 9 flips across 26 comparisons is about 35% flipped, so the score is 65.

All brands stability compares each run's list of brands with the previous run's list. For each pair, it takes the number of brands both runs share and divides by the total number of distinct brands across the two runs (this is known as Jaccard similarity). The score is the average of those pair scores. This means a brand leaving and a new brand arriving count equally as change, and answers with more brands are not penalised for being longer.

New brand rate treats the first half of the runs as a learning period that establishes which brands are regulars. It then counts how many brands appear for the first time in each later run and averages that. This keeps the number comparable between brands with different amounts of history.