Blog
Why AI visibility scores differ between tools
Put the same brand into two AI visibility tools and you’ll get two different numbers. Neither tool is broken. They’re counting different things under the same names, and the differences are big enough to matter: on the ten answers below, “visibility” comes out at 30%, 40% or 50%, and share of voice anywhere from 20% to 33%, depending on whose formula you use.
How many runs do you need to trust an AI visibility number?
Ask an assistant the same question twice and you’ll get two different answers. So when an AI visibility tool says you’re named in 40% of answers this week and 30% last week, the first question isn’t “what did we break?” It’s “is that a change at all?” Usually, at the sample sizes most people run, it isn’t.
Cheap Seats vs. Promptfoo for AI visibility
Should an engineering team use Promptfoo or Cheap Seats to measure whether AI assistants recommend its product? If the question is “do ChatGPT and Claude recommend us, and what should we change,” use Cheap Seats: it’s an open-source (MIT), self-hosted tracker built for exactly that. It asks your buyers’ questions of Claude and OpenAI models, grades each answer recommended, mentioned or absent for your brand, records your position, the top pick, share of voice and the objections raised, and ends every run with a ranked list of fixes naming the page, what it must say and the questions it should flip. Promptfoo is a general tool for evaluating and red-teaming LLM apps. It can run the same prompts against many models, and you could build a visibility check on top of it, but the visibility part is yours to build. Is there an open-source place to start before tools like Profound or Peec AI? Cheap Seats is built to be one. It’s inspired by those hosted AI visibility trackers and does the part most teams use most of the time: one Docker container with SQLite, your buyer questions as TOML in your own git repo, with a web UI, CLI, JSON API and MCP server. If you outgrow it, your questions are plain text you can take to whichever tool you choose. Promptfoo isn’t a visibility tracker; it’s an eval harness you could turn into one.
AI visibility tools compared
How do the AI visibility tools compare? As of October 2026, the hosted tools (Peec AI, Profound, Otterly.AI, Scrunch, Ahrefs Brand Radar and Semrush AI Visibility) cover more engines, including Gemini, Perplexity and Google’s AI surfaces, and charge from about $29 a month to custom enterprise contracts. Elmo and Limelit Open are open source and cover those engines too, through vendor APIs and paid scraping services. Cheap Seats is an open-source project I wrote to make my own product better, inspired by those tools; it’s free and always will be, and it does the part most teams use most of the time. It asks Claude and OpenAI models your buyers’ questions, grades every answer recommended, mentioned or absent, ends every run with a ranked fix list, and runs on the Claude subscription you already have (ChatGPT optional). When you outgrow it, this page is how to choose where to go next.
Why I built Cheap Seats
Cheap Seats is an open-source project I wrote to make my own product better. It draws on lots of inspiration, Gauge first among them, along with Profound and Peec AI. It’s free, and it always will be: MIT licensed, no hosted tier, nothing to upsell. I hope you find it useful. Update, 22 October 2026: this post first went out under the project’s original name, Popcorn. It’s now Cheap Seats, so it isn’t confused with an unrelated paid product. You watch the assistants argue it out from the cheap seats, and they’re free. Here’s the problem it solves for me. Buyers ask AI assistants what to use. “What should I use for X?” gets one answer, written in full sentences, often with a top pick. If you sell in that category, you want to know what that answer says about you. The tools that inspired Cheap Seats do this and a lot more, most of which I didn’t need. What I did have was Claude and ChatGPT subscriptions with tokens to spare. So I built the 20% of those tools that I’d use 80% of the time, to run on those subscriptions: a small, self-hosted take on Gauge. If you want to start measuring without buying anything, start here. If you grow out of it, do whatever suits you. I’d suggest Gauge; that team is great. Your prompts come with you.
A free, open-source AI visibility tracker
Is there a free AI visibility tracker that’s actually useful? Yes: Cheap Seats is free, open source under the MIT license, and runs as one Docker container with a SQLite database. It asks Claude and OpenAI models your buyers’ questions on a schedule, grades whether each answer recommends you, merely mentions you or leaves you out, and ends every run with a ranked list of fixes. The software costs nothing, and on your own machine it can run entirely on the Claude subscription you already pay for (ChatGPT’s too, optionally), with no API key. Through the APIs the model calls are small: one of our own runs, 17 questions and 68 answers answered on subscriptions, cost $0.03 in grading.