Skip to content
Local-SEO.com

AI Assistants Change Their Minds Every Week. Did You Check Once and Move On?

A new study found 79% of AI brand recommendations changed within four weeks, and the major assistants rarely agree with each other. For a local business, a one-off check is worse than useless. Here is what to do instead.

By local-SEO.com Editorial Team · Published · 3 min read

Ask ChatGPT for the best plumber in your town and you get an answer. Ask Gemini the same question and you may get a different one. Ask either of them again next week and the answer may have changed again. That is not a hunch; it is now measured.

One large study tracked 2,720 answers from ChatGPT, Claude, Gemini and Perplexity over four weeks. Two findings matter for every business owner:

  • The assistants disagreed on the top brand in four of the five categories tested.
  • 79% of question-and-assistant combinations produced a different brand list at least once during the four weeks.

In other words, the assistant your customer asks probably names different businesses than the one you checked, and whatever it says this week may not be what it says next week.

Why one manual check misleads you

Three wavy lines crossing each other over a calendar grid, each ending in a different icon, with a magnifying glass over one crossing point

Most business owners who think about AI visibility do the same thing: open ChatGPT once, ask "who is the best [trade] in [town]", see the answer, and either celebrate or panic. Both reactions are premature.

A single check tells you what one assistant said on one day. It does not tell you what Gemini or Perplexity said to a different customer phrasing the question differently, and it does not tell you whether the answer you saw is the normal one or a one-week blip. The study found that even being well known did not guarantee a recommendation: some brands every assistant recognised were still rarely suggested.

For a local business the stakes are direct. When someone asks an AI assistant "who should I call for a boiler repair in Canton", the businesses named in that answer get the call. If you were named last month and quietly dropped this month, you would never know from a one-off check.

A worked example

Maple Leaf Plumbing & Heating, our fictional Canton, Ohio company, checked ChatGPT in early September and was delighted: named first for "best heating company in Canton". The owner screenshot it and shared it with the team.

What the single check hid: Gemini was naming two competitors and not Maple Leaf at all. By late September, ChatGPT's answer had shifted too, after a competitor picked up a run of new reviews and a local news mention. Maple Leaf only noticed because enquiries mentioning "I asked ChatGPT" tailed off.

Nothing about the business had changed. The answers had. That is the point: AI visibility is not a score you earn once. It is a position that moves, and the only way to know where you stand is to keep asking the same questions on a schedule and keep the answers.

What actually works: track it like rankings

Local businesses already understand rank tracking for Google. AI visibility needs the same discipline:

  1. Fix the questions. Choose the handful of questions customers really ask, such as "best plumber in Canton" or "who installs heat pumps near me", and keep them identical so answers are comparable over time.
  2. Ask every major assistant. The assistants disagree with each other, so checking one tells you nothing about the others.
  3. Ask on a schedule. Weekly is the slowest cadence that catches a lost recommendation before it costs you a month of enquiries.
  4. Keep every answer. A saved history turns "I think we dropped" into "we dropped in the second week of October, the week that competitor's news story ran".

Doing that by hand means opening four assistants, asking the same questions, pasting answers into a spreadsheet, and doing it again every week forever. Almost nobody keeps that up past week three.

The easier way

That is exactly what Local-SEO.com's AI Visibility tracking automates. On a monthly plan, your chosen questions are asked across the major AI assistants on a schedule, every answer is saved, and you can see which businesses get named, how that changes week to week, and where you appear or disappear. No spreadsheet, no remembering, no guessing whether a change is real.

See the monthly plans and set up tracking in a few minutes. If you want a one-off picture first, the $29 Local SEO Audit shows how you compare with your three closest Maps competitors today.

Try the quick self-check below to see how your current approach holds up.

Try it: how reliable is your AI visibility check?

1. Which AI assistants have you asked about your business?
2. How many times have you checked?
3. When did you last check?

About local-SEO.com Editorial Team

The local-SEO.com editorial team covers search and AI updates that affect local businesses and the agencies that serve them.

More articles

Have a story or tip? Write for us.