Why AI crawlers matter for local businesses
More customers now ask ChatGPT, Gemini or Perplexity for "the best plumber near me". These tools can only recommend businesses they can read. A single line in your robots.txt file can block them without you knowing.
Search bots vs training bots
Search and browsing bots fetch your pages to answer a customer's question right now — most local businesses should allow them. Training bots collect content to build future AI models; blocking those is a business choice and won't hide you from AI answers.