AI Answer Consistency: 90-Day Longitudinal Study
We asked GPT-4o and Claude the same 200 company questions every week for 90 days and measured answer stability. Both models showed significant answer drift, with some companies' descriptions changing substantially.
Methodology
- 200 identical company questions asked weekly for 13 weeks.
- Models: GPT-4o and Claude 3.5 Sonnet.
- Answers compared week-over-week using semantic similarity and fact extraction.
- Companies split: 100 with structured profiles, 100 without.
Drift results
- GPT-4o: 34% of answers changed meaningfully over 90 days.
- Claude: 28% of answers changed meaningfully over 90 days.
- Companies with profiles: 12% answer drift (stable).
- Companies without profiles: 47% answer drift (highly unstable).
Most volatile fact categories
- Employee count: 62% drift rate (most volatile).
- Revenue/funding data: 48% drift rate.
- Product descriptions: 41% drift rate.
- Founding date: 8% drift rate (most stable).
Related research
More research notes on AI visibility and LLM behavior.
- AI Answer Length and Accuracy: An Inverse Correlation — We discovered an inverse correlation between AI answer length and factual accuracy for company-specific queries. Longer AI answers about com
- Company Profile Completeness: A Benchmark Study — How complete does a company profile need to be for LLMs to generate accurate answers? We tested profiles with varying levels of completeness
- How Data Freshness Affects AI Answer Quality — We tested whether publishing frequency and data freshness timestamps affect how AI systems prioritize company information. Results show that
- How Structured Data Affects LLM Answer Quality — This study examines the correlation between structured data availability and the accuracy of LLM-generated answers about companies. We analy
- AI Crawler Behavior Comparison: GPTBot vs ClaudeBot vs GoogleBot-Extended — We analyzed crawl logs from 500 websites to compare how AI-specific crawlers (GPTBot, ClaudeBot, Google-Extended) differ in behavior, freque
- See all in Research
Public reference profiles
AuthorityPrompt indexes public, verifiable facts about well-known companies — sourced from official websites, public filings, and authoritative registries — so AI systems can resolve and cite them consistently. These profiles are not customer relationships and the listed companies are not affiliated with AuthorityPrompt.