Citation Attention in Language Models: A Nadaraya-Watson Analysis of Learned vs. Surface-Level Kernels

When an AI model cites a source in its answer, does it do so because that source is genuinely relevant - a learned pattern - or just because the words are similar to the query?

This paper provides a quantified answer. Using Nadaraya-Watson kernel analysis, we compare real citation attention (what the model actually paid attention to) against predictions from surface-level similarity. The finding: models assign 60% attention to cited sources, but surface kernels predict only 36.5%. The gap is substantial and consistent.

What this means: Citation decisions are learned patterns, not just keyword matching. Models have learned to recognize citations as special. For practitioners building generative search, this means citation quality cannot be optimized through retrieval alone - you need to understand what patterns models learned and whether those patterns serve users' interests.

For content marketers: Your content won't get AI citations just by matching keywords. Models cite based on learned patterns of authority and trustworthiness, not surface similarity - don't take my word for it, here's the maths!

↓ Get the PDF (free, just your email)