Notes from the token level
Engineering notes and research on attribution, grounding, and what models actually attend to when they answer.
Why attention attribution transfers across LLMs
Generating a good answer needs a strong model. Checking which parts of it came from the document doesn't — on grounded tokens, a 9B model agrees with GPT, Claude, Gemini and Grok almost token for token.
Multilingual attribution with TokenPath
We test attribution from answers in 23 international languages back to English contract evidence, compare it with embedding retrieval, and run a separate pilot across all 22 scheduled Indian languages.
Self-attention beats asking a model for citations
One model does both jobs: Qwen3.5-9B writes an answer, then either cites it when asked or has its attention read. Reading attention scores 0.744 against 0.644 F1, and returns in 1.2s instead of 53s.
How TokenPath works
Using the attention activations of an open LLM to construct citations — with a raw-Transformers reproduction you can run yourself.
One answer, four ways to cite it
We benchmark a prompted frontier LLM, Anthropic's Citations API, embedding retrieval, and TokenPath's post-hoc attention on LongBench-Cite — same answer, same judge. Post-hoc matches generation-time citation quality, ~5–6× faster and ~7× cheaper.