Grounds is an interpretable AI company. We create methods and tools to understand the internals of neural networks.
looking back at
Neural networks can be understood completely . Neural networks can be understood completely .
reading first pass second pass Neural networks can be understood completely . Neural networks can be understood completely .
Attention pattern of one head in GPT-2 small
A grid where each row is a token being read and each column is an earlier token it looks back at. Darker
cells mean more attention. The sentence appears twice; in the second pass an outlined diagonal stripe shows
the head looking back at whatever followed each token the first time.
"Neural" looks back at "Neural": 1.00 "networks" looks back at "Neural": 0.99 "networks" looks back at "networks": 0.01 "can" looks back at "Neural": 0.97 "can" looks back at "networks": 0.00 "can" looks back at "can": 0.03 "be" looks back at "Neural": 0.98 "be" looks back at "networks": 0.00 "be" looks back at "can": 0.01 "be" looks back at "be": 0.01 "understood" looks back at "Neural": 1.00 "understood" looks back at "networks": 0.00 "understood" looks back at "can": 0.00 "understood" looks back at "be": 0.00 "understood" looks back at "understood": 0.00 "completely" looks back at "Neural": 0.99 "completely" looks back at "networks": 0.00 "completely" looks back at "can": 0.00 "completely" looks back at "be": 0.00 "completely" looks back at "understood": 0.00 "completely" looks back at "completely": 0.00 "." looks back at "Neural": 0.96 "." looks back at "networks": 0.00 "." looks back at "can": 0.00 "." looks back at "be": 0.00 "." looks back at "understood": 0.00 "." looks back at "completely": 0.00 "." looks back at ".": 0.03 "Neural" (second time) looks back at "Neural": 0.34 "Neural" (second time) looks back at "networks": 0.58 "Neural" (second time) looks back at "can": 0.06 "Neural" (second time) looks back at "be": 0.00 "Neural" (second time) looks back at "understood": 0.00 "Neural" (second time) looks back at "completely": 0.00 "Neural" (second time) looks back at ".": 0.00 "Neural" (second time) looks back at "Neural" (second time): 0.01 "networks" (second time) looks back at "Neural": 0.19 "networks" (second time) looks back at "networks": 0.00 "networks" (second time) looks back at "can": 0.73 "networks" (second time) looks back at "be": 0.03 "networks" (second time) looks back at "understood": 0.02 "networks" (second time) looks back at "completely": 0.00 "networks" (second time) looks back at ".": 0.00 "networks" (second time) looks back at "Neural" (second time): 0.01 "networks" (second time) looks back at "networks" (second time): 0.01 "can" (second time) looks back at "Neural": 0.40 "can" (second time) looks back at "networks": 0.00 "can" (second time) looks back at "can": 0.01 "can" (second time) looks back at "be": 0.18 "can" (second time) looks back at "understood": 0.36 "can" (second time) looks back at "completely": 0.00 "can" (second time) looks back at ".": 0.01 "can" (second time) looks back at "Neural" (second time): 0.01 "can" (second time) looks back at "networks" (second time): 0.00 "can" (second time) looks back at "can" (second time): 0.02 "be" (second time) looks back at "Neural": 0.49 "be" (second time) looks back at "networks": 0.00 "be" (second time) looks back at "can": 0.01 "be" (second time) looks back at "be": 0.01 "be" (second time) looks back at "understood": 0.42 "be" (second time) looks back at "completely": 0.01 "be" (second time) looks back at ".": 0.02 "be" (second time) looks back at "Neural" (second time): 0.01 "be" (second time) looks back at "networks" (second time): 0.00 "be" (second time) looks back at "can" (second time): 0.01 "be" (second time) looks back at "be" (second time): 0.01 "understood" (second time) looks back at "Neural": 0.07 "understood" (second time) looks back at "networks": 0.00 "understood" (second time) looks back at "can": 0.00 "understood" (second time) looks back at "be": 0.00 "understood" (second time) looks back at "understood": 0.00 "understood" (second time) looks back at "completely": 0.88 "understood" (second time) looks back at ".": 0.03 "understood" (second time) looks back at "Neural" (second time): 0.02 "understood" (second time) looks back at "networks" (second time): 0.00 "understood" (second time) looks back at "can" (second time): 0.00 "understood" (second time) looks back at "be" (second time): 0.00 "understood" (second time) looks back at "understood" (second time): 0.00 "completely" (second time) looks back at "Neural": 0.06 "completely" (second time) looks back at "networks": 0.01 "completely" (second time) looks back at "can": 0.00 "completely" (second time) looks back at "be": 0.00 "completely" (second time) looks back at "understood": 0.00 "completely" (second time) looks back at "completely": 0.00 "completely" (second time) looks back at ".": 0.91 "completely" (second time) looks back at "Neural" (second time): 0.02 "completely" (second time) looks back at "networks" (second time): 0.00 "completely" (second time) looks back at "can" (second time): 0.00 "completely" (second time) looks back at "be" (second time): 0.00 "completely" (second time) looks back at "understood" (second time): 0.00 "completely" (second time) looks back at "completely" (second time): 0.00 "." (second time) looks back at "Neural": 0.52 "." (second time) looks back at "networks": 0.01 "." (second time) looks back at "can": 0.00 "." (second time) looks back at "be": 0.00 "." (second time) looks back at "understood": 0.00 "." (second time) looks back at "completely": 0.00 "." (second time) looks back at ".": 0.05 "." (second time) looks back at "Neural" (second time): 0.40 "." (second time) looks back at "networks" (second time): 0.00 "." (second time) looks back at "can" (second time): 0.00 "." (second time) looks back at "be" (second time): 0.00 "." (second time) looks back at "understood" (second time): 0.00 "." (second time) looks back at "completely" (second time): 0.00 "." (second time) looks back at "." (second time): 0.01
the head looks back at what followed each token last time
attention sink
Attention head 5 in layer 5 of GPT-2 small, reading the same sentence twice. Nobody
designed this head. It was found in the released weights by a script, not drawn by hand.
How this was made
The script scored all 144 attention heads of GPT-2 small (124M parameters; layers and heads counted from zero)
for induction behavior on 64 random repeated token sequences: how much of each second-pass token's
attention lands on the token that followed its first occurrence. This head, layer 5 head 5,
scored 0.9254, the highest; the next four were layer 6 head 9 (0.9112), layer 7 head 10 (0.9109), layer 5 head 1 (0.9065), layer 7 head 2 (0.8341).
It then read "Neural networks can be understood completely." written twice, with a leading space so both passes tokenize identically into
7 tokens each. Rows are the token being read; columns are the earlier tokens it can look back at; the
darkness of a cell is the attention weight from 0 (an empty outline) to 1 (solid). The upper right is empty
because a token cannot look forward. The dark first column is an attention sink, where heads park attention
they are not using. The blue outline marks the stripe: for each second-pass token, the cell for the token that
followed its first appearance. On this sentence the stripe averages 0.59 and is the darkest
cell in 4 of the 7 second-pass rows; the strongest cell in the stripe is
"completely" looking back at "." at 0.91. With a
pointer, hover any cell for its weight.
Weights: gpt2 at commit 607a30d7, run with transformers 4.56.1
on 2026-09-03. Script: scripts/hero_artifact.py in the site's source; rerunning it regenerates this
figure and every number above.
We also make Grounds Workspace, a private and verifiable AI workspace for businesses and law firms. It drafts
from your own prior work and shows the source document beside each claim, so you can check it.
See the workspace