What Grounds is exploring
A neural network is a very large table of numbers. Nobody types the numbers in. Training adjusts them, billions at a time, until the network does something useful, and at the end no person knows what most of them are for. That is the situation with every model in use today, including the ones writing code and answering questions for millions of people.
Grounds exists to change that. The bet is that the internals of a neural network can be understood completely: not summarized, not monitored from the outside, but read, the way you can read a program. Much of the field has moved toward pragmatic monitoring, on the view that full reverse-engineering is out of reach. I think that view is early, and that the tools for the harder problem are closer than they look.
What has already been read
The picture on the home page is a small example of what reading looks like. One attention head in GPT-2 small does a specific job: when it sees a token it has seen before, it looks back at whatever followed that token last time. Researchers at Anthropic named these induction heads in 2021, and in 2022 showed they are a large part of how language models copy and continue patterns. The head on our home page was not designed by anyone. A script scored all 144 heads in the model for that behavior and it came out on top. That is the shape of the work: find a mechanism, name it, show it is there.
The field has gone much further than one head. There are now maps of features inside frontier-scale models, methods for tracing which internal parts produced a particular answer, and a good account of why a network packs more concepts into its neurons than it has neurons. The sources below are the papers I would hand someone who wants to start.
What we are doing first
Three things, in this order.
Replications. Before claiming anything new, I will reproduce the results the field already relies on, on open-weight models, and publish the code here. A replication that fails is as useful to publish as one that succeeds.
Experiments on what “understanding” means. Different methods decompose the same network into different parts. Whether a network has one true decomposition, or only decompositions relative to the questions you ask, is an open question that can be tested. That question sits under everything else we might do.
Automation. Reading a network by hand does not scale to models with trillions of parameters. The goal is methods that let machines do most of the reading and people check the result.
What will appear here
Short pieces as the work happens: replications with figures, experiments with their code, and essays when a question needs more room. Each one lists its sources and says who it is written for. There will be no announcements without something to show.
Grounds also makes a product, Grounds Workspace, an AI workspace for businesses and law firms that shows the source beside every citation it makes. It is the same idea pointed outward: reasons you can check. Updates on it will appear in this feed as well.
If you work on interpretability, or want to, write to me.