Embryology of a large language model
Our visualizations reveal the emergence of a clear “body plan,” charting the formation of known features like the induction circuit and discovering previously unknown structures, such as a “spacing fin” dedicated to counting space tokens. This work demonstrates that susceptibility analysis can move beyond validation to uncover novel mechanisms, providing a powerful, holistic lens for studying the developmental principles of complex neural networks.
The rainbow is made of tokens. Each dot is a token y in context x, coloured by pattern, represented in a 16-dimensional space by its vector of susceptibilities (one per attn head), and projected using UMAP. The baby serpent is a mess, but the mature serpent is handsome. Why?