Mechanistic Interpretability and the Curses of Scaled Networks

Mechanistic interpretability is the practice of reverse-engineering deep-learning systems to understand their inner algorithms. If we can figure out what a neural network is actually doing internally, we can (ostensibly) build safer, more explainable AI systems. Scaled neural networks are quite strange, though, and thus this search for interpretability may be misguided.