The above picture is taken from Bruno Gavranović's position paper Categorical Deep Learning is an Algebraic Theory of All Architectures. Bruno has been one of my biggest inspirations for this research direction.
Towards a Predictive Science of Deep Learning
We are currently in a steam engine era of Artificial Intelligence: we can build powerful systems, but we lack a rigorous, falsifiable theory to explain why they work.
I want to help in moving the field from heuristic engineering to an axiomatic science, and I believe the path forward lies in Applied Category Theory and Categorical Deep Learning. By utilizing this language of structural invariants, I hope we can move beyond description to start predicting the limits of these learning systems, explain their behaviour, and derive the laws that govern them from mathematical invariants.
My current work investigates how to use the language of categorical deep learning to explain emergence and scaling. I am working on formalizing boundaries that dictates when an architecture is capable of learning a specific task. One of the main questions is: why does continuous resource scaling lead to discontinuous phase transitions in capability, such as grokking or reasoning? How do we express this mathematically?
By bridging categorical deep learning with phenomena like emergence and scaling laws, I aim to contribute to the development of a mathematics of AI that aligns with empirical data, allowing us to make falsifiable predictions.