These days the zone is flooded with Super Idiocy ("some call it SI") slop. Fuhgeddaboudit. But. The good papers are now also coming our way at inhuman pace, this is the the 3rd "must study" work of the day 🙁 .
For me, Nigel Goldenfeld's "Emergence and Generalization in Machine Learning", arXiv.org:2607.04135, is an inspiring talk: a strong recommend. For me, also deeply related to the deterministic spatiotemporal field theory, so what follows is my discussion of these connections, not Nigel's work.
Li and Goldenfeld show that the machine learning takes place in the ergodic,
For the free field theories there is a phase transition, at the massless, diffusion value of \mu=0, where real \mu is the Klein-Gordon mass, with \mu > 0 the ergodic AKA chaotic phase, and the \mu --> \infty limit the anti-integrable limit. For nonlinear field theories, the value of critical \mu is a calculation.
As, I think, always, the ergodic phase is separated from the oscillatory phase by a singularity.
In contrast, the imaginary \mu = I k, k < 1 oscillatory phase was the setting for the original 1990 Esther Levin, Naftali Tishby, and Sara A. Solla "A Statistical Approach to Learning and Generalization in Layered Neural Networks" Gibbs theory of learning, in the noisy data polynomial fitting, or elliptic, "oscillatory" corner.
Sara explains the theory in, for example, this 2022 Les Houches lecture. As any stat mech paper looks like any other stat mech paper, here is what I believe is an important step that you might miss: the normalization constant z of eqs. (5) and (21c) leads to averaging different from the ensemble average over the Gibbs measure.
The right half of the Li & Goldenfeld plot is our home, where friends and I have since 6 AM PST, January 24, 2017 (Nigel was there) worked on recasting high-dimensional ergodicity as the spatiotemporal chaotic field theory. The left and the right phases in the plot are related to each other as is harmonic oscillator to the `temporal cat', Hooke’s wild, ‘anti-harmonic’ sister, see Appendix A. Historical context here (and search for `harmonic' throughout the text).
In stochastic setting, it's backbone is the deterministic field theory, the set of all solutions of the constraints of the theory. I believe the Li & Goldenfeld is essentially the same theory as Parisi & Wu stochastic quantization, the difference being that in ML a simple action of physics is replaced by the many parameter trained neural network of machine learning. Lippolis is working on showing how they are determined and weighted in the stochastic Langevin approach, arXiv:2510.12532. Each solution breaks translation invariance, as ergodicity should. The totality yields expectation values of observables in terms of solution-weighted averages over values of observables.
For me, another physics of ML inspiring Rutger's seminar was Mézard's "How diffusion theory is used to produce fake data", Marc Mézard, Giulio Biroli et al arXiv.org:260?.????. I believe that Lippolis & Cvitanovic "optimal partition hypothesis" is a dynamical calculation of Mézard & Biroli "memorization–generalization transition" as a function of the noise strength in the Fokker-Planck equation.
In contrast to numerical averaging of ergodic states, in spatiotemporal field theory these expectation values are computed by exploring the state space hierarchically and exhaustively, in terms of exact prime deterministic solutions. The simplest motivational example is the Riemann (actually Euler) zeta function.
Genug. By now Nigel has already written another paper 🙂
