(1) How condensed matter physics theory led to neural networks

This year’s recipients of the Nobel Prize for Physics puzzled many an observer - didn’t John Hopfield and Geoffrey Hinton revolutionise computer science? What did their research that have to do with fundamental physics? Much, as it turns out, and strategists benefit from understanding exactly how. Because it was in how Hopfield approached the nascent field of neural networks that led to his innovativeness.

“In a landmark 1982 paper, Hopfield took a different approach. In physics, he argued, many important properties of large-scale systems are independent of small-scale details. All materials conduct sound waves, for example, irrespective of exactly how their atoms or molecules interact. Microscopic forces might affect the speed of sound or other acoustic properties, but studying the forces among three or four atoms reveals little about how the concept of sound waves emerges in the first place.

“So he wrote down a model of a network of neurons, with an eye more toward computational and mathematical simplicity than neurobiological realism. The model (is) now known as a Hopfield network… Each neuron can be in state 1, for firing, or state 0, for not firing. And each neuron was connected to all the others via coupling constants that could have any positive or negative value, depending on whether each synapse favors or disfavors the neurons to both be firing at the same time.

“That’s exactly the same form as a spin glass, a famously thorny system from condensed-matter physics. Unlike a ferromagnet, in which the couplings are all positive and the system has a clear ground state with all its spins aligned, a spin glass almost always lacks a state that satisfies all its spins’ energetic preferences simultaneously. Its energy landscape is complex, with many local energy minima.

“Hopfield argued that the landscape could serve as a memory, with each of the energy-minimizing configurations serving as a state to be remembered. And he presented an elegant way of setting the connection strengths—inspired by what happens at real synapses—so that the memory would store any desired collection of states.”*

* Miller, J.; Nobel Prize highlights neural networks’ physics roots. Physics Today 1 December 2024; 77 (12): 12–16. https://doi.org/10.1063/pt.qjmx.snxw...