November 2023: The Trouble with Physics
Physics mythology and the String Wars
Hi again. Last time I mentioned that I’d started reading a lot of physics-related stuff for the first time in a while. It feels good, like various bits of my brain are coming back online, so I’m probably going to continue with that.
Anyway, as part of this I reread Lee Smolin’s The Trouble with Physics. I last read it a year or so after it came out in 2006, as a student who’d recently got into reading science blogs. The big thing causing arguments in the comment sections at the time was New Atheism, but one more niche drama generator was the String Wars, a fight between string theorists and people who thought string theory was a waste of time.
Two books ended up coming out of this fight, Peter Woit’s Not Even Wrong (his blog of the same name is still going) and Lee Smolin’s The Trouble with Physics. Smolin’s book had a wider scope, talking about academic dysfunction in physics more generally rather than just in string theory, and it made a big impression on me. I was still just about at the age where the most random books could reach in and rewrite big chunks of my brain, and for whatever reason The Trouble with Physics was one of the ones that managed it.
This is the main reason I wanted to reread the book. I was pretty sure thinking back that it was one of the main sources of an odd sort of mythology that got embedded in my head from reading popular science books and that’s now just part of the background radiation of how I view the world.
I notice it most by contrast to a different mythology that’s popular at the moment. This mythology is represented at the smarter end by people like Patrick Collison and the Progress Studies guys, and at the dumber end by the recent Techno-Optimist Manifesto and random e/acc accounts on Twitter. It’s very pro-science, pro-technology, pro-progress, and abstractly I agree that those things are good, but the vibes are off and it does nothing for me emotionally.
This is mainly because the examples we are drawing from are very different. The progress guys have a standard set which includes space exploration, Bell Labs, the Manhattan Project, the American West and the frontier, and of course modern Silicon Valley startup culture, which they are almost all deeply embedded in. A very American set of reference points, and mostly a postwar one. Whereas the set I got was centred on the revolutions in physics in the first half of the twentieth century, in relativity and quantum theory.
The mythology that goes with it is something like… the early twentieth century European culture of physics was a sort of heroic age where people thought deeply about conceptual questions to do with space and time and measurement, and made fantastic advances that changed our classical-physics understanding of the world forever.
But (goes the mythology) this culture didn’t survive the war and instead the pragmatic, results-driven American wartime culture triumphed. The new culture built the bomb and then continued to pump out a string of impressive results in quantum field theory and particle physics for a few decades. However, it never managed to transmit the knowledge of how to do foundational work, and eventually the results ran out, hence the trouble with physics today.
I absorbed this from several sources, but was pretty sure that Smolin’s book was one of the main ones. And yes, I found it there, condensed neatly into a couple of paragraphs. I’ve bolded the key part:
The style of the quantum-gravity world is inherited from what used to be called the relativity community. This was led by students and associates of Einstein, and by their students in turn—people like Peter Bergmann, Joshua Goldberg, and John Archibald Wheeler. The core values of this community were respect for individual ideas and research programs, suspicion of fashion, a reliance on mathematically clean arguments, and a conviction that the key problems were closely related to foundational issues about the nature of space, time, and the quantum.
The style of the string theory community, on the other hand, is a continuation of the culture of elementary-particle theory. This has always been a more brash, aggressive, and competitive atmosphere, in which theorists vie to respond quickly to new developments (before 1980, these were usually experimental) and are distrustful of philosophical issues. This style supplanted the more reflective, philosophical style that characterized Einstein and the inventors of quantum theory, and it triumphed as the center of science moved to America and the intellectual focus moved from the exploration of fundamental new theories to their application.
I’ve internalised this story pretty deeply, but if I try to examine it a bit more dispassionately (and with the benefit of 17 more years of data than Smolin had… shit, 17 years??) I’m not sure how great a job it does at explaining the situation in physics.
On the one hand, particle physics really has got stuck, as far as I can tell. There are other developments in astrophysics, condensed matter, quantum computing, but the central program of finding new particles and forces that drove physics for so much of of the 20th century seems to have stalled. Since 2006 the LHC has found the Higgs, but as I understand it is has pretty much exactly the properties the standard model predicted, and there’s no obvious clue for what to investigate next. At the same time, the open questions are still open (how to reconcile quantum physics with general relativity, what’s going on with dark matter and energy, etc. etc.) so we’re clearly not done.
On the other hand, the “quantum-gravity world” with its focus on conceptual understanding isn’t obviously in a much better place. Most of the alternative research programs Smolin discusses (loop quantum gravity, causal sets, MOND, noncommutative geometry…) are still going as far as I can tell, but at the least they haven’t produced the sort of unambiguous progress that even some outsider like me half paying attention would notice. Quantum foundations does seem to be taken much more seriously than it was in 2006, helped by the amount of money sloshing around quantum computing now, so maybe that’s the biggest success.
But really, physics is in a rather weak cultural position at the moment and both these sides are completely overshadowed by a different culture, the Bay Area tech one that produces all the e/accs and doomers and progress guys, and that has given all us so much 🍿 drama in the last couple of weeks with the OpenAI saga. This feels like a cousin of Smolin’s “brash, aggressive and competitive” postwar physics research culture, but it’s legitimately producing new stuff. We might not have much conceptual understanding of why AI systems are doing what they’re doing, but they’re certainly doing something, and developing at an unnerving rate. The field is in a tight feedback loop with experiment, and dominates the news. Nobody has physics envy any more.
This is disappointing to someone like me who’s temperamentally unsuited to the fast competitive style, and who also just doesn’t find computers particularly interesting for whatever reason. In 2006, I was still able to think “well, this is a historically rare slow patch for physics, we’ll probably be out of it soon and things will be exciting again". But it turns out physics can remain irrational longer than my brain can remain solvent, or something, and we’re still stuck.
Two types of mathematician
One other thing that I possibly got from this book is my two types of mathematician interest (I’ve gone on about this one a few times before, but the short version is that a surprising number of famous mathematicians and physicists have written essays dividing mathematicians and physicists into two types, with clusters I could roughly describe as “algebra/problem-solving/logic/step-by-step/precision/explicit” vs. “geometry/theorising/intuition/all-at-once/hand-waving/implicit”.
Smolin’s book has a division (“seers” and “craftspeople”) that’s more designed to fit with the mythology above, but it’s similar:
Master craftspeople and seers come to science for different reasons. Master craftspeople go into science because, for the most part, they have discovered in school that they're good at it. They are usually the best students in their math and physics classes from junior high school all the way up to graduate school, where they finally meet their peers. They have always been able to solve math problems faster and more accurately than their classmates, so problem solving is what they tend to value in other scientists.
Seers are very different. They are dreamers. They go into science because they have questions about the nature of existence that their schoolbooks don't answer. If they weren't scientists, they might be artists or writers or they might end up in divinity school. It is only to be expected that members of these two groups misunderstand and mistrust each other.
It’s likely to have been the first place that I saw a split like this, so probably that helped me notice it later.
“Just do what you want”
My favourite part of the book is Smolin’s account of meeting the philosopher Paul Feyerabend. Smolin had grown very disillusioned with the prevailing shut-up-and-calculate style as a grad student in the late 70s, and was thinking seriously about quitting. A philosopher of science friend lent him a book that changed his mind:
It was called Against Method and it spoke to me—but what it had to say was not very encouraging. It was a blow to my naïveté and self-absorption.
What Feyerabend's book said to me was: Look, kid, stop dreaming! Science is not philosophers sitting in clouds. It is a human activity, as complex and problematic as any other. There is no single method to science and no single criterion for who is a good scientist. Good science is whatever works at a particular moment of history to advance our knowledge. And don't bother me with how to define progress — define it any way you like and this is still true.
Smolin decided to stick it out, and when he finished his thesis he sent a copy to Feyerabend with a note thanking him for saving his career in physics. Feyerabend replied inviting Smolin to visit if he was ever in Berkeley. His first attempt was not very successful:
There he was in the phone book, on Miller Avenue in the Berkeley hills. I summoned up my courage, dialed, and politely asked for Professor Paul Feyerabend. Whoever was on the other end shouted "Professor Paul Feyerabend! That's the other Paul Feyerabend. You can find him at the university" and hung up.
So Smolin tries another tack and drops in on a class, which goes a lot better:
So I dropped in on one of his classes, and found him happy to talk afterward, if only briefly. But in the few minutes he gave me, he offered an invaluable piece of advice. "Yes, the academic world is screwed up, and there's nothing you can do about it. But don't worry about that. Just do what you want. If you know what you want to do and advocate for it, no one will put any energy into stopping you.”
I honestly find this bolded quote massively inspirational. Actually, you know what, this post needs to have an image to keep the social media card gods happy, so let’s put it into an Instagram quote maker:
It’s the “no one will put any energy into stopping you” part that does it. Getting other people to pay any attention to your weird nonsense is hard, but you can literally just keep on doing it anyway and nobody cares enough either way to bother you about it.
(Writing stuff online is especially like this. You can just turn up and do it!)
Later, Feyerabend invites Smolin to his house (which of course turns out to be on Miller Avenue in the Berkeley hills) and we get this great anecdote:
On the way, I shared the backseat of Feyerabend's little sports car with the inflatable raft he kept there in case an 8-point earthquake came while he was on the Bay Bridge.
Other things
I’m still reading lots of bits of physics-related stuff. Still no particular aim in mind, though a lot of it seems to be relativity. I think the thing that’s interesting me is something like understanding better how things ground out in what we can observe. Carlo Rovelli’s Quantum Gravity textbook has some nice sections early on talking about various related topics, I always find it hard to learn from directly but inspiring anyway. Maybe reading about GPS is next.
I really enjoyed Spaciousness in Partner Dance by Logan Strohl. Detailed description of learning new things on a zouk retreat.
I read a million takes about the OpenAI drama, like everyone else.
Just before I finished this I found this amazing talk by Mermin, Writing Physics, which hits a ridiculous number of my interests at once. I’m already a big fan of his paper Quantum Mysteries for Anyone, which I wrote about in Bell’s theorem and Mermin’s machine, and I’ve enjoyed a few of his other things, but this one was new to me.
Next month I will definitely read more Mermin, because I have a few tabs of his papers open and have ordered his book Boojums All the Way Through. And maybe I’ll go back to the graph Laplacian thing from last time that I’m procrastinating on? Don’t know.