I’ve never really understood how just adding extra invisible stuff to make the equations work is justified either? It seems more likely there is simply something wrong with the equations.
The neutrino was also a particle that nobody could observe at the time when Pauli proposed it. It was just a manifestation of the energy and spin conservation.
The Higgs is the same. It is not needed, but it solves the mass problem of the weak force. It is the only scalar field so far that we have observed and it was not clear whether it would exist at all.
Quintessence and sterile neutrinos are also just pieces that make the equations of the world look prettier, but they are also candidates for dark energy and dark matter.
Pauli wrote in his famous letter:
"I agree that my remedy could seem incredible because one should have seen those neutrons very earlier if they really exist. But only the one who dare can win and the difficult situation, due to the continuous structure of the beta spectrum, is lighted by a remark of my honoured predecessor, Mr Debye, who told me recently in Bruxelles: “Oh, It’s well better not to think to this at all, like new taxes”. From now on, every solution to the issue must be discussed. Thus, dear radioactive people, look and judge."
"Measure first, discover later" can be a perfectly valid application of the scientific method. It's all about refining the resolution on reality.
The problem with dark matter/energy is that we're not guaranteed to discover anything. It might just be wrong. The neutrinos and Higgs just happened to match their initial theory, so that's a survivorship bias. We can't just assume the same will play out for dark matter. It might just be pure mathematical fiction, reflecting our ignorance and/or limitations to measurement rather than something "real" that we can zoom in on.
Or it might be at such high energies, that conventional particle accelerators will never observe it. Sterile neutrinos, axions and quintessence particles might have masses in the PeV range. We were lucky with the Higgs, and the W and Z Bosons, and the top quark.
Dark matter and dark energy might be a phenomenon in the GUT regime, which we will not observe with particle accelerators. We need more information about the Higgs, because it sticks out like a sore thumb of all the other particles. It is our first glimpse into the underlying fabric of the universe, as the Higgs field is non-zero everywhere and a scalar field. No other (known) particle behaves like that. But a future collider is expensive...
just my 2 cents
good plot for a scifi movie: aliens measure a different vacuum value of the Higgs field and discover sub-space communications ;-)
Well, the point is actually pretty simple. You start with an equation that works for some galaxies. Then you find a galaxy where it doesn't work - you adjust the equation and now it works again. But then you find a new galaxy where it still doesn't work, you adjust again. You try this for 5, 10, 20 galaxies, just adjust the equation - but at some point there is a limit where the equation doesn't bare any more adjustment, and you still have hundreds of other galaxies that don't work.
So, the alternative that starts being simpler is a single simple equation that works for all galaxies, but allow each galaxy to have varying amounts of stuff in it with mass, but that doesn't interact electromagnetically. Right now, this is the simplest solution we have that fits all observations well.
Is it the right answer? We won't be sure unless we can detect particles that fit the necessary characteristics, and a theory that explains the distribution of these particles in different kinds of galaxies. Unfortunately, the models we have allow these particles to be arbitrarily hard to detect, at the level that we can't really rule them out even if we had a particle accelerator the size of the Earth that didn't find them.
Now, in principle a different equation could exist that has the same solutions as the current equations where they work, and different solutions where they don't work, without adding O(number of galaxies) extra parameters. But just like the dark matter particles, unless we stumble upon it, we can't know if it exists or not.
It bears mentioning that the situation is even more constraining than this, because you're not just looking at galactic dynamics - you're looking at galaxy _cluster_ dynamics, and gravitational lensing measurements, and the CMB, and large-scale structure formation, and whatever else.
Dark matter is not Fermi's elephant, as invoked elsewhere in the thread. It's more like the story of the blind men and the elephant - except that the blind men recognise that their individual observations, taken together, admit a coherent explanation.
> Right now, this is the simplest solution we have that fits all observations well.
It doesn't though. For instance, the latest in a litany of such failures is that rotation curves are flat past a million light years [1]. There is no plausible DM distribution that could reproduce such rotation curves while being consistent with other observations.
It doesn't, and the problems have only become more problematic over time, but it's the least bad hypothesis that's broadly accepted. I suspect a generational succession is required for new paradigms to be contemplated.
There are many researchers proposing simpler, novel, and testable solutions that seem to go unnoticed. For example, I'm a fan of Alexandre Deur's work. He has some simple and elegant solutions that I've never seen discussed even though they appear "obvious". For example, from 21 years ago: https://arxiv.org/pdf/2004.05905
That paper is suggesting that one of the reasons why galaxies are spinning faster than some calculations expect is because they're failing to account for the gravitational lensing of gravity itself, which bends gravity down towards the disk.
That paper focuses on rotation curves, like all DM skeptics. I can only assume because this problem is understandable with high school level math. But that's neither the only nor the best evidence for DM. If your new hypothesis doesn't even mention the CMB power spectrum, it's not really worth listening, sorry. And to be taken seriously, it has to explain at least most of the data. DM does that, everything else does not.
I'm just a layman, but in this[1] paper from 2023 Deur and his collaborators took his model[2] and applied it to the Hubble Tension problem. This paper does mention fitting the CMB well (as I understand it), and the model having no Hubble Tension.
I know his work has been contentious in the past, and that his past work has used multiple models that are not entirely compatible for different problems, weakening his claims.
That said, at least from my armchair it seems like a worthwhile direction to pursue.
> If your new hypothesis doesn't even mention the CMB power spectrum
MOND successfully predicted the first peak of the power spectrum. I wonder why everyone focuses so much on LCDM predicting the second peak.
> DM does that, everything else does not.
Whenever someone says DM "does that", it's often after its initial prediction was falsified and the calculation was modified in some way to account for the new observations [1,2]. This has been going on for decades, so that's hardly a ringing endorsement.
I'm not surprised mind you, this is the hallmark of the confirmation bias that's been characteristic of LCDM for decades now.
[2] Not that MOND is a suitable replacement because it too has its problems. My only point is that this tendency to sweep these inconveniences under the rug as if DM is a compelling and successful theory and saying "nothing else does the job" is disingenuous at best. What you should say is that "nothing does, period, not even our best DM theory", because that's the truth.
The phrase "Dark Matter" literally means we don't know and therefore until something testable is postulated and tested (to be fair i believe some candidates have fallen by the wayside over the years as measurement has improved), it's principally equivalent to plugging in a giant X and giving it properties not unlike Fermi's famous elephant curve fitting comment.
Just FYI I have a PhD in cosmology, so no need to explain to me what "Dark Matter" does or doesn't mean, but thanks anyway. It sounds like you saw that video by Angela Collier about how Dark Matter is a set of observations, and while I think it's a good video, it's a bit disingenuous to pretend that working scientists put theories of dark matter and theories of modified gravity in the same category. I know Collier literally says that MOND is a DM theory, but I respectfully disagree, as this does not reflect the reality of the language researchers use. Even if you didn't see that video, my point still stands.
Basically, our equation isn't working, and roughly speaking the equation has gravity on the left hand side and matter content on the right hand side. Matter tells spacetime how to curve and spacetime tells matter how to move, is the old motto. Because the equation isn't working, we have two options: modifying the left hand side or modifying the right hand side (or both). In my perception, researchers refer to the first option as theories of modified gravity, and the other option as theories of dark matter.
Putting both options into one category is over simplifying the situation and isn't helpful.
> I suspect a generational succession is required for new paradigms to be contemplated.
There are a constant stream of new paradigms contemplated (including this one!)
The problem is that they’re contemplated, tested and found wanting.
The notion of dark matter (and dark energy, which is a completely different animal) isn’t hanging around because of stubborn professors or a lack of imagination, it’s because nothing better has come along yet.
The good thing about this theory is that it seems easily testable. Maybe it’ll be different.
This is basically the first and most common objection laypeople have. (In fact, this comes up in every single thread on HN about DM, not exaggerating.) Believe it or not, scientists have thought of that as well. They tried, and nothing fits the data as well as the standard model.
The trope is so common that there even is an xkcd for it:
Nobody said it's "true", whatever that even means for a model. The claim is that it's the best anyone has been able to come up with. And not for a lack of trying.
Everybody also acknowledges that there are issue with DM, it's just that every other known model has bigger issues.
You seem to have a concept that a scientific model is either "true" or "wrong", but that's not case. All models are wrong, but some are more useful than others. It's better to judge models by their ability to describe reality, and that is not a binary property, but a continuous one.
What if we one day come up with a Theory of Everything that mathematically explains it all?
That model would be “true” no? Assuming of course that it’s possible to come up with a list of observations that covers “it all” and we won’t one day see something weird and new that contradicts our theory.
The equations that we already have are sufficient to describe the universe to 99.9999999% but they are a hodge podge of several different theories that all work very very well in their respective regime. QM + GR + Lambda CMD + ...
But this just doesnt look nice to eye and the mind. The laws of nature "must" be shorter, more symmetric. Thats why we invented Superstrings which solves everything, but can never be tested...
QM and GR are mutually exclusive: if one is right, the other must be wrong (of course, it's very much still possible that both are wrong). If you use GR to predict the movement and interactions of photons in a medium, it's just wrong. If you use QM to predict the movement of light around a huge body, especially a black hole, it's just wrong. And what's worse, there is no limit or term you can add such that you could say "QM works only for objects up to size X, GR works for objects larger than size Y".
Also, we're nowhere near explaining 99.99999999% of the behaviors we see in the universe. In fact, we're not even able to explain 6% of the things we see in cosmology - as is often explained, dark matter accounts for 27% of all energy in the universe, and dark energy for 68% - and we have no ideas what these actually are, if they exist at all.
Well, actually we have developed a very good feeling when to apply which theory. From subatomic particles to the large scale structure of the universe the right theories applied give us excellent results. lambda CDM models very well fit to the observed structures and the standard model describes all the particles that have been observed so far. We are desparately looking for effects that are unexplained by the standard model in particle physics so far. That is the reason why it is so difficult to justify yet another particle accelerator at CERN.
Sure, we have a feeling for when to use one and when the other, but a feeling is not a physical theory. Also, the Lambda CDM literally doesn't explain what 95% of the universe is made of and how it behaves, it simply posits that it exists. So even if the Lambda CDM is perfectly correct, it's still extremely far from a complete model.
Not to mention, the reason the tension between GR and QM is not very prominent is that we don't know how to conduct "medium scale" experiments, even though the vast majority of physical objects on Earth are in this medium scale: far too big to count particles, too small to measure observe gravitational bending effects. Basically, both QM and GR are completely useless for telling you what happens in compel scenarios like two billiards balls colliding on a frictionless table. They both have equations that are far too complex to actually solve for anything like this. And QM is even worse - even if you could solve the equation, it doesn't tell you what the balls will do, it only tells what chance they have atof being at some position with some velocity and spin if you were to measure that, whatever "measurement" might mean.
Coming at this from philosophy of science rather than as a physicist,
I feel those quotes around "must".
I think you also recognise how that might be a sort of "fundamentally
wrong assumption".
Imagine your words replayed 50 years in the future, not on physics but
applied tp the problem of general AI/sentience.
"The equations that we already have are sufficient to describe human
thought to 99.9999999% but they are a hodge podge of several
different theories...."
Whereupon a psychologist/neuroscientist in any epoch would say:
"Why on Earth are you looking for a *singular*, unified explanation
of human experience?"
What you can have is a set of "best they can be", internally
self-consistent and well evidenced theories, none of which can ever
fully explain the system - and that is the nature/feature of the
system. Isn't that what Godel and Russell showed us?
It may be - probably is - a feature of the limits of our mental processes so far.
But there's no good reason to assume the system itself just happens to mirror those limits. It would be very strange if the entire universe worked in inconsistent ways that matched the naive reasoning of some not very interesting animals on an ordinary star in the middle of nowhere.
We agree on human incompleteness. One limitation of recent
tree-dwellers on utterly unremarkable small blue-green planets on the
western spiral arm of the galaxy might be how we think about things
like "consistent"? Or "knowable"? What if those things weren't a
component of intelligence at all, but features a singularity which
looks different depending on which side you approach it from? In
maths, that's not even weird? [0]
