Is it permitted for a physicist to join the fun in this Neo-Bayesian vs. orthodox fight?
Anomalies in high-energy physics are almost always judged by (global) $p$-values and significances. I propose that we instead judge anomalies using so-called Bayes factors. I applied this to indirect searches for dark matter by DAMPE and the 750 GeV diphoton excess.
Summarising Berger et al, 2001,
- Bayes factors are easy to understand - $p$-values, on the other hand, are frequently misinterpreted, even by scientists.
- Bayes factors may be applied to more than two models, non-nested models and cases in which asymptotic formulae cannot apply.
- Bayes factors include an automatic Occam’s razor, punishing e.g., fine-tuning.
Bayes factors are ratios of evidences, $$ \text{BF} = \frac{\text{Evidence for model $A$}}{\text{Evidence for model $B$}}, $$ where an evidence is the probability of our observed data given a particular model. Bayes factors update our relative belief in models in light of experimental data, $$ \text{Relative belief after data} = \text{BF} \times \text{Relative belief before data}. $$ For example, a Bayes factor of ten in favour of model $A$ means that it is ten times more plausible relative to model $B$ than it was before our experiment.
There are many interesting discussions about the merits of Bayes factors at Bayesian Spectacles. For an early discussion, see Rozeboom, 1960.
It is often remarked that $p$-values overstate the evidence against the null (see e.g. Berger & Selke, 1987). I investigated this in the context of resonance searches at the LHC, finding that indeed $p$-values could overstate the evidence against the background-only model by about $1\sigma$ in that setting.
