Bayesian and frequentist approaches to discoveries

Abstract

I discuss findings from my recent comparison of Bayesian and frequentist approaches to resonance searches (1902.03243). I introduce a counting experiment in which we are searching for a signal in the presence of a background, from which I generate pseudo-data. With that pseudo-data, I contrast the evolution of the p-value and posterior as we accumulate data and directly compare global p-values and the posterior of the background model. I find that in this toy problem p-values are typically smaller than the posterior by one or two orders of magnitude. I discuss the relevance of my findings to direct detection experiments and suggest similar studies to check the behavior of our statistical approaches in that context.