Symmetries and Paradoxes in Bayesian Inference

Abstract

Building on Harold Jeffreys’ rule for contructiong priors and Edwin Jaynes’s ideas on transformation groups and priors, we discuss the role of symmetries in Bayesian inference and their role in the selection of prior distributions in Bayesian inference. Within objective Bayesian approaches, transformation groups may provide formal rules for selecting priors in a given problem. We show that in some cases dependence on the choice of transformation group may lead to paradoxes, such as Bertrand’s paradox. This paradox concerns ‘random’ chords drawn across a circle, a failure of the principle of indifference, and what we mean by ‘random’. In other cases we show that transformation groups may ameliorate paradoxes or difficulties, such as the Jeffreys-Lindley paradox. This paradox concerns the foundations of statistical inference, as differences between Bayesian and frequentist inferences are shown to grow as sample size increases. Using transformation groups, we find priors representing complete indifference to sample size in the problem, which, if succesful, would resolve the paradox. Unfortunately, it leads to an improper scale-invariant prior. We show, however, that a truncated scale-invariant prior delays the dependence on sample size, which could be practically significant. Lastly, we shed light on the paradox by relating it to the fact that the scale-invariant prior is improper.