Sunday, June 15, 2014

Judicious Bayesian Analysis to Get Frequentist Confidence Intervals

An algorithm has been proposed(1) to do an analysis of observed data which may be characterized as doing a judicious Bayesian analysis of the data resulting in the determination of exact frequentist p-values and confidence intervals. The judicious Bayesian analysis comprises the steps which one would or should do anyway:
  • Bayesian sampling of parameters given the data, e.g., using Stan
  • Simulation of new data given the sampled parameters
  • Comparison of the simulations with actually observed data
Using frequentist concepts to do the comparison of simulations with observations, one obtains frequentist p-values and confidence intervals. The frequentist p-values and confidence intervals are exact in the limit of investing sufficient computational time. This holds true independent of the probability model used, and independent of whether the observed data consists of a few or many observations. As such the algorithm is a valid if not superior alternative to bootstrap sampling of frequentis parameter estimates.
In the evaluation of the proposed algorithm, it has also been investigated in how far Bayesian estimates may be used as a frequentist test procedure. It has been shown that this is feasible, simple and results are comparable to those obtained with likelihood-ratio tests.

(1) Bartels, Christian (2014): Efficient generic integration algorithm to determine confidence intervals and p-values for hypothesis testing. figshare.


No comments:

Post a Comment