Adaptive Intervals for Hypothesis Testing (aiht): A Synthesis of Procedures

Luis Raúl Pericchi(University of Puerto Rico)

Very little has been proposed in the literature, which may be teachable in elementary courses, to bridge above the main disagreement between Schools of Statistics, and in particular to alleviate the discrepancy between Significance Hypothesis Testing (and Testing via Intervals with Fixed Size) and Bayesian Hypothesis Testing.

It is well known, that huge samples lead to an almost sure rejection of Null Hypothesis by Classical Hypothesis Testing and by Bayesian Intervals with fixed probability that do not adapt with the amount of information and sample size.

We propose here a calibration of p-values and significance levels that convey specific guidelines in how to diminish the alpha-levels (or increase the posterior probability of a Bayesian Interval) as the sample sizes grows. This effectively alleviates the discrepancy of Bayes Factors and Classical Testing, and makes probability intervals apt for performing a hypothesis test with large sample sizes. Furthermore, the resulting adaptive alpha levels imply enlarged intervals that can be used for testing hypothesis with similar decisions for both Schools of Statistics. Our basic position is that the posterior probability of a model has the same interpretation for any sample size, but that a p-value, or an alpha-level is heavily dependent on the sample size and should be corrected in order to be close to a posterior model probability. Once this is done, the "disagreement" in Statistics, largely disappears.