The fallacy of placing confidence in confidence intervals

Today we’ve discussed the following paper about the reliability of confidence intervals:

The fallacy of placing confidence in confidence intervals

Richard D. Morey, Rink HoekstraJeffrey N. Rouder, Michael D. Lee & Eric-Jan Wagenmakers

Theoretical Review. Open Access. Published: 

Morey and colleagues define interval estimates and confidence intervals as follows:

  • Interval estimates are a key component of statistical analysis and there are several kinds of interval estimates although the most popular are confidence intervals (CIs).
  • Confidence intervals (CIs) are intervals that contain the true parameter value in some known proportion of repeated samples, on average, ans satisfy the following properties:
    • The width of CIs is thought to index the precision of an estimate
    • CIs are thought to be a guide to which parameter values are reasonable.
    • The confidence coefficient of the interval (e.g., 95%) is thought to index the plausibility that the true parameter is included in the interval.

What the authors showed in this paper is a number of examples in which CIs do not necessarily have any of these properties, and can lead to unjustified or arbitrary inferences. For this reason, they caution against relying upon confidence interval theory to justify interval estimates, and suggest that other theories of interval estimation should be used instead.

Sources

Feature image from Pexels – C0 license.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s