Positive bias

From LessWrong
Jump to navigation Jump to search
Wikipedia has an article about

Positive or confirmation bias is a tendency to test hypotheses with positive rather than negative examples, thus risking to miss obvious disconfirming tests.

Kevin Kelly argues that negative results should be "saved, shared, compiled and analyzed, instead of being dumped. Positive results may increase their credibility when linked to negative results." [1] If so then this bias is particularly dangerous as the lack of negative results would themselves cast doubt on even entirely valid conclusions.

As an extreme example, imagine one hundred algorithms for stock market prediction placed in one hundred safety deposit boxes under a hundred different assumed names. Ten years later, to great fanfare, only one box, the one containing the most accurate post-facto results, is opened. There is no universally accepted and undeniable way to prove that any of the other 99 did or didn't exist without a discipline that forces negative result reporting. Even a person who had filed away only one algorithm under their own name, once, would therefore be suspect.

Blog posts

External links


  • P.C. Wason (1960). "On the failure to eliminate hypotheses in a conceptual task". Quarterly Journal of Experimental Psychology 12: 129-140.  (PDF)

See also