Cognitive and neural foundations of probability judgment

An enduring question in cognitive psychology concerns how people generate estimates of confidence and probability. What are the principles of coherence that underlie our beliefs? What makes evidence seem consistent with a hypothesis? In what sense and to what degree do our judgments have a rational basis?

Intuitions about probability frequently deviate dramatically from the dictates of probability theory. One form of deviation is notorious: people’s tendency to neglect base-rates in favor of specific case data. Consider, for example, the following medical diagnosis problem.

The probability of breast cancer is 1% for a woman at age forty who participates in routine screening. If a woman has breast cancer, the probability is 80% that she will get a positive mammography. If a woman does not have breast cancer, the probability is 9.6% that she will also get a positive mammography. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer? __%

According to Bayes’ theorem, the probability that the patient has breast cancer given that she has a positive mammography is 7.8 per cent. Evidence that people’s judgments conform to Bayes’ theorem would be consistent with the claim that the mind embodies a calculus of probability, whereas the lack of such a correspondence would demonstrate that people’s judgments can be at variance with sound probabilistic principles. Thus, the extent to which probability estimates conform to Bayes’ theorem has implications for the nature of human judgment. In the case of this problem, fewer than 20 per cent of the respondents typically generate the Bayesian solution.

A number of theorists have argued that such errors reveal little more than experimenters’ failure to ask about uncertainty in a form that naïve respondents can understand, specifically in the form of a question about natural frequencies. Consider, for example, the following formally identical problem.

10 out of every 1,000 women at age forty who participate in routine screening have breast cancer. 8 out of every 10 women with breast cancer will get a positive mammography. 95 out of every 990 women without breast cancer will also get a positive mammography. Here is a new representative sample of women at age forty who got a positive mammography in routine screening. How many of these women do you expect to actually have breast cancer? ___ out of ___.

The proportion of responses conforming to Bayes’ theorem increase by a factor of about three in this case, 46 per cent under natural frequency formats versus 16 per cent under a single-event probability format. This finding has motivated researchers to argue that coherent probability judgment depends on representing events in the form of natural frequencies. Advocates of the natural frequency hypothesis propose the mind embodies a Bayesian algorithm or module that is designed to process natural frequencies.

We have advanced an alternative framework, which proposes that people are adept at using rules that consist of elementary set operations and that natural frequency formats are an effective cue to the representation of the set structure of the problem. Thus, the nested sets theory accounts for the observed findings in terms of the set structure of the problem, rather than appealing to an evolutionary adaptation to process natural frequencies.

This research supports five main conclusions (Barbey & Sloman, 2007). First, the facilitory effect of natural frequencies on Bayesian inference varies considerably within the literature, potentially resulting from differences in the general intelligence level and motivation of participants. These findings support the nested sets hypothesis to the degree that intelligence and motivation reflect the operation of domain general and strategic – rather than automatic (i.e., modular) cognitive processes.

Second, questions that prompt use of category instances and divide the solution into the sets needed to compute the Bayesian ratio facilitate probability judgment, suggesting that facilitation depends on cues to the set structure of the problem rather than an evolved capacity to process natural frequencies. In further support of this conclusion, partitioning the data into nested sets facilitates Bayesian inference regardless of whether natural frequencies or single-event probabilities are employed.

Third, frequency judgments are guided by inferential strategies that reflect incomplete, fragmentary memories that do not entail the base-rates, suggesting that Bayesian inference does not derive from the accurate encoding and retrieval of natural frequencies. In addition, natural frequencies and single-event probabilities are rated similarly in their perceived clarity, understandability, and impact on the respondent’s behavior, further suggesting that the mind does not embody inductive reasoning mechanisms that are designed to process natural frequencies.

Fourth, people (a) do not accurately weight and combine event frequencies, and (b) utilize event frequencies that are irrelevant in the calculation of Bayes’ theorem, suggesting that the cognitive operations that underlie Bayesian inference do not conform to Bayes’ theorem. Furthermore, base-rate usage increases under frequentist representations, suggesting that facilitation results from the property of natural frequencies to represent the sample and effect sizes, which highlight the set structure of the problem and make transparent what is relevant for problem solving.

Finally, nested set representations facilitate reasoning in a range of classic deductive and inductive reasoning tasks, supporting the nested set hypothesis that the mind embodies a domain general capacity to perform elementary set operations and that these operations can be induced by cues to the set structure of the problem to facilitate reasoning in any context where people tend to rely on associative rather than extensional, rule-based processes.

Our current neuroscience research extends this work by investigating how judgments of confidence and probability are implemented within neural systems. In particular, this work investigates the neural systems mediating elementary set operations (taking the union and intersection of sets) and their role in normative versus non-normative estimates of probability.

References

Galesic, M., Barbey, A.K., Krueger, F., Grafman, J. & Gigerenzer, G. (2008). Assessing the role of estimation strategy and representation format in statistical learning. In Proceedings of the Thirtieth Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.

Barbey, A.K. & Sloman, S.A. (2007). Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences, 30, 241-254. [pdf]

Barbey, A.K. & Sloman, S.A. (2007). Base-rate respect: From statistical formats to cognitive structures. Behavioral and Brain Sciences, 30, 287-292. [pdf]

Brase, G.L. & Barbey, A.K. (2006). Mental representations of statistical information. In Shohov, S. P. (Ed.), Advances in Psychological Research, Hauppauge, NY: Nova Science Publishers, Inc. [pdf]