ERRORS OF JUDGMENT AND CHOICE----Episode 5

         ON WHAT DO PEOPLE BASE THEIR DECISIONS 

Insensibility to predictability.  People are sometimes called upon to make such numerical predictions as the future value of a stock, the demand for a commodity, or the outcome of a football game. Such predictions are often made by representativeness. For example, suppose one is given a description of a company and is asked to predict its future profit. If the description of the company is very favorable, a very high profit will appear most representative of that description ; if the description is mediocre, a mediocre performance will appear most representative.  The degree to which the description is favorable is unaffected by the reliability of that description or by the degree to which it permits accurate prediction.  Hence, if people predict solely in terms of the favorableness of the description their predictions will be insensitive to the reliability of the evidence and to the expected accuracy of the prediction. 

   This mode of judgment violates the normative statistical theory in which the extremeness and the range of predictions are controlled by considerations of predictability. When predictability is nil, the same prediction should be made in all cases. For example, if the description of companies provide no information relevant to profit, then the same value [such as average profit] should be predicted for all companies. If predictability is perfect, of course, the values predicted will match the actual values and the range of predictions will equal the range of outcomes. In general, the higher the predictability, the wider the range of predicted values. 

   Several studies of numerical prediction have demonstrated that intuitive predictions violate this rule, and that subjects show little or no considerations of predictability. In one of these studies, subjects were presented with several paragraphs, each describing the performance of a student teacher during a particular practice lesson. Some subjects were asked to evaluate the quality of the lesson described in the paragraph in percentile scores, relative to a specified population.  Other subjects were asked to predict, also in percentile scores, the standing of each student teacher 5 years after the practice lesson. The judgment made under the two conditions were identical. That is, the prediction of a remote criterion [success of a teacher after 5 years] was identical to the evaluation of the information on which the prediction was based [the quality of the practice lesson] . The students who made these predictions were undoubtedly aware of the limited predictability of teaching competence on the basis of a single trial lesson 5 years earlier ; nevertheless, their predictions were as extreme as their evaluations. 

The illusion of validity.     As we have seen, people often predict by selecting the outcome [for example, an occupation] that is most representative of the input [ for example, the description of a person] . The confidence they have in their prediction depends primarily on the degree of representativeness [that is, on the quality of the match between the selected outcome and the input] with little or no regard for the factors that limit predictive accuracy. Thus, people express great confidence in the prediction that a person is a librarian when given a  description of his personality which matches the stereotype of librarians, even if the description is scanty, unreliable, or outdated. The unwarranted confidence which is produced by a good fit between the predicted outcome and the input information may be called the illusion of validity. This illusion persists even when the judge is aware of the factors that limit the accuracy of his predictions. It is a common observation that psychologists who conduct selection often experience considerable confidence in their predictions, even when they know of the vast literature that shows interviews to be highly fallible. The continued reliance on the clinical interview for selection, despite repeated demonstrations of its inadequacy amply attests to the strength of this effect. 
   The internal consistency of a pattern of inputs is a major determinant of one's confidence in prediction based on these inputs. For example, people express more confidence in predicting the final grade point average of a student whose first-year record consists entirely of B's than in predicting the grade point average of a student whose first-year record includes many A's and C's. Highly consistent patterns are most often observed when the input variables are highly redundant or correlated. Hence, people tend to have great confidence in predictions based on redundant input variables. However, an elementary result in the statistics of correlation asserts that, given input variables of stated validity, a prediction based on several such inputs can achieve higher accuracy when they are independent of each other than when they are redundant or correlated. Thus, redundancy among inputs decreases accuracy even as it increases confidence, and people are often confident in predictions that are quite likely to be off the mark.