A new paper suggests that a "perfection premium" is a real phenomena and is also a distorting cognitive bias. It isn't obvious that this is a correct assessment.
The paper and its abstract are as follows:
This research documents a perfection premium in evaluative judgments wherein individuals disproportionately reward perfection on an attribute compared to near-perfect values on the same attribute. For example, individuals consider a student who earns a perfect score of 36 on the American College Test to be more intelligent than a student who earns a near-perfect 35, and this difference in perceived intelligence is significantly greater than the difference between students whose scores are 35 versus 34. The authors also show that the perfection premium occurs because people spontaneously place perfect items into a separate mental category than other items. As a result of this categorization process, the perceived evaluative distance between perfect and near-perfect items is exaggerated. Four experiments provide evidence in favor of the perfection premium and support for the proposed underlying mechanism in both social cognition and decision-making contexts.
Mathew S. Isaac, Katie Spangenberg "The Perfection Premium" (September 10, 2020) https://doi.org/10.1177/1948550620944313
The trouble is that I'm not sure that a perfection premium is a cognitive bias. Most scoring systems, like ACT scores, have a domain of validity. They discriminate meaningfully in a normal distribution over a range of performance. Consider the ACT:
The top score included not just everyone who is one unit better in a normal distribution broken up into 36 units than the previous score (a 2.62 sigma above average result). It also includes everyone who would have a score more than 36 if the normal distribution were continued beyond the domain of applicability of the test.
If having a genius student in the student body who would score 42 on the ACT is its scoring range extended that far is more valuable to the institution than having a student who is merely very smart and would score 36 on the ACT even if its scoring extended higher.
These are sometimes called "ceiling effects."
If a significant number of applicants with a perfect score receive a 36 only because the scoring range doesn't go any higher, it may be perfectly rational, and not a cognitive basis to have a "perfection premium".
Also, the correct amount of the "perfection premium" depends upon the extent to which the institution values having a small number of ultra-high performers (who may bring outsized recognition to the institution) relative to merely very high performers. There is no way to determine the correct premium for ultra-high performers in a vacuum.
A handful of people who would have scored much higher than 36 on the ACT if it were possible might very well bring outsized benefits to the institution in terms of awards, recognition, admissions to top grad schools that look good for the institution, much sought after organization leaders, faculty research assistants to mentor, or what have you. So a perfection premium may be appropriate.
I also question how real a "perfection premium" really is. Normally, admissions offices and hiring departments triage people into almost automatically deny, almost automatically admit/hire, and a middle ground of further deliberation. Everyone with a perfect or near perfect application goes into the almost automatically admit/hire and the threshold for decision making is almost never near the perfect threshold. If it was, the process would be reformed to be harder so it would be easier to distinguish between large numbers of seemingly perfect applicants.
Hat tip to Marginal Revolution.
No comments:
Post a Comment