Filetype PDF | Posted on 20 Sep 2022 | 4 years ago
Authentication
digital resources
259x Filetype PDF File size 1.71 MB Source: jenni.uchicago.edu
Personality and Individual Differences 51 (2011) 315–320
Contents lists available at ScienceDirect
Personality and Individual Differences
journal homepage: www.elsevier.com/locate/paid
Identication problems in personality psychology
a a,b c,d,e,⇑ c
Lex Borghans , Bart H.H. Golsteyn , James Heckman , John Eric Humphries
aMaastricht University, Tongersestraat 53, 6200 MD, Maastricht, The Netherlands
bSOFI, Stockholm University, Universitetsvägen 10F, SE-10691, Stockholm, Sweden
cUniversity of Chicago, Department of Economics, 1126 E. 59th Street, Chicago, IL 60637, USA
dUniversity College Dublin, Geary Institute, Beleld, Dublin 4, Ireland
eAmerican Bar Foundation, 750 North Lake Shore Drive, Chicago, IL 60611, USA
article info abstract
Article history: Thispaperdiscussesandillustratesidenticationproblemsinpersonalitypsychology.Themeasuresused
Received 12 March 2011 by psychologists to infer traits are based on behaviors, broadly dened. These behaviors are produced
Accepted 18 March 2011 from multiple traits interacting with incentives in situations. In general, measures are determined by
Available online 6 May 2011 these multiple traits and do not identify any particular trait unless incentives and other traits are con-
trolled for. Using two data sets, we show, that substantial portions of the variance in achievement test
Keywords: scores and grades, which are often used as measures of cognition, are explained by personality variables.
Identication problem 2011Elsevier Ltd. All rights reserved.
Personality
Psychology
Achievement test
Grades
1. Introduction This denition, or closely related versions, is used throughout per-
2
sonality psychology.
There is a growing interest by economists in personality psy- Roberts denition of personality traits refers to the stability of
chology to better understand the diversity of responses of agents certain patterns of conduct, such as actions or responses to situa-
to similar circumstances. Many economists now include personal- tions that people take, including patterns of thoughts or feelings.
ity measures and proxies for cognition in their empirical analyses. Perceptions, expectations of future events and preferences also
Howshould one interpret these estimated relationships? shape behavior, feelings and thoughts. In this way, cognitive activ-
1
Personality psychology attempts to describe the whole person. ities help to determine measuredpersonality.Inlightofthesecom-
It considers both universal traits and individual differences. It exam- mon-sense observations, how should one interpret widely used
ines the ways in which people are unique. As a sign of its breadth, measures of personality and cognition?
personality psychology considers both cognitive functioning and 3 A
Manydifferent models of personality have been formulated.
personality traits as aspects of personality. prototypical model is developed by Roberts (2006). He presents the
Characterizing what personality psychologists analyze, it is schematic displayed in Fig. 1 to relate personality traits to measured
helpful to distinguish personality traits, personality as a response behaviors, thoughts, and feelings. He distinguishes mental abilities
function, and measured personality (Almlund, Duckworth, Heck- from personality traits, although both are aspects of personality
man, & Kautz, 2011). Personality is a response function that maps broadly dened. These traits and abilities, along with preferences
personality traits to measured (manifest) personality. One leading (motives,interests, and values) and narratives (the stories people tell
personality psychologist denes personality traits in the following themselves in organizing their lives and making meanings of them),
way: shape a persons identity and reputation. This includes the views of
Personality traits are the relatively enduring patterns of thoughts, the person by others and the persons perception of how others per-
feelings, and behaviors that reect the tendency to respond in cer- ceive him. Identity and reputation shape the roles of individuals in
tain ways under certain circumstances. (Roberts, 2009, p. 140)
2 However, some personality psychologists use this or a very similar denition to
dene personality and not personality traits. Thus Cervone and Pervin (2009) dene
⇑ Corresponding author at: University of Chicago, Department of Economics, personality aspsychological qualities that contribute to an individuals enduring and
1126 E. 59th Street, Chicago, IL 60637, USA. distinctive patterns of thinking, feeling and behaving (p. 8).
E-mail address: jjh@uchicago.edu (J. Heckman). 3 See the models presented in John, Robins, and Pervin (2008) and the survey in
1 See Cervone and Pervin (2009). Cervone and Pervin (2009).
0191-8869/$ - see front matter 2011 Elsevier Ltd. All rights reserved.
doi:10.1016/j.paid.2011.03.029
316 L. Borghans et al./Personality and Individual Differences 51 (2011) 315–320
Fig. 1. Roberts (2006) model of personality as the output of a system.
the economyandthesocietytowhichtheybelong.Personalityisthe 2. IQ test scores reect incentives and capture both cognitive
system of relationships that map traits and other determinants of and personality traits
4 Measured
behavior, thoughts, and feelings into measured actions.
personality results from interactions among the components of the Isolating a pure measure of intelligence is difcult. Intelligence
system. Personality traits are only one determinant of measured is commonlyregardedasdistinctfromnoncognitiveorpersonal-
personality. ity traits. By the denition of personality given in Section 1, intel-
Figure 1 illustrates the origins of the identication problem dis- ligence is one aspect of personality. It is a measure of how well a
5
cussed in this paper. Measurements of thoughts, feelings, and person responds to (performs on) intelligence tests (see Almlund
behaviors that arise from responses to incentives and social interac- et al., 2011).
tions are used to infer personality traits and abilities. Personality Performanceonintelligence(andachievement)testsdependsin
traits and cognitive abilities, along with the other units of analysis partonthepersonalitytraitsofthetesttakersincludingtheirmoti-
in Fig. 1, produce the measures that are used to infer the generating 6 A smart child unable to sit still during an exam
vation to perform.
traits. oruninterestedinexertingmucheffortcanproducelowscoresonan
Behaviors include actions taken by agents whether in a task in IQ test.
the workplace, in interactions with others observed by third par- It is sometimes claimed that IQ tests measure maximal perfor-
ties, or as measured by scores on tests of cognition or personality. mance, i.e. that IQ scores reect the application of the maximal
Toinfertraitsandabilitiesfrommeasuresrequiresparsingoutor 7
capacityofthepersontothetests. InspectionofFig.1suggeststhat
standardizing for all of the other factors that also produce the ob- IQ scores should be standardized for effort. A series of studies con-
served behavior, including incentives created by the situations in ducted over the past 40 years support this concern.
which people are placed. This is a challenging task. The difculty ThesestudiesshowthatamongindividualswithlowbaselineIQ
in isolating traits from behaviors, thoughts or feelings gives rise scores, performance on subsequent IQ tests can be increased up to
to a fundamental identication problem. We illustrate this prob- a full standard deviation by offering incentives such as money or
lem with two examples: (a) interpreting what IQ tests measure candy for correct answers, particularly on group-administered
and (b) interpreting what achievement tests measure. tests and particularly for individuals at the low-end of the IQ
In Section 2, we report evidence that scores on IQ tests are 8 Engaging in complex thinking is effortful, not automatic
spectrum.
determined by incentives and personality. Section 3 shows that
scores on achievement tests and grades, often used as measures 6 It is likely that performance on personality tests can also depend on cognitive
of cognition, are determinedinsubstantialpartbypersonality.Sec- ability, but that is less well documented. For example, it is likely that more intelligent
tion 4 concludes. people can ascertain the rewards to performance on a personality inventory test.
Motivation is sometimes, but not often, counted as a personality trait (see Borghans,
Meijers, & ter Weel, 2008).
4 This system is formalized in Almlund et al. (2011). 7 A leading psychometrician, Carroll (1993), discusses this claim but does not
5 Almlund et al. (2011) present a formal characterization of the identication accept the notion that IQ captures maximal effort.
problem and solutions to it. 8 The incentives for invoking effort vary across studies.
L. Borghans et al./Personality and Individual Differences 51 (2011) 315–320 317
(Schmeichel, Vohs, & Baumeister, 2003), and therefore motivation to Table 1
exert effort affects performance. Borghans et al. (2008) and Almlund Correlations among NLSY79 measures of cognition.
et al. (2011) summarize the literature on the effects of incentives on Correlation between IQ, AFQT, and GPA
IQ tests. See Table 1 in the Web Appendix, taken from Almlund et al. IQ Achievement (AFQT) Grade Point Average (GPA)
9
(2011). IQ 1
The response to incentives depends on personality traits. It is AFQT 0.65 1
not enough to standardize for incentives to measure intelligence GPA(9th) 0.42 0.54 1
withIQtests. One shouldalso standardize for the personality traits Source: National Longitudinal Survey of Youth (NLSY79). Pooled male and female
that govern the response to incentives. Segal (2008) shows that random sample.
introducing performance-based cash incentives in a low-stakes Notes: The Armed Forces Qualifying Test (AFQT) was administered in 1980 when
administration of the coding speed test of the Armed Services subjects were 15–22. AFQT is adjusted for schooling at the time of the test condi-
Vocational Battery (ASVAB) increases performance substantially, tional on nal schooling, following the procedure in Hansen, Heckman and Mullen
butonlyforroughlyone-thirdofparticipants. Men withlowerlev- (2004). AFQT is constructed from Arithmetic Reasoning, Word Knowledge, Math
els of the Big Five trait Conscientiousness are particularly moti- Knowledge, and Paragraph Comprehension tests. IQ and GPA are from high school
transcripts. IQ is pooled across several IQ tests using IQ percentiles. GPA is the
vated by incentives. individuals core-subject GPA measured in 9th grade when virtually all sample
Borghans et al. (2008) show that adults spend substantially participants are enrolled. Differences between males and females are slight. For the
more time answering IQ questions when rewards are higher, but sake of brevity we report pooled results.
subjects high in the Big Five traits Emotional Stability and Consci-
entiousnessarelessaffectedbysuchincentives.Theyalreadyoper- scores indirectly through the greater knowledge acquired by indi-
ate at a high level even without these incentives. Similarly, Pailing viduals with high levels of specic personality traits in addition to
and Segalowitz (2004) nd that an event-related potential (ERP) the motivational factors previously discussed. Hansen et al. (2004)
indexing the emotional response to making an error increases in and Heckman, Stixrud, and Urzua (2006) show that schooling and
amplitude when incentives are offered for superior test perfor- other acquired traits substantially causally affect measured cogni-
10 Thus, IQ scores do not accurately reect maximal intellec- tive and personality test scores.
mance.
tual performance for individuals who are low in Conscientiousness Achievement tests are typically designed to capture general
and Emotional Stability. Performance on IQ tests encodes, in part, knowledge, i.e., not knowledge of specic facts or the contents
howeffective persons may be in the application of their intelligence, of specic courses, but the knowledge required to function effec-
11 Achievement tests attempt to capture
that is, how people are likely to perform in some real-world settings. tively in modern society.
Like low motivation, test anxiety can signicantly impair per- different aspects of cognition than are captured by IQ tests, but
formance (Hembree, 1988). That is, subjects do worse when they scores on the two types of tests are highly correlated. As a result,
worry excessively about their performance, which causes their the two types of measures are sometimes used interchangeably in
autonomic nervous system to over-react by increasing perspira- popular and academic discussions. For example, Herrnstein and
tion, heart rate, and so on. Individuals who are higher in Big Five Murray (1994) use an achievement test as a measure of IQ. In later
Neuroticism are more likely to experience test anxiety. This is an- work, Nisbett (2009) uses achievement test scores as a measure of
other mechanism, beyond its interaction with incentives, through intelligence. We show that this is a dangerous practice. Achievement
whichEmotionalStabilitycanimpactIQscores(Mouta,Furnham, test scores depend on both personality and IQ. Empirical demonstra-
&Tsaousis, 2006). tions of the importance of intelligence, based on scores on achieve-
Thus, IQ test performance captures not only pure intelligence, ment tests or grades, are also demonstrations of the power of
but also personality traits (including anxiety), intrinsic motivation, personality.
and reactions to extrinsic incentives to perform well, as suggested Table1displaysthecorrelationsamongthreewidelyusedmea-
by Fig. 1. The relative impurity of IQ tests likely varies from test to sures of cognition recorded in the adolescent years, IQ, an achieve-
test and individual to individual. Little effort to date has been made ment test (the Armed Forces Qualifying Test or AFQT), and report
12,13 The correlations are large but by
to standardize the context and incentives of tests. To capture pure card grades (in ninth grade).
intelligence, it is necessary to adjust for incentives, motivations, no means do the measures perfectly correlate.
and context in which the measurements are taken. It is well established that measures of intelligence and academic
achievement predict a variety of social and economic outcomes
2 of such predictive relationships rarely exceeds
3. Interpreting what grades and achievement tests measure although the R
14
10–15%. Borghans, Golsteyn, Heckman, and Humphries (2011)
The sameissues discussed in regard to IQ tests apply with even examine the predictive power of grades, IQ and achievement tests
greater force to achievement tests and grades. Achievement tests measured in the adolescent years for a variety of life outcomes past
15 The R2 of most relationships is below 0.10.
require factual knowledge acquired through schooling and life age 30.
experience, which are, in part, determined by the motivation, curi- A general pattern emerges from their work. Achievement test
osity, and persistence of the test taker. Cunha and Heckman (2008) scores are more highly correlated with outcomes than are IQ tests.
andCunha,Heckman,andSchennach(2010)showthatpersonality
traits facilitate the accumulation of cognitive skills as measured by 11 See Lindquist, Van Dyke, and Yale (1948). Lindquist, along with Ralph Tyler,
achievement tests. Thus, personality traits affect achievement test pioneeredtheconceptofgeneralknowledge,whichmotivatedtheachievementtest
movement.
12 The AFQT consists of four subtests: word knowledge, paragraph comprehension,
9 Zigler and Buttereld (1968) found that early intervention (nursery school, for arithmetic reasoning, and mathematics knowledge (Roberts et al., 2000, p. 19).
example) for low-SES children may have a benecial effect on motivation, not on 13 Many interpret the AFQT as an IQ test. For discussion of the contrast between
cognitive ability per se. In their study, the benets of intervention (in comparison to a achievement and IQ tests see the collection of papers in Green (1974). Many of the
no-treatment control group) on IQ were not apparent under testing conditions where contributors to that book do not think any distinction is meaningful.
motivation to perform well was maximal. Raver and Zigler (1997) present further 14 For evidence on the predictive power of cognitive measures, see, for example,
evidenceonthispoint.Heckman,Molofeeva,Pinto,andSavelyev(2011)showthatthe Herrnstein and Murray (1994), Gottfredson (2008), Cawley, Heckman, and Vytlacil
Perry Preschool program improved productive personality traits but did not raise IQ. (1999), Heckman et al. (2006), Taubman and Wales (1973), Jencks et al. (1972), and
The intervention has a 7–10% annual rate of return. Bowles, Gintis, and Osborne (2001).
10 An ERP is an electrophysiological response of characteristic form and timing to a 15 The outcomes include wages, income, hours worked, depression, smoking,
particular category of stimuli. physical activity, health, voting, divorce and unemployment.
318 L. Borghans et al./Personality and Individual Differences 51 (2011) 315–320
Fig. 2. Decomposing achievement tests and grades into IQ and personality.
The correlation of grades with outcomes is intermediate between TheNLSYisanationally representative sample of American youth.
IQ and achievement tests. Achievement tests and grades capture Youthwere14–22atthedateofinitialenrollment(1979)andhave
traits valued in economic and social life other than measured been followed ever since. The second sample (results displayed in
intelligence. Panel B) shows the predictive power of IQ and personality mea-
Gradesandachievementtestscorespredictadultoutcomesbet- sures on achievement scores and grades for a single Dutch high
ter than IQ because they also capture personality traits. This ex- school (Stella Maris) sample in 2008. There is no long-term fol-
plains why achievement tests and grades have more predictive 17
low-up of this sample.
power than IQ.16 Another interpretation of this evidence is that ac- The NLSY data have relatively weak measures of personality:
quired knowledge as captured by achievement tests and grades is the Rosenberg measure of self-esteem and the Rotter locus of con-
more predictive than uid intelligence as measured by IQ. As previ- trol. They are related to some of the Big Five traits. (See the discus-
ously noted, personality traits affect the accumulation of knowledge. sion in Almlund et al., 2011). The Dutch data, while less
Figure 2 presents evidence from two samples on the joint and representative and subject to the problem of restriction on range
individualcontributionsofIQandpersonalitymeasurestoexplain- (only the students from the upper and middle level tracks are sam-
ing the variance in achievement test scores and grades as mea- pled, students from the lower track are not) have measurements of
2. The rst sample (results displayed in Panel A),
sured by R all of the Big Five inventory plus the Grit measure of persistence
extracted from the National Longitudinal Survey of Youth developed by Duckworth, Peterson, Matthews, and Kelly (2007).
(NLSY79), is used to produce the correlations reported in Table 1. The Differential Aptitude Test (DAT) measurement of achievement
is very similar in content to the AFQT achievement test in the
NLSY79. The range of correlations between DAT scores and AFQT
16 Duckworth, Quinn, and Tsukayama (2010) present related evidence. See also
Bowen, Chingos, and McPherson (2009), Willingham, Pollack, and Lewis (2002), and
Duckworth and Seligman (2005). 17 A more comprehensive description of the data is given in the Web Appendix.
no reviews yet
Please Login to review.
