Powers, D. E., Kaufman, J. C. (2004). Do standardized tests penalize deep-thinking, creative, or conscientious students? Some personality correlates of Graduate Record Examinations test scores. Intelligence, 32 145-153.
A. Purpose of Study
1. Provide evidence regarding the relationship of selected personality traits to GRE scores
2. Intelligence related personality traits measured: conscientiousness, rationality, ingenuity, quickness, creativity, and depth.
B. GRE Criticisms
1. Test privileges “superficial-thinking students” and not “deep thinkers”
2. Disadvantage “intellectually honest candidates with subtle, probing, critical, or creative minds”
3. Tap only a very narrow set of skills and abilities
4. Inability to measure elements of creativity
5. Not measuring empathy and altruism, traits linked to career ability/success
C. Results of Study
1. GRE score correlated strongest with creativity
2. Quickness second strongest trait correlated to GRE score
a. Test seems to privilege test takers who work rapidly
3. GRE score correlated significantly and slightly negatively with conscientiousness
a. “Able to accomplish work on time” and “careful to avoid making mistakes.”
i. Potentially losing time on a time critical test
D. Critique of Results
1. The authors did not define how many logical and analytical tests were taken
2. Study sample not representative of all GRE test takers
a. Sample included greater proportion of minority test takers than true GRE population
3. Study requirements required diligent participants
a. Potentially fielding out “average” GRE takers
i. Causing participants who stayed in the study to be naturally more conscientious-possibly
Standardized tests such as the Scholastic Achievement Test (SAT), and the Graduate Record Examinations (GRE) have undergone much scrutiny and criticism in the past few years. Banesh Hoffman criticized such tests for “favoring simple-minded students at the expense of deep thinking students” (Powers & Kaufman, 2004). He further described that GRE type tests hindered creative impulses and thus did not provide applicants a fair chance at competing for admittance into Graduate School (Powers & Kaufman, 2004). Other criticisms convey the consensus that standardized tests fail to reflect many skills and abilities that determine academic success, that only a confined set of skills and abilities are assessed. Test makers disagree on this point however, suggesting that GRE scores can complement other desirable traits that may not be strongly correlated or emphasized. In order to investigate this dimension, Goff and Ackerman (1992), studied relations between ACT scores on both fluid and crystallized measures of intelligence and the big five personality traits (Powers & Kaufman, 2004). Results from their study indicated slight correlations between the big five personality traits and ACT scores. Schmidt and Hunter (1998) in their study specifically examined the trait conscientiousness with job performance and found a significant correlation between the two variables.
Using these studies as a backdrop, Kaufman and Powers (2004), studied the relationship of selected personality traits, specifically conscientiousness, rationality, creativity, quickness, and ingenuity, to scores on the GRE. Their study included 342 students who volunteered from a random sample of 3100 test takers who had taken the GRE General Test in May 2001. Researchers attempted to parallel GRE population and thus created a study where the subjects were 77% women, 49% white in ethnicity, 21% Black/African Americans, 14% Asian Americans and 16% Hispanic Americans. Participants were given short tests of analytical and logical reasoning skills and were asked to complete several Inventory scales from the International Personality Item Pool. Statistically significant correlations were found between GRE scores and the specific personality traits. “Each GRE score correlated most strongly with creativity and next most strongly with quickness. GRE scores also correlated significantly, but only slightly (negatively), with conscientiousness” (Powers & Kaufman, 2004). The Results from this study suggest some fallacies with earlier arguments against standardized testing. Kaufman and Powers’ (2004) study indicated no evidence that deep-thinking or creative students were discriminated against in such tests. On the contrary, creativity consistently correlated with GRE scores, and highly creative students did not test worse than their less creative counterparts. The most unusual of these findings is the negative correlation found between conscientiousness and GRE scores. The evidence does not suggest any clear reasons why high conscientiousness is slightly negatively related to lower GRE scores, but one hypothesis is that this trait does not serve well when the test is timed. While this study provides some compelling findings, it is important to note that the population studied is not entirely representative of the general population of students that take the GRE, especially regarding ethnicity. Age of the students is also not taken into account. Furthermore, personality traits of students were measured using self-reports, thus giving way for social desirability effects and bias.