The third possibility is that cognitive ability itself—sheer intellectual horsepower, independent of education—has market value. Seen from this perspective, the college degree is not a credential but an indirect measure of intelligence. People with college degrees tend to be smarter than people without them and, by extension, more valuable in the marketplace. Employers recruit at Stanford or Yale not because graduates of those schools know more than graduates of less prestigious schools but for the same generic reason that Willie Sutton gave for robbing banks. Places like Stanford and Yale are where you find the coin of cognitive talent.
The first two explanations have some validity for some occupations. Even the brightest child needs formal education, and some jobs require many years of advanced training. The problem of credentialing is widespread and real: the B. A. is a bogus requirement for many management jobs, the requirement for teaching certificates often impedes hiring good teachers in elementary and secondary schools, and the Ph.D. is irrelevant to the work that many Ph.D.s really do.
But whatever the mix of truth and fiction in the first two explanations, the third explanation is almost always relevant and almost always ignored. The process described in the previous chapter is driven by a characteristic of cognitive ability that is at once little recognized and essential for understanding how society is evolving: intelligence is fundamentally related to productivity. This relationship holds not only for highly skilled professions but for jobs across the spectrum. The power of the relationship is sufficient to give every business some incentive to use IQ as an important selection criterion.
That in brief is the thesis of the chapter. We begin by reviewing the received wisdom about the links between IQ and success in life, then the evidence specifically linking cognitive ability to job productivity.
THE RECEIVED WISDOM
“Test scores have a modest correlation with first-year grades and no correlation at all with what you do in the rest of your life,” wrote Derek Bok, then president of Harvard University, in 1985, referring to the SATs that all Harvard applicants take.1 Bok was poetically correct in ways that a college president understandably wants to emphasize. A 17-year-old who has gotten back a disappointing SAT score should not think that the future is bleak. Perhaps a freshman with an SAT math score of 500 had better not have his heart set on being a mathematician, but if instead he wants to run his own business, become a U.S. senator, or make a million dollars, he should not put aside those dreams because some of his friends have higher scores. The link between test scores and those achievements is dwarfed by the totality of other characteristics that he brings to his life, and that’s the fact that individuals should remember when they look at their test scores. Bok was correct in that, for practical purposes, the futures of most of the 18-year-olds that he was addressing are open to most of the possibilities that attract them.
President Bok was also technically correct about the students at his own university. If one were to assemble the SATs of the incoming freshmen at Harvard and twenty years later match those scores against some quantitative measure of professional success, the impact could be modest, for reasons we shall discuss. Indeed, if the measure of success was the most obvious one, cash income, then the relationship between IQ and success among Harvard graduates could be less than modest; it could be nil or even negative.2
Finally, President Bok could assert that test scores were meaningless as predictors of what you do in the rest of your life without fear of contradiction, because he was expressing what “everyone knows” about test scores and success. The received wisdom, promulgated not only in feature stories in the press but codified in landmark Supreme Court decisions, has held that, first of all, the relation between IQ scores and job performance is weak, and, second, whatever weak relationship there is depends not on general intellectual capacity but on the particular mental capacities or skills required by a particular job.3
There have been several reasons for the broad acceptance of the conclusions President Bok drew. Briefly:
A Primer on the Correlation Coefficient
We have periodically mentioned the “correlation coefficient” without saying much except that it varies from −1 to +1. It is time for a bit more detail, with even more to be found in Appendix 1. As in the case of standard deviations, we urge readers who shy from statistics to take the few minutes required to understand the concept. The nature of “correlation” will be increasingly important as we go along.
A correlation coefficient represents the degree to which one phenomenon is linked to another. Height and weight, for example, have a positive correlation (the taller, the heavier, usually). A positive correlation is one that falls between zero and +1, with +1 being an absolutely reliable, linear relationship. A negative correlation falls between O and −1, with −1 also representing an absolutely reliable, linear relationship, but in the inverse direction. A correlation of O means no linear relationship whatsoever.4
A crucial point to keep in mind about correlation coefficients, now and throughout the rest of the book, is that correlations in the social sciences are seldom much higher than .5 (or lower than −.5) and often much weaker—because social events are imprecisely measured and are usually affected by variables besides the ones that happened to be included in any particular body of data. A correlation of .2 can nevertheless be “big” for many social science topics. In terms of social phenomena, modest correlations can produce large aggregate effects. Witness the prosperity of casinos despite the statistically modest edge they hold over their customers.
Moderate correlations mean many exceptions. We all know people who do not seem all that smart but who handle their jobs much more effectively than colleagues who probably have more raw intelligence. The correlations between IQ and various job-related measures are generally in the .2 to .6 range. Throughout the rest of the book, keep the following figure in mind, for it is what a highly significant correlation in the social sciences looks like. The figure uses actual data from a randomly selected 1 percent of a nationally representative sample, using two variables that are universally acknowledged to have a large and socially important relationship, income and education, with the line showing the expected change in income for each increment in years of education.5 For this sample, the correlation was a statistically significant .33, and the expected value of an additional year of education was an additional $2,800 in family income—a major substantive increase. Yet look at how numerous are the exceptions; note especially how people with twelfth-grade educations are spread out all along the income continuum. For virtually every topic we will be discussing throughout the rest of the book, a plot of the raw data would reveal as many or more exceptions to the general statistical relationship, and this must always be remembered in trying to translate the general rule to individuals.
The variation among individuals that lies behind a significant correlation coefficient
The exceptions associated with modest correlations mean that a wide range of IQ scores can be observed in almost any job, including complex jobs such as engineer or physician, a fact that provides President Bok and other critics of the importance of IQ with an abundant supply of exceptions to any general relationship. The exceptions do not invalidate the importance of a statistically significant correlation.
Restriction of range. In any particular job setting, there is a restricted range of cognitive ability, and the relationship between IQ scores and job performance is probably very weak in that setting. Forget about IQ for a moment and think about weight as a qualification for being an offensive tackle in the National Football League. The All-Pro probably is not the heaviest player. On the other hand, the lightest tackle in the league weighs about 250 pounds. That is what we mean by restriction of range. In terms of correlation coefficients, if we were to rate the performance of every NFL offensive tackle and then correlate those ratings with their weights, the result would probably be a correlation near zero. Should we then approach the head coaches of the NFL and rec
ommend that they try out a superbly talented 150-pound athlete at offensive tackle? The answer is no. We would be right in concluding that performance does not correlate much with weight among NFL tackles, whose weights range upward from around 250, but not about the correlation in the general population. Imagine a sample of ordinary people drawn from the general population and inserted into an offensive line. The correlation between the performance of these people as tackles in football games and their weights would be large indeed. The difference between these two correlations—one for the actual tackles in the NFL and the other a hypothetical one for people at large—illustrates the impact of restriction of range on correlation coefficients.6
Confusion between a credential and a correlation. Would it be silly to require someone to have a minimum score on an IQ test to get a license as a barber? Yes. Is it nonetheless possible that IQ scores are correlated with barbering skills? Yes. Later in the chapter, we discuss the economic pros and cons of using a weakly correlated score as a credential for hiring, but here we note simply that some people confuse a well-founded opposition to credentialing with a less well-founded denial that IQ correlates with job performance.7
The weaknesses of individual studies. Until the last decade, even the experts had reason to think that the relationship must be negligible. Scattered across journals, books, technical reports, conference proceedings, and the records of numberless personnel departments were thousands of samples of workers for whom there were two measurements: a cognitive ability test score of some sort and an estimate of proficiency or productivity of some sort. Hundreds of such findings were published, but every aspect of this literature confounded any attempt to draw general conclusions. The samples were usually small, the measures of performance and of worker characteristics varied and were more or less unreliable and invalid, and the ranges were restricted for both the test score and the performance measure. This fragmented literature seemed to support the received wisdom: Tests were often barely predictive of worker performance and different jobs seemed to call for different predictors. And yet millions of people are hired for jobs every year in competition with other applicants. Employers make those millions of choices by trying to guess which will be the best worker. What then is a fair way for the employer to make those hiring decisions?
Since 1971, the answer to that question has been governed by a landmark Supreme Court decision, Griggs v. Duke Power Co.8 The Court held that any job requirement, including a minimum cutoff score on a mental test, must have a “manifest relationship to the employment in question” and that it was up to the employer to prove that it did.9 In practice, this evolved into a doctrine: Employment tests must focus on the skills that are specifically needed to perform the job in question.10 An applicant for a job as a mechanic should be judged on how well he does on a mechanical aptitude test, while an applicant for a job as a clerk should be judged on tests measuring clerical skills, and so forth. So decreed the Supreme Court, and why not? In addition to the expert testimony before the Court favoring it, it seemed to make good common sense.
THE RECEIVED WISDOM OVERTURNED
The problem is that common sense turned out to be wrong. In the last decade, the received wisdom has been repudiated by research and by common agreement of the leading contemporary scholars.11 The most comprehensive modern surveys of the use of tests for hiring, promotion, and licensing, in civilian, military, private, and government occupations, repeatedly point to three conclusions about worker performance, as follows.
Job training and job performance in many common occupations are well predicted by any broadly based test of intelligence, as compared to narrower tests more specifically targeted to the routines of the job. As a corollary: Narrower tests that predict well do so largely because they happen themselves to be correlated with tests of general cognitive ability.
Mental tests predict job performance largely via their loading on g.
The correlations between tested intelligence and job performance or training are higher than had been estimated prior to the 1980s. They are high enough to have economic consequences.
We state these conclusions qualitatively rather than quantitatively so as to span the range of expert opinion. Whereas experts in employee selection accept the existence of the relationship between cognitive ability and job performance, they often disagree with each other’s numerical conclusions. Our qualitative characterizations should be acceptable to those who tend to minimize the economic importance of general cognitive ability and to those at the other end of the range.12
Why has expert opinion shifted? The answer lies in a powerful method of statistical analysis that was developing during the 1970s and came of age in the 1980s. Known as meta-analysis, it combines the results from many separate studies and extracts broad and stable conclusions.13 In the case of job performance, it was able to combine the results from hundreds of studies. Experts had long known that the small samples and the varying validities, reliabilities, and restrictions of range in such studies were responsible to some extent for the low, negligible, or unstable correlations. What few realized was how different the picture would look when these sources of error and underestimation were taken into account through meta-analysis.14 Taken individually, the studies said little that could be trusted or generalized; properly pooled, they were full of gold. The leaders in this effort—psychologists John Hunter and Frank Schmidt have been the most prominent—launched a new epoch in understanding the link between individual traits and economic productivity.
THE LINK BETWEEN COGNITIVE ABILITY AND JOB PERFORMANCE
We begin with a review of the evidence that an important statistical link between IQ and job performance does in fact exist. In reading the discussion that follows, remember that job performance does vary in the real world, and the variations are not small. Think of your own workplace and of the people who hold similar jobs. How large is the difference between the best manager and the worst? The best and worst secretary? If your workplace is anything like ours have been, the answer is that the differences are large indeed. Outside the workplace, what is it worth to you to have the name of a first-rate plumber instead of a poor one? A first-rate auto mechanic instead of a poor one? Once again, the common experience is that job performance varies widely, with important, tangible consequences for our everyday lives.
Nor is variation in job performance limited to skilled jobs. Readers who have ever held menial jobs know this firsthand. In restaurants, there are better and worse dishwashers, better and worse busboys. There are better and worse ditch diggers and garbage collectors. People who work in industry know that no matter how apparently mindless a job is, the job can still be done better or worse, with significant economic consequences. If the consequences are significant, it is worth knowing what accounts for the difference.
Job performance may be measured in many different ways.15 Sometimes it is expressed as a natural quantitative measure (how many units a person produces per hour, for example), sometimes as structured ratings by supervisors or peers, sometimes as analyses of a work sample. When these measures of job productivity are correlated with measures of intelligence, the overall correlation, averaged over many tests and many jobs, is about .4. In the study of job performance and tests, the correlation between a test and job performance is usually referred to as the validity of the test, and we shall so refer to it for the rest of the discussion.16 Mathematically, validity and the correlation coefficient are identical. Later in the chapter we will show that a validity of .4 has large economic implications, and even validities half as large may warrant worrying about.
This figure of .4 is no more than a point of reference. As one might expect, the validities are higher for complex jobs than for simple ones. In Edwin Ghiselli’s mammoth compilation of job performance studies, mostly from the first half of the century, a reanalysis by John Hunter found a mean validity of .53 for the job family labeled “manager” and .46 for a “trades and crafts worker.” Even an “elementary industrial w
orker” had a mean validity of 37.17
The Ghiselli data were extremely heterogeneous, with different studies using many different measures of cognitive ability, and include data that are decades old. A more recent set of data is available from a meta-analysis of 425 studies of job proficiency as predicted by the General Aptitude Test Battery (GATB), the U.S. Labor Department’s cognitive ability test for the screening of workers. The table below summarizes the results of John and Ronda Hunter’s reanalysis of these databases.18
The average validity in the meta-analysis of the GATB studies was 4519 The only job category with validity lower than .40 was the industrial category of “feeding/offbearing”—putting something into a machine or taking it out—which occupies fewer than 3 percent of U.S. workers in any case. Even at that bottom-most level of unskilled labor, measured intelligence did not entirely lose its predictiveness, with a mean validity of .23.
The Validity of the GATB for Different Types of Jobs
GATB Validity for:
Job Complexity Proficiency Ratings Training Success % of U.S. Workers in These Occupations
Source: Hunter and Hunter 1984, Table 2.
General job families
High (synthesizing/coordinating) .58 .50 14.7
Medium (compiling/computing) .51 .57 62.7
Low (comparing/copying) .40 .54 17.7
Industrial job families
The Bell Curve: Intelligence and Class Structure in American Life Page 10
