January 27, 2005
How not to refute Larry Summers

From a news follow-up article, “Do genes play a role in science gender gap?”, by Carolyn Y. Johnson of the Boston Globe:

Summers was faulted by some conference participants and others for drawing a link between test scores and career success -- a link that they said has been disproved by research.

Really? Disproved? Research has shown that there’s no link? This is research I’d like to see.

Yu Xie, a sociology professor at the University of Michigan who spoke at the conference, has shown that men outnumber women 2-to-1 in the top 5 percent of scorers on math tests -- research that Summers cited in his talk. But Xie said in an interview that his studies show that not all the best scorers succeed professionally, and not all men with successful science careers received the top scores.

Sorry, those aren't the research results we need to show that there’s no link. Of course not 100% of the top-5% scorers end up in the top (say) 10% of their fields. But surely more than 10% do? What is the percentage? Of course not 100% of all men with the most successful (top 5%) careers are among the top 5% of test scorers. But surely more than 5% of the most successful scientists are? What is the percentage? I would find it hard to believe that there’s no link, that the top 5% of test scorers are not more heavily represented than others (are not more than 5% of the population) among successful career scientists. I await the evidence to the contrary.

Let me borrow an analogy from David Bernstein of The Volokh Conspiracy. Suppose a hypothetical Larry Winters says that perhaps, in part, goyishe men dominate the National Basketball Association because there is a link between height and basketball success, and goyishe men are more heavily represented among the very tallest. The analog to Xie’s comments would be: “But wait! Not all the tallest people have successful NBA careers! And not all successful NBA players are above the 95th percentile in height!” True and true, but beside the point. Less-than-perfect correlation doesn’t show that height doesn’t matter, or that there’s no link between height and professional basketball success.

Xie and Kimberlee Shauman of the University of California at Davis, who coauthored ''Women in Science: Career Processes and Outcomes," concluded that no single factor could explain why women did not make it into the most elite science positions, and that a combination of social, cultural, and psychological factors were probably at work.

Again, not the evidence we need. Lots of other factors also matter, sure. Test scores are not a perfect predictor (do not have an r-squared of 1.00), sure. That doesn’t show that test scores have no predictive value, other things equal. Or even that test scores aren’t a better predictor than any other single factor or combination of factors.

To be fair to Prof. Xie, please notice that he isn't quoted as saying that the existence of a link has been disproved. That claim the reporter only attributes to an unnamed "they".

Posted by Lawrence H. White at 03:54 PM  ·  TrackBack (5)

The statesman who should attempt to direct private people in what manner they ought to employ their capitals would not only load himself with a most unnecessary attention, but assume an authority which could safely be trusted, not only to no single person, but to no council or senate whatever, and which would nowhere be so dangerous as in the hands of a man who had folly and presumption enough to fancy himself fit to exercise it. -Adam Smith

Our Bloggers
Joshua Hall
Robert Lawson
E. Frank Stephenson
Michael C. Munger
Lawrence H. White
Craig Depken
Tim Shaughnessy
Edward J. Lopez
Brad Smith
Mike DeBow
Wilson Mixon
Art Carden

Blogroll

Search

Archives
By Author:
Joshua Hall
Robert Lawson
E. Frank Stephenson
Michael C. Munger
Lawrence H. White
Edward Bierhanzl
Craig Depken
Ralph R. Frasca
Tim Shaughnessy
Edward J. Lopez
Brad Smith
Mike DeBow
Wilson Mixon
Art Carden

By Month:
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
February 2005
January 2005
December 2004
November 2004
October 2004
September 2004
August 2004
July 2004

Powered by
Movable Type 2.661

Site design by
Sekimori

XML