My economic researcher friend, who works at another think tank I won’t name (because it’s already named the Urban Institute), writes the following about the Lott Affair:
I don’t know if it’s that much better to publish a number that you know is statistically meaningless–and it is meaningless if you can say 98% isn’t significantly different from 38% or even from 80%–than to totally make numbers up. Just because you made the effort to conduct a survey doesn’t mean you can publish its invalid result. The only way that 80% could fall within the margin of error around 98% is if the sample size was way too small (or if the standard error is really big, but since the data is a simple “yes” or “no”, this can’t be the case). This is why we have standards in statistics! Sorry about my ranting, but it’s stuff like this that makes people who don’t really understand statistics feel justified in dismissing carefully conducted research because “you can make statistics say anything.” Well, yeah, but only if you’re dumb or purposefully deceitful. And since I sit around all day doing research…well, it’s frustrating. So maybe Bellesiles is worse because he fabricated more stuff, but what Lott did is equivalent to pulling numbers out of the air. Okay, I’ll stop.
Her comments are, of course, strictly her own.