Thursday, April 02, 2015

Of course, Science has its own issues...

Peer review?  What peer review?  We have algorithms!!!
The SCIgen scandal shows that even old-fashioned big-budget scientific publishing houses are sometimes faking the process of scholarly peer review.

The SCIgen algorithm is more than a decade old now; written by MIT students, it strings together nonsensical phrases larded with computer science buzzwords, slaps on a graph or two, throws in some phony references ... voila! You’ve got a reasonable facsimile of a comp-sci paper. Reasonable, that is, if you know absolutely zilch about computers and have no reading comprehension skills. Take, for example, this introduction to a SCIgen paper:
The development of congestion control has synthesized checksums, and current trends suggest that the exploration of scatter/gather I/O will soon emerge. The notion that analysts connect with compilers is usually well-received. The notion that biologists collude with 802.11b is usually considered robust. However, simulated annealing alone cannot fulfill the need for the construction of 802.11b.
It’s not just you; nobody on the planet can figure out what that paragraph means. And the more you know about computers, the less sense that passage makes. You might not know that 802.11b is a standard for wireless networking, but if you did, you’d have a hard time figuring out how biologists could collude with it. Checksums aren’t synthesized, and if they were, they certainly wouldn’t be synthesized by the development of congestion control. Pretty much every single sentence in the entire article is a meaningless hash of jargon. The likelihood of a real computer scientist being fooled by a SCIgen paper is roughly the same as that of a classicist mistaking pig Latin for an undiscovered Cato oration. No peer reviewer could ever have let it slip by.
Yet in 2007, that particular SCIgen paper, “Cooperative, Compact Algorithms for Randomized Algorithms” was accepted by and printed in a “peer-review” journal: Applied Mathematics and Computation, published by Elsevier, one of the giants of the scientific publishing industry.

No comments:

Post a Comment