The current Review of Financial Studies’ editor Matthew Spiegel recently wrote a brilliant piece for the recent RFS March 2012 edition titled “Reviewing less — progressing more”. This editorial piece concerns recent and worrying trends in the publication process of academic finance (and probably other) studies.
Presumably, academic journals exist and publish articles to disseminate new ideas. Somehow that simple goal has been lost. Today, articles appear in print only after a referee is convinced that all other alternative explanations for its results have been ruled out. In reality, no article can exclude every possible alternative, so this is basically an exercise in futility. The criterion for publication should be that once an article crosses some threshold it is good enough to publish. The problem seemingly lies in our
inability to say “good enough.” But this is a problem we can fix.
This is a must read! (I particularly adore the last paragraph on page 8 and its following on page nine…“What if an article is important—that is, people read and cite it? In that case, academics will dissect its every aspect. Some will examine the article’s data filters; others will check for coding errors; still others will look for missing factors or reverse causality explanations. The list is endless. But, that is the point. The list is endless. Authors, referees, and editors cannot even scratch the surface. Nor do they have to. The fact that our colleagues will stress test any important publication means our profession’s received canon of knowledge has a self-correcting mechanism built in. We have faith that important articles are “right” because their results have been tested over and over again in myriad ways. Instead of trying to preempt this by demanding authors perform ever more tests, we should instead just let the academic process work by publishing articles if they are likely to be interesting. After that, let the profession determine if the author’s conclusions are accurate—or rather, that they are interesting enough to even make checking for problems worthwhile”)