Shorter Papers

Ben Leubsdorf at the WSJ does a great job of covering the discussion within economics over too-long papers, picky editing and refereeing, and other issues.

Defensive writing is certainly part of the issue

“If you want to publish a paper in a top journal, even if you think you have one key insight that can be conveyed succinctly, the referees are not going to take it,” Ms. Finkelstein said.

I think Amy would want to clarify this means referees at other journals. Editors are also to blame. We must remember, referees do not take or reject papers, referees advise editors, and it is always the editor's job to make publication decisions.

From an early stage of an academic career, “it becomes pretty clear that you need to check off a pretty long list of items to really convince people that the way you’re interpreting your results is indeed the right way to do it,” Mr. Bazzi said.

..... When you’re trying to anticipate possible criticisms on a controversial topic like the minimum wage, and situate your research in the deep existing literature on the subject, it “quickly adds up to a long paper,” said University of Massachusetts-Amherst economist Arindrajit Dube,....

Mr. Dube said that paper is now in the process of being revised ahead of publication—including acting on a request to make it shorter.

However, journals don't encourage length, and there is some sense to the current equilibrium.  You write a paper with lots of defensive "what if this what if that." You send it to journals. My typical paper is rejected at 2-3 journals, so by the time it's published I have 6 to 12 reports.  My referees are typically thoughtful and diligent, and the paper grows in addressing all of their what-abouts too. Since I haven't been doing detailed empirical work lately, the requests are not nearly as extensive as those authors receive. Then we finally arrive at publication, and the editor says "now cut it down to 40 pages. You can stuff all that into an internet appendix if you like." Which nobody reads.

This isn't necessarily a bad equilibrium.


Way too many papers are published in the first place. Way too many of them have pretty fatal flaws: didn't the (12!) referees notice the huge selection bias? Simply less reviewing isn't necessarily the answer -- though journals are pretty wantonly using free referee time. And it seems to me that everyone else's papers get better as they go through the process. As a reader, short papers that have been through the wringer are better!

(On the other hand, I did once send a paper that was less than 10 pages to a finance journal only to be told that the journal does not print short papers. The editor invited me to expand the topic and send in the usual 60 pages.)

There is also some selection bias in the remembered history. Yes, there are a few great short papers, and papers that have trouble getting published, have been immensely influential. (In addition to the mentioned papers by Samuelson and Nash, Akerlof's lemons, Bob Shiller's volatility tests -- an AER papers and proceedings with one sexy graph, that would never have been published as a regular paper -- Bob Lucas' Critique stand out in my  mind.)  But there are many other terrible short papers, and I have seen many papers that were confused messes in the first draft get a lot better. The process, laborious as it is, does add value.

The whole discussion raises the question, just what are journals for anyway? Journals started as the internet of the 1700s -- a way to communicate results, using this great technology of printing and paper. The combination of the internet of the 1990s and increasing publication lags mean journals do not have that function anymore. "Publishing" is now a moment of setting a particular work in stone.

I think journals are trying hard to make papers perfect on publication, especially given the replication crisis in social sciences, and the number of prominent economics papers that have fallen apart under scrutiny. There is a vision that it should not be "published" unless it's "right" so that anything "published" is reliably true. I think we need to give up on that hope! Publication is the start of a conversation. Most papers are forgettable. The ones that matter will have others tear them apart. Yes, papers should be not full of obvious problems, but the 70% or more of weight that is what about this and what about that is probably not that useful.

Writers could do a lot to help the whole process. Writers: Get to the point! Why not write the short paper that Amy Finkelstein describes and then tack on all the extra stuff? Don't force the average reader to go through a lot of junk to get to the results.

Massachusetts Institute of Technology professor David Autor compared a 94-page working paper about the minimum wage to “being bludgeoned to death with a Nerf bat” and started a Twitter hashtag, #ThePaperIsTooDamnedLong.

Well, if the paper (I have not read, I'm inferring) had gotten to the point -- the 10 page paper followed by the what-ifs, then most readers would not have had to read 83 pages.

Current papers tend to be structured:

• Motivation -- why you should care. Useless as it's about a result you don't understand yet. 
• What everyone else is doing on the subject and what's wrong with it. Generally impossible to follow unless you already know it already.
• Review of literature, what other people are doing on the topic. Ditto
• Preview of results. But without what you did to get the results, usually useless. 
• If empirical: Data sources and transformations. Plots of data. Preliminary analysis. Cute facts. Simple results. Finally, around table 4, page 35, and an hour and 10 minutes into the seminar presentation, the main result. Then extended what if this what if that. 
• If theoretical: (good papers) A simple example that shows the basic idea, then the main result. (most papers) An incredibly general setup. Then simplifying assumptions, and finally the main result. Then the simple example. 
• Conclusion outlining the researcher's plans for the rest of career
• Long appendix. 

Often this has gone on so long that the actual meat of the paper is stuffed in the appendix. Theory papers don't include proofs or derivations any more! Empirical papers don't include an actual complete specification of the specification. This is the paper. It's what you did. Throw the rest out!

Writers could help a lot by focusing laser-like on what this paper does, in the first 10 pages. Then stuff the rest in the back. Motivate it after we know what it is. Tell us how it fits with what everyone else is doing after we know what it is. (Or leave that to historians). And so forth. All it takes from journals and referees is a little tolerance for unusual organization. That is not easy.

Journals are changing. I think Glenn Ellison deserves a lot of credit for starting the modern wave of self-examination in The Slowdown of the Economics Publishing Process. (Publishing this paper was also one of my proudest moments as JPE editor.) Since then, journals have all been moving to process papers more quickly, to see themselves as making publish or not decisions, rather than endless refinement; a one round and then yes or no ethic is taking hold, desk rejections of hopeless papers are increasing, and editors are becoming much more decisive.

The process is ongoing. Open refereeing and discussion forum are hot topics that I hope will bear fruit sooner rather than later. We are wasting a lot of time on a faux judicial process. The insights of people who have read and thought about the paper, such as conference discussants, are lost in the editorial process; first round referee points are also lost, and a huge amount of effort is duplicated. Achieving such an open forum with a modicum of civility is an important challenge.

Hmm. A post about too long papers has gone on far too long.