Doing Economics

6.4 Other Types of Peer Review
There are miscellaneous other kinds of peer review you are likely to be
asked to do, such as reviewing conference submissions, grant proposals,
book proposals, and the manuscripts of friends and colleagues who solicit
your advice.
Organizing a conference, however small, is a lot of work, with the
organizing committee having to make decisions ranging from lofty
academic ones (e.g., whom to invite as keynote speaker) to resolutely un-
academic ones (e.g., what vegan option to offer for lunch on the second

---

day). Because of that, organizing committees will often delegate the task of
selecting papers to a group of reviewers, and so you are highly likely to get
asked to review papers for a conference at some point in your career if you
have not done so already. When reviewing papers for conferences, you will
either get asked to review abstracts (often so-called structured abstracts, i.e.,
two-page summaries of the papers submitted) or full papers. The task of
reviewing papers for a conference is substantially easier than that of
reviewing manuscripts for journals, and I believe any submission can be
assessed in 15 minutes or so by asking yourself a few questions. First off, is
the paper asking a research question that is of interest to those who will
attend the conference (for a field conference like, say, the Society of Labor
Economists’ annual meeting) or a research question that is likely to make
attendees attend the particular session it is slotted in (for a general
conference like, say, the Western Economic Association International’s
annual conference)? Second, does the paper seem competently done? In
other words, if you were to present this paper instead of the authors, would
you be embarrassed by the execution? Finally, and this is a distant third, are
the findings likely to stimulate discussion? Most conferences will ask you
for an “up-or-out,” “accept-or-reject” decision, and even when a conference
you may review for asks for specific reasons behind such a decision or
comments to the authors, those should be kept short.
I had much more to say about getting grants in the previous chapter, but
if you are someone who relies on external funding for your research, you
should expect to do a certain amount of reviewing for funding agencies and
donors. Typically, you will usually get asked to review a single proposal at
a time.
Less typically, you might get paid for your review. The pesky thing about
grant proposals is that in this interest of minimizing review heterogeneity,
funding agencies and donors tend to give you a scoring sheet which you
have to fill out, and they often ask you questions you might feel less than
ideally qualified to answer (e.g., “Are the investigators realistically
assessing the various costs of undertaking this research project?”). To make
things worse, the documents you need to review are often spread out across
several files. When reviewing grant proposals, my rule is to judge what I
feel qualified to judge, to provide comments that will aim to improve the
investigators’ methodology (since this is all ex ante of them doing the
research, you can afford to send them back to the drawing board), and to

---

confess my lack of qualification when it comes to the things I cannot
realistically assess. Ultimately, funding agencies and donors are interested
in knowing whether you think the authors are going to spend money on
something the funder deems worthwhile. More specifically: Are the authors
asking an important research question? Are the proposed methods suitable
to answer it? Is the requested amount of money likely to allow them to get
there? This is thus not the occasion for you to focus on details like standard
errors or your favorite semiparametric estimator.
Another form of review wherein you will usually be given a rubric to
assess the work at hand is when you review book proposals. Academic
books such as the one you are currently reading go through peer review, just
as journal articles do. When I submitted this book to MIT Press, it was sent
to two academics, who were then asked to assess whether the press should
agree to enter into a contract with me for this book. After I did sign the
publication contract with MIT Press for this book and submitted a full draft
of it, that draft was once again sent out to two academics to solicit their
comments on the full manuscript. Usually when you are asked to review a
book by a press, you will be sent a proposal, a few sample chapters (chapter
2 was the sample chapter for this book), and a list of questions about the
proposal. Book proposals tend to be more fun to review than manuscripts
for journals or grant proposals, because they get you to think about different
things than when you review journal articles. The things you will likely be
asked to comment on are the proposed book’s relevance to the field or the
discipline, which other works do you see as competing with it, the author’s
qualification, whether you can foresee the book being adopted in university-
level classes, whether you think the book would sell well, and so on.
Because each press is its own entity, none of these things is set in stone. For
instance, you will sometimes get paid for reviewing book proposals. More
commonly if the author is a young academic with little to no track record,
you may be sent the draft of a whole book instead of a few chapters. And of
course, the exact questions you will need to answer will vary from press to
press, and between academic and commercial presses.
The last kind of reviewing, which is something you will get asked to do
starting from early on in your career is “friendly reviewing,” when friends
and colleagues solicit your comments on a working paper before submitting
it. This is probably the most impactful kind of reviewing, if only because it
usually has you review the work of someone whose career success you are

---

invested in, and because it usually has you review an early version of a
paper which, assuming increasing and concave returns to comments, means
that your comments have a higher expected return at the margin. If you are
reading this book at any time after the second year of your doctoral studies,
you have almost surely been asked to provide friendly comments. Here,
there is not really any set standard. When friends and colleagues send me
their papers, unlike when I write a review for a journal, I usually write my
comments in an email, in order of appearance in the paper first. Then before
I click “send,” I usually go back to the beginning of the email to add some
general thoughts if I have any. When writing a friendly review, anything is
fair game.
Ultimately, a lot of the reviewing you do will be thankless, because you
will not get paid for it, and because the authors will never know who their
reviewers were. At best, you will be thanked as an anonymous reviewer and
get an automatic (and perfunctory) email from the journal’s editorial system
informing you of the editor’s decision and thanking you for your service.
7
Yet even a purely selfish individual should recognize the value of
reviewing to their career. Over the years, I have found that besides learning
by doing (i.e., writing your own papers), one of the best ways to improve as
a researcher is to learn from others. Obviously, this means that you should
read good papers—-but not only good papers, as I discussed in chapter 2.
Many economists see refereeing as an unfortunate tax they need to pay to
get their own papers reviewed and published. Unlike a tax, however, there
is almost always something to be learned from reviewing—and from
reviewing widely.