Here is a nice article on how to write good referee reports on journal articles. The article is by Henry Roediger, a distinguished psychologist at Washington University in St. Louis (thanks to his colleague Jeff Zacks for the pointer), so it focuses on psychology. But mutatis mutandis virtually all of his advice can be tranferred to philosophy refereeing.
Two minor additional comments:
1. I think in philosophy (unlike psychology) it’s very important to always begin a referee report with a summary of the paper, to make it clear what the referee thinks (rightly or wrongly) is the main thesis and argument of the paper.
2. I like the advice to sign referee reports (so the author knows you wrote it) but you might want to wait until after you get tenure before you make this a habit.
I agree, excellent article.
Today’s journalism is so often “gotcha”, that I’m sure many people extend that to reviews.
Not sure that the reviews should be signed, don’t most journals enforce or at least encourage anonymity anyway?
Anonymity is the default, but if the refereee wants to make herself known there is no reason (or rule) against it.
Levels of anonymity are also viewed as levels of standards for refereeing: so a double-blind review is the gold standard. If the referee lets the author know during the process of refereeing, that affects the level. It may also put the author in something of an awkward position: if journal says double-blind, but the referee signs, then should the author assume the referee knows the author knows, or not?
Anonymous refereeing is a protection for the referee, so the referee is free to give it up. If that happens, the author has no reason to assume that the referee knew her identity. (Although remember that lots of referees know the identity of authors, either because the refereeing is not blind or because they figure out who the author is.)
I join. All above told the truth. Let’s discuss this question.
It is remarkable, very good piece