The term peer review is used to describe a system whereby a paper is scrutinized by people who were not involved in its creation but are considered knowledgeable about the subject. 1 So it should be considered an evaluation by an expert on research of other experts in the same field.2 Although well recognized, unfortunately this technique is not formally taught but may improve with practice.

In the past 50 years the use of peer review has become the “gold standard” by which biomedical journals judge their papers. The first description of peer review took place in 1731 with a report from the Royal Society of Edinburgh.3 In 1893, the British Medical Journal and its editor Ernest Hart were among the first to implement a formal peer review system. However the necessity to implement this concept into medical journals took a long time, and only after the Second World War was peer review developed.4 With the specialization of medical journals many editors recognized this need and in 1986, Drummond Rennie 5 in response to a letter in the New England Journal of Medicine6 decided to organize a congress on Peer Review in Biomedical Publication. After this meeting in 1989, subsequent conferences took place adding to the considerable knowledge that we have today on this topic.7–12 As a result, research into the editorial processes has evolved and now became evidence based editing.13 Moreover the World Association of Medical Editors (www.wame.org) founded in 1995, was also created with the aim to foster international cooperation among and education of medical journal editors as well as to promote peer review as a quality assurance in medicine.14

There is now evidence that peer review increases the quality of articles 15 and editors rely on peer reviewers to guarantee the appropriateness and scientific quality of the manuscripts they publish. However there is subjectivity in their role as demonstrated by studies showing lack of agreement between reviewers 16 and nationality biases (with US reviewers having a significant preference for US papers). 17 It is important that editors use grading instruments to assist in peer review and make formative assessment of their editorial competence. 18 Moreover use of several reviewers can also dilute the effects of a biased reviewer.

It has been shown that the reviewers that produce the best-quality report tend to be younger, work at top academic institutions or are known to the editors. 19 More recently, another study has come to the same conclusion and has advised on recruitment of reviewers among those with training in epidemiology or statistics, and near 40 years of age. 20 Moreover spending more than 3 hours on a review did not increase review quality 19 and written feedback to reviewers (other reviewers' reports and the editor's decision letter) produced no improvement in performance.21 Of course one basic rule is that we should select reviewers who know the subject content of the work!

Although half-day workshop training did not improve subsequent review quality scores in average reviewers, 22 e-learning based and more intensive programs may be better and warrant investigation.

Most journal editors inherit a database of reviewers within different areas of expertise that can be expanded by identifying researchers with similar articles cited in MEDLINE. New electronic platforms allow the editor to track deadlines and record the performance of reviewers.

Typically Journal editors choose 2–3 reviewers, however having 2 or three reviews does no seem to change the rejection rate. 23

There is some general belief that masking the reviewers to the identification of the authors may improve quality of peer review. Apart from not being easy to do as units can be readily identified, the success rate is low 12,24 and the effect is negligible. 25 More recently some journals started to implement open peer review (where the identities of the author and the reviewer are known to each other) and the results are encouraging 26 although more research is needed on the value of open review and also if this inhibits especially younger reviewers from taking part.

Concerning reviewer selection, some journals also invite authors to suggest up to four suitable peer reviewers for their work. Indeed, examining the submission of original papers to Thorax, Hurst et al 27 have shown the outcomes were not much different though author selected reviewers tended to be more positive, the first decision was more likely to be positive and discordance with the editor's final decision was significantly higher.

The editors should choose reviewers to obtain a balance between content expertise, methodological expertise and educational relevance. 28

The reviewer has technical as well as ethical duties. Eight of the most important duties are summarized in Table 1 (adapted from ref 29).

So, the reviewer apart from wearing the hat of the journal's advocate should also wear the hat of the authors' advocate. 29

In a study performed in 1992, including the top 100 US journals, it was concluded that the review process was not uniform!30 So establishing uniform editorial practices is a major task!

Good editorial practice is well defined in the “Uniform requirements for manuscripts submitted to Biomedical Journals”.31 Every journal should accept, apply and follow good editorial practice (summarized in Table 2).32

Editorial decision-making should be directed at selecting the best manuscripts and those that better match their readership.

The job of a reviewer is to assess the validity and importance of the work in the manuscript. The reviewers' reports will inform the decision of the journal editor that has the responsibility to accept or reject the paper. Reviewing a manuscript still remains a process based on the experience and personal background of the reviewer, as there is insufficient evidence to establish firm rules or recommendations. Nevertheless, different authors have put forward advices and practical tips based on experience. 2,33,34 Some of these practical tips are summarized below. While mostly directed to inexperienced reviewers these tips may drive experienced reviewers to critically reassess their practice. Also, authors can take them in consideration when planning, conducting and reporting their studies.

A manuscript may be rejected at editorial level, before the editor sends it to reviewers. Often, the reasons for immediate rejection are being inappropriate for the journal's readers (wrong journal), not fitting ubmission. The paper will not be accepted if it addresses a topic outside the scope of the Journal or is in a style/format completely different from the rest of the Journal's content.38

After peer-review, the decision on the paper will depend on the comments presented by the reviewers and the editor's judgment about the priority for publication of the manuscript.

A few studies assessed the reasons for rejection of manuscripts in peer-reviewed journals.28,40–42

A summary of these reasons is presented in Table 4. Most of these reasons can be corrected by the authors when revising the manuscript. While unrelated to the study quality, poor writing style can have a strong influence on the overall impression of the manuscript by both reviewers and editors. The main issue in poor writing is difficulty in following the logical flow of the manuscript rather than grammar errors or language issues.

An important reason for failure to publish a paper is not revising and resubmitting the manuscript after the peer review. Too often authors give up after the first rejection or just chose another journal to submit the manuscript. A second submission to the same journal, after careful revision of the paper (based on the reviewers' and editors comments) will substantially improve the quality of the paper.

The process of writing the review report is quite personal. As a reviewer gets experience establishes his/her own routine. We find the personal example described by Frederic G. Hoppin, Jr. very useful to the beginner.43

Today, the reviewing process usually starts with an email from the editor with an invitation to review a manuscript. General details of the paper are included in the email. Also, the time frame to conclude the review and a link to accept or refuse are provided. Before having access to the full paper, the prospective peer reviewer should consult with the editors about any potential conflict of interest. If the invitation is accepted, access to the full manuscript is provided. At this time, the reviewer should reassess if the full text of the manuscript changes the decision to accept (is the content different from the abstract? Is there a conflict of interest previously undetected?).

Before starting writing the review report a number of general questions help the reviewer to appraise the manuscript.44

• –

  Why was the study done? Is it important?

• –

  Have the authors adequately reviewed existing research?

• –

  Does the work add enough to what is already in the published literature?

• –

  Was there a clearly defined question?

• –

  Was the design right for the question?

• –

  Was the study ethical?

• –

  Are the conclusions justified?

• –

  Is there a clear message?

• –

  Is it written in a clear, appealing style?

• –

  Is this paper of interest to the readers of this journal?

In addition to these general questions it is very helpful to use specific checklists available to assess each study design. The EQUATOR network keeps updated resources on checklists and guidelines on reporting medical research literature.45

The review report is now usually performed online in a web application that often includes a review form, confidential comments to the editor (not available for the authors), the recommendation to the editor to accept or reject the paper and comments to the authors.

Most journals have a checklist or a form to be filled by the reviewer about the manuscript quality and its suitability/priority for publication in the Journal. This form aims to collect information more objectively and can help the reviewer to organize his/her opinions about the manuscript.

The “confidential comments to the editors” are a section of the review report not accessible to the authors. This is an opportunity for the reviewer to emphasize his/her views on why the manuscript is appropriate or not for that specific Journal. These comments should not repeat what was stated in the comments to the authors but rather to provide information to the editor to help his/her final decision.

The comments to the authors include the following:

• 1.

  A brief, one-paragraph summary with the reviewer's interpretation of the work. This helps the editor to remember the essence of the manuscript and ensure the authors the reviewer understood it.

• 2.

  The reviewer's recommendation to the editor; in general a manuscript can be considered acceptable to publication as is, with minor (optional) changes or with major (mandatory) changes or considered unsuitable for publication. If rejected, the reviewer can suggest the manuscript may be resubmitted to the same journal after correcting the problems identified. A useful global rating of the manuscript is: Accept, Accept Pending Revisions, Reconsider After Major Revisions, and Reject.36

• 3.

  General comments. Mostly on the quality, importance and novelty of the manuscript. For example, “The study design is not adequate to the research question” or “The manuscript is well structured and written in a clear manner”.

• 4.

  Specific comments. These are related to a particular part of the manuscript. For example, “The number of participants that completed the study are 86 in table 2 and 89 in the last sentence on page 2, paragraph 5. Please clarify”.

It is useful to address the issues in the manuscript in order of importance. Both general and specific comments can be organized in major and minor comments. Major comments include all the aspects that the reviewer feels need to be addressed before the paper is ready to be published.

A reviewer's comments should be direct, constructive and written as clear suggestions or observations. Avoid asking direct questions. These may result in answers by the authors without actual changes in the manuscript. Also, questions can have more than one answer making the authors uncertain on which answer the reviewer intended. The peer review is about the manuscript not the persons or groups who wrote it. Comments about the authors are inappropriate and do not contribute to the aims of peer-review. Also, the reviewer should try to minimize the influence of knowing the authors on the tone and contents of his/her comments.

Although peer review is not perfect and reviewers have a poor detection rate of errors in manuscripts,46 it is the only available method to improve the quality of published papers. Until now nobody has produced a satisfactory alternative to it! The “gold standard” for the quality of any paper remains time-whether it survives a dozen years to be incorporated into review articles or textbooks.47

Journal editors have to continually audit their procedures and apply the results of others to their own practices.47

Reviewers receive very little preparation for performing reviews as part of their formal education, and short training interventions do not seem to improve their performance.46

However in these times of materialism, it is encouraging that there are large numbers of professionals who are willing to offer many hours of their time to work without financial incentive! Peer review is an important service to the Medical and Research Communities. Participating in this process is valuable, voluntary work and, for the reviewer, is also an enjoyable task (most of the times at least).

In the case of the Portuguese Journal of Pulmonology we have to keep the tradition of respect, collegiality and empathy in all interactions during the Peer Review process. We have to feel honored and privileged to be selected as reviewers and to have the opportunity to interact constructively and make the work well.

As Bruce Squires stated about the creation of World Association of Medical Editors: “the fundamental purpose of medical journals (and their editors) should be to promote the science and art of medicine and the betterment of health”.14