Referoo  or  : making scientific peer‑reviewing efficient.
Manage your referee reports

Manage all your peer-reviewing requests, tasks, and history in one central place, whether you are acting as a referee, an editor or an author.

Get Feedback

We will make sure that when you referee a paper, you get notified of the outcome of your work.

Find reviewers

As an editor, quickly find potential reviewers. Find an explicit common agreement on deadlines and feedback.

Reduce work duplication

As an author, forward your reviews from one submission to the next and avoid redundant work for referees.

Smooth transition

Through its continuous evolution, Referoo will promote a smooth transition towards better and more efficient peer-reviewing and publishing models.

The opportunity costs of redundant refereeing

By on May 01, 2018

Although there are no direct costs associated with refereeing tasks of researchers, those tasks incur significant opportunity costs. Time is the most precious ressource for a researcher, and refereeing an article requires their full attention for several hours, time which could otherwise have been spent on original research projects.

Estimating the value of the opportunity costs of redundant refereeing is difficult as most refereeing is done anonymously and confidentially. A referee spends on average over 4 hours to review a paper (1). An hour of work from a reviewer costs at least 20 Euros (2), therefore the cost of one review should be well over 80 Euros. Submitted articles require an average of 2.5 reviews [1], thus the average cost of the reviews for a submission is over 200 Euros.

About 2.5 million articles are published every year worldwide. Weller [3] estimates that 21.6% of all published papers were previously rejected. So out of 2.5 million published articles yearly, at least 500,000 went through the refereeing process at least twice. This of course ignores the large number of articles that were rejected multiple times and those that were never published. Rejection rates of journals vary greatly by field and journal quality, between 19% and 90%. Also in fields (such as Computer Science) where it is customary to submit the same article both to refereed conferences and journals, nearly every article go through the refereeing process twice. For instance, over 150,000 articles are published every year in Computer Science.

The opportunity costs of this work duplication is thus likely well over 100 million Euros per year. Reducing even a small fraction of this duplication would have a considerable impact.

Several proposals have been made to solve the peer-review duplication problem. One such proposal was to make reviews portable between journals through mutual agreements. One notable example is the Neuroscience Peer Review Consortium, which allows an author to forward articles rejected from one journal of the consortium to another. Other similar consortiums have been setup since then, most of them limited to a restricted group of editors or to a field. Axios review, 2013–2017, was another such initiative. Authors submitted their article through the Axios website to a list of journals, reviews were collected by Axios review and the article was then submitted along with its reviews to the journals on the list. Axios then collected a fee for each successfully published article. Unfortunately the system did not succeed in attracting enough submissions. Peerage of science is a similar initiative that proposes authors to get their articles pre-reviewed on their system before submitting to a participating journal. The website provides no statistics indicating how succesful they are. The most recent article advertised on their website was published in 2016.

All those initiatives, while valuable, are limited to individual agreements with certain journals, and give no significant incentive to authors to actually forward reviews if they were negative. Other recent initiatives aim to significantly reform the peer-review process. However, as the most prestigious publication venues are often the most ancient and traditional ones, any solution that would require changing the existing peer-review process of the journal is likely to encounter significant resistance.

Transfering referee reports will be a central feature of Referoo.org, while maintaining strict confidentiality standards. And since Referoo can be used by the referees and editors of all journals and publishers, no special agreements or consortiums are necessary. All actors of the publication process are free to use this feature or not. Authors can forward their Referoo paper ID during their next submission to forward data about their previous submissions, editors get valuable information that they can choose to use to expedite the refereeing process, and referees always get to decide if their information is transmitted. And everyone will get a quantitative assessment of how much work they have accomplished and how much work duplication they have helped avoid.

Footnotes

(1) Tenopir and King (2000) [1] estimate that a researcher needs 3 to 6 hours of continuous work to review a paper. The more recent study from the Publishing Research Consortium [2] reports an average of 8.5 hours per review, with a median of 5. Therefore 4 is a conservative estimate.

(2) The average annual salary of a researcher in Europe before taxes is over 50000 Euros. A year contains about 250 business days, and each business day no more than 10 hours of work on average.

References

[1] Carol Tenopir and Donald W. King. Towards electronic journals: realities for scientists, librarians, and publishers. Special Libraries Association, Washington, DC, 2000.

[2] Mark Ware and Mike Monkman. Peer review in scholarly journals – an international study into the perspective of the scholarly community. Technical report, Publishing Research Consortium, 2008.

[3] Ann C. Weller. Editorial Peer Review: Its Strengths and Weaknesses. Information Today Inc, June 2001.

Introducing Referoo

By on April 16, 2018

The goal of Referoo is to be an amazingly user-friendly web portal for managing the refereeing tasks of researchers. The portal will be designed in such a way as to encourage referees to perform their duties conscientiously and efficiently, and provide them with clear incentives.

Because right now, being an anonymous referee is a thankless job. Referees are asked to peer-review articles, often anonymously, using some cumbersome publisher web interface. Once the job is done, the most they get is a confidential thank you note. Rarely, the reviewer gets notified of the outcome of their work. They do not, in general, receive the final version of the article incorporating their feedback. Of course, referees do get some personal satisfaction from contributing to their field, and from having a first peek at the latest discoveries. But they get very little actual recognition and respect for the tremendeous and crucial work they accomplish.

And this causes significant problems for the other actors of the publication process. Editors have a very hard time getting referees to agree to review papers, or to send their reviews in a timely manner. Authors have to wait forever to get their papers reviewed and published.

Referees are also getting very little in terms of tools, service and convenience to do their job, while several systems exist tailored to the needs of editors (OJS), conference PC members (Easychair) and authors (ORCID, Researchgate). Referoo intends to fill this gap. At the same time, the portal will implement tools to avoid the inefficiencies and the work duplication of the usual refereeing process.

While mainly focused on supporting referees, the portal will provide benefits to all other actors of the scientific community. For journal editors and conference organizers, the portal will contribute a more efficient way to find willing and qualified referees, and a better way to communicate with them. For authors, the portal will provide tools to manage their list of publications automatically and accurately. The portal will also inherently encourage the early diffusion of articles on open access preprint servers.

Through the continuous evolution of the portal and other tools that will be developed, the long term goal of this project is to empower the scientific community to transition towards better and more efficient publication models.

Follow our blog

By on March 14, 2018

Our domain referoo.org, landing page and development infrastructure (DB, backend, frontend) is now live. Follow us on referoo.org to keep up to date with our progress.

We are funded!

By on February 21, 2018

Vietsch Foundation Logo The Vietsch Foundation is a non-profit, funding and promoting research into and development of advanced technology for scientific research and higher education.

The Foundation has studied our project and decided to fund referoo.org, which means that you can expect to see our progress in the forthcoming months!

is funded by and developed by teams at