Societies Research Libraries // April 22nd, 2014

Striving for innovation: a report from the NAS Journal summit


Last month, the National Academy of Sciences hosted its biannual Journal Summit in Washington, DC.  Formerly called the e-Journal Summit, the meeting gathers innovators from across the world of STM journal publishing to discuss ways to embrace new technology for the benefit of scientific communication. The meeting assembles a unique group of stakeholders including: publishers, editors, researchers (authors), leaders of scientific societies, funders, and policy makers. This year’s meeting was filled with enthusiastic presentations and discussion about how to best improve scientific communication. Below are some highlights from the day’s discussions. 

NAS rotunda3

Dome of the rotunda at the National Academy of Sciences. The inscription reads, “Ages and cycles of nature in ceaseless sequence moving.”

Peer Review Cascades
The morning started with brief presentations by several speakers with experience in different types of peer review cascades. Both cross-publisher and internal publisher and society efforts were discussed.

  • Authors are more likely to accept an invitation to forward a rejected manuscript to a second journal if the transfer of the manuscript and reviews is made easy (i.e. done for them).
  • Authors are more receptive to the idea of referral to another journal if they are asked to name a second choice journal ahead of time.
  • Most internal publisher cascade transfers are predominantly made up of referrals that were rejected without reviews.
  • If asked, 70-80% of reviewers give permission to forward their review with their name attached.

Extra-journal Peer Review
As publishers and others experiment with peer review, a new movement has emerged to decouple the peer review process from the journals that will eventually publish the articles. We heard from Janne-Toumas Seppanen of Peerage of Science and Tim Vines of Axios Review on the progress being made in this area. Some features of extra-journal peer review include:

  • Authors submit to a service or platform that coordinates peer review independent of any journal.
  • Peer review may be open so that anyone can read the reviews.
  • Peer review may be organized around subject-specific editorial boards or dependent on volunteers finding the submission and choosing to review it.
  • Reviewers may be volunteers or paid.
  • Reviewers may be rated on the quality of their peer reviews.
  • At the end of the process, the peer review service may either: recommend journals to submit to, rely on preset author journal rankings, or allow journals to make offers for articles on the platform

Open Access and Data Access
Representatives from government agencies spoke about progress in providing access to research and associated literature. The Clearing House for the Open Research of the United States (CHORUS) was discussed as one method that government research funding agencies can use to provide access to research articles published by their grantees. The next step is to provide public access to data acquired from government funding.  The White House Office of Science and Technology Policy, will be looking to pilot a project to create an international data commons that will collect and index research data. 

Metrics, Measurement of Research Impact, and Data Citation
Can we move on from Impact Factor measurements when considering faculty tenure and advancement?  What alternative forms of evaluation exist?  These were also questions explored at the summit.

  • Funders and University departments are interested in new alternative metrics such social media and news media impact. However, most review panels are not using this information to make granting and promotion decisions at this time.
  • Trackable citation of data and datasets may help to give credit to those whose work is being undervalued in the current system.

While various publishers are taking advantage of the digital landscape, what was most striking is how slowly the behavior of readers, authors, and editors is changing in response to new technology.  One panelist, speaking as a researcher, remarked that he was surprised that none of what we were discussing was truly innovative.  One could imagine a future NAS gathering where the journals discussed are all using open peer review and seamlessly integrated with open data repositories, but at this pace it doesn’t seem like that will be the case at the next Journal Summit in 2016.  Until then, we may have to make do with incremental progress.

  • Robert Dingwall

    Why is anyone surprised that most of the community is not using open peer review and data repositories? What we have here is a highly evolved ecosystem that works very satisfactorily for most scholars because that is what evolution does. There are potentially disruptive technological innovations that have been energetically promoted by a handful of enthusiasts – but the sociology of science has extensive experience of noting how most innovation gets hyped and then founders on a host of practical problems. I have already written here before about how the privacy issues associated with open data, as these affect the social sciences and humanities, have been seriously underestimated by people who only know about chemical entities. Personally, I will not do peer review for any journal that does not operate a double-blind system and I do not have a great deal of confidence in open peer review, which is too similar to ticking a like box on Facebook or favoriting a tweet. I might also observe that it is curious that only movement in one direction is described as ‘progress’ – again very bad social science. No-one has believed in this kind of teleological Whig view of history for more than a generation.

    • Matt Giampoala

      Thank you for your comment, Robert. My last paragraph was mostly meant to emphasize that authors are slow to change and that this will obviously slow the course of change in publishing. I don’t necessarily think that open peer review is the way to go…it was a carelessly written sentence. I agree that publishing has evolved over time to its current state. However, it is possible to evolve to fill a certain niche and then find that you are no longer needed if the market changes drastically. If research funders begin mandating deposit of certain types of research data, I’ll want to make sure my journals are ready to meet the needs of their authors.