Name:
COVID-19 Changes in Scholarly Communication: What Pandemic Changes Will Result in Permanent Changes?
Description:
COVID-19 Changes in Scholarly Communication: What Pandemic Changes Will Result in Permanent Changes?
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/d99f50d4-2a01-4c88-ae64-d1f5c18d823f/thumbnails/d99f50d4-2a01-4c88-ae64-d1f5c18d823f.png
Duration:
T00H59M59S
Embed URL:
https://stream.cadmore.media/player/d99f50d4-2a01-4c88-ae64-d1f5c18d823f
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/d99f50d4-2a01-4c88-ae64-d1f5c18d823f/AM21 Session 2C - COVID-19 Changes in Scholarly Communicatio.mp4?sv=2019-02-02&sr=c&sig=wsYKKfO%2BJu%2BMjRWHJJ51isvjCKrj%2BG%2Bof%2BsmEm0x%2FQg%3D&st=2024-09-08T00%3A31%3A09Z&se=2024-09-08T02%3A36%3A09Z&sp=r
Upload Date:
2024-02-02T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
Aries Systems: Aries Systems is a leading technology workflow solutions provider. Aries is highly customizable, flexible, and innovative workflow solutions to help transform the way scholarly publishers bring high value content to the world. Editorial Manager is the leading cloud-based manuscript submission and peer-review tracking system chosen by thousands of scholarly journals worldwide. Editorial Manager streamlines the editorial process for authors, editors, and reviewers and works seamlessly with production tracking systems, such as Aries ProduXion Manager to create a true end-to-end solution.
Aries Systems: For more information or to request a free demonstration visit ariessys.com
JASON POINTE: OK. Hello and welcome to this session, COVID-19 Changes in Scholarly Communication. What pandemic changes will result in permanent changes? My name is Jason Pointe, I'm the moderator for this panel. The COVID-19 pandemic has had an effect on every segment of the scholarly research continuum. It has affected the way journals approach their review processes, has increased the number of researchers using preprints, has had consequences on research or phone activity in terms of papers published and used for-- COVID-19 researches have had to sift through huge paper volumes, and indexers are looking towards different curation and sharing approaches for COVID-19 research.
JASON POINTE: Our panelists represent several key perspectives as we ask 1-- how have the pressures of the pandemic led to changes in how they do what they do? 2-- what aspects of these changes are likely to continue post pandemic? And 3-- do they think these changes are good or bad? Our first speaker will be Dr. Miriam Sabin, Senior Editor at The Lancet. And she will speak about the effect of the pandemic on submissions, reviewers and reviews, and more that's leading journal.
JASON POINTE: Then Dr. Bahar Mehmani, Reviewer Experience Lead at Elsevier, will discuss recently published study on the differences between how the pandemic has affected the publishing reviewing activities of men and women. Shirley Decker-Lucke, Content Director at SSRN, a preprint server, will provide an overview of the use of preprints in the dissemination of COVID-19 research and the changes the pandemic has brought to the medical preprint landscape.
JASON POINTE: And finally, Katy Funk, Program Manager at PubMed Central, will talk about the special work PMC is doing with COVID-19 research collections using journal articles and preprints. During the course of the presentations, please submit your questions via chat in the path of the platform. The panelists will address them during the Q&A following their presentations.
JASON POINTE: And with that, we'll go to Dr. Sabin.
DR. MIRIAM SABIN: Thank you very much. So I wanted to talk to you a little bit today about what I'm calling a journal of the plague year at The Lancet, which was the title of a book that was written in London in the 1600s. Next slide, please.
DR. MIRIAM SABIN: Thank you, Jason. Thank you. Wonderful. So there was a book written by Daniel Defoe, a very well known book about one of the major plague episodes that happened in the 1600s. It happened worldwide but, of course, in London, a city of great population, it was particularly terrible, and a lot of people sort of reflected on that this year, because there were so many parallels to what happened there in regard to that pandemic as happened last year, last spring when we were first learning about the COVID epidemic.
DR. MIRIAM SABIN: And so here we were-- The Lancet first started in the 1800s in London, and then there they were in The Lancet London offices early January 2020 receiving desperate messages from colleagues in China talking about what was happening, and The Lancet published one of the first papers looking at the clinical features of patients infected with this novel coronavirus. And in fact, our editor-in-chief, Richard Horton, has said that this was really an attempt to try to say to the world wake up, there's something happening here, we don't need to be sleeping now, because in another month or two, it's going to be too late.
DR. MIRIAM SABIN: And that was the warning that was given 1600s, and then again in 2020 here we were. Next slide, please. Thank you, Jason. But the amazing thing is first to reflect on a little over one year later on the cover of The Lancet of about a week or so ago was a report actually that I was fortunate to be editor for an article looking at vaccination in Israel, which only used one vaccine and has been able to serve as a natural experiment in terms of how the vaccine is efficacious.
DR. MIRIAM SABIN: And a year later, here we are talking about eventual control of the SARS-CoV-2 outbreak as vaccination programs ramp up across the rest of the world. But a lot has happened in that last year, and I'd like to look a little bit with you about what we did at The Lancet during that time. Next slide, please. So how have these pressures led to changes in how The Lancet works?
DR. MIRIAM SABIN: And I'd like to first say that when I talk about The Lancet, we have the Lancet weekly, and that's the original Lancet. And then we have The Lancet family journals, which are a number of other journals. So I'm talking sometimes specifically about the weekly and sometimes across the journals. And then we also have Lancet or Preprints with The Lancet on SSRN, which I will also refer to. If you look at the graph, you can see that, it looks like actually one of these COVID graphs that we see now, the tremendous uptake in papers that we saw at The Lancet weekly, the flagship of weekly, submissions starting from 1st of January in 2020 and then going through all of 2020.
DR. MIRIAM SABIN: Lancet already receives a tremendous amount of submissions, as you might guess. And so this really represented almost like a type of wave of submissions, which was also really heartening, because it meant that so many researchers around the world were snapping into action and wanted to share the work that they were doing to help stem the tide of this pandemic.
DR. MIRIAM SABIN: If you look at the other side of this slide at Preprints with The Lancet, just to show you a little bit in 2020, we had about 232,370 downloads and nearly 2 million abstract views. And you compare that in 2019, there were about 35,000 downloads and about 154, 572 abstract views, a huge difference. The increase is in part likely also due to the greater awareness of the preprint server.
DR. MIRIAM SABIN: And I think the COVID epidemic has also led to greater awareness of this server on SSRN, but certainly a large part I can attest to that, because I look at the papers related to the tremendous amount of COVID research. Next slide, please. So I wanted to mention, every organization, every journal that we know about these days has a motto, has a vision statement, and The Lancet it's, "The best science for better lives." And our manifesto is the highest standards for medical science, improving lives is the only end goal, and increasing the social impact of science.
DR. MIRIAM SABIN: And we like to talk a lot about the importance of accountability at The Lancet, but, boy, this was a moment to share this commitment on a global scale to support the fight against COVID-19. And we really felt it, because there were times when people in Congress who were holding up a copy of The Lancet, whether it was an editorial, or it was a piece of research in a presentation being led by Tony Fauci, or by leaders in the UK, or all around the world there was pressure on us, of course, because we realized that what we put out there would really make a difference.
DR. MIRIAM SABIN: Next slide, please. So we recognized early on then as a result that we needed to employ extraordinary measures to support scientific sharing. So these are some of the things we did. We first immediately triaged prioritization of COVID-19 papers across all functions in all Lancet family journals.
DR. MIRIAM SABIN: And The Lancet does many other things, we do commissions, we have a series of papers, we have a number of other pieces, but like many of the researchers that we work with, we all had to put a lot of that aside to address this new virus that we didn't know much about. And we also recognized that we had to ensure the same level of detail was paid to all stages of the paper's journey from submission to publication as was done prior to the pandemic.
DR. MIRIAM SABIN: And that has been really, really critical. We also had outstanding commitment by peer reviewers who just really stepped up to the challenge of reviewing a number of really important papers through our fast track process in which case, they have sometimes, in regard to this pandemic, maybe no more than 48 hours to look at a paper, to turn around revisions, to get pieces out, whether that was in vaccine research or clinical trials.
DR. MIRIAM SABIN: We made all our COVID content free to access. We created a global hub for that, that is available so that people could easily look at our content. And with Preprints with The Lancet on SSRN, that became really very clearly an important place to rapidly disseminate some of this new information. And in fact, the UK has done this very well through one of their scientific centers, where some of the leading researchers there on vaccines and so forth were able to get research up on the preprint server to improve it, a great deal prior to its being ultimately peer-reviewed and then accepted.
DR. MIRIAM SABIN: And I think they found that a very positive experience. Next slide, please. So we also made some really practical-- we strengthened some editorial practices, because we realized also in a pandemic comes a lot of work. You may also have some researchers who are being as careful perhaps as they normally would or trying to perhaps capitalize on a situation like a pandemic, as we see whether it's from in India with various selling of oxygen canisters too, it could be related to an article.
DR. MIRIAM SABIN: So we strengthened some practices, such as that more than one author must take responsibility for having access study data, and verifying it can only have one author. These authors are now named in the contributors statement. So it's very clear who had access. For research articles that are the result of an academic and commercial partnership, one of the authors named as having accessed and verified the data must be from the academic team, they can't be from outside, they can't be from some company that's looking to profit off the pandemic or whatever.
DR. MIRIAM SABIN: All authors must sign a statement saying that they have full access to the data reported and accept responsibility for submitting the article for publication. It's not OK for authors to say, well, I had my name on it, but I didn't really look too closely. I didn't do too much of the work. So I don't really know if there was an error. All research papers, irrespective of method, must now also include a data-sharing statement that details what data and what documents will be shared.
DR. MIRIAM SABIN: And also new requirements for papers, we now have, that use large real world datasets, data sets that may be new, that many reviewers may not be aware of, and for reviewers who are familiar with that data set are also needed. Next slide, please. And so will changes be made during the 2020, that happened during 2020?
DR. MIRIAM SABIN: And so, yes, they think the really positive. And we think that the COVID-19 pandemic has made us more resilient, agile, and stronger to deliver the best signs for better lives. So thank you.
JASON POINTE: OK. Thank you very much, Miriam. I will stop sharing, and our next speaker Dr. Bahar Mehmani.
DR. BAHAR MEHMANI: No, can you hear me? OK.
JASON POINTE: We can.
DR. BAHAR MEHMANI: Thank you so much. So what I'm going to present today is the study that we ran earlier this year, Elsevier journals, to see how COVID-19 lockdown measures in the first wave of pandemic has impacted women and men. And this is a study that we did in collaboration with academics, Professor Flaminio Squazzoni who is a social scientist from the University of Milan, Professor Francisco Grimaldo is a data scientist from University of Valencia, and Professor Giangiacomo Bravo from University of Linnaeus in Sweden who is a statistician.
DR. BAHAR MEHMANI: And he had previous collaborations namely, the study of gender bias in peer-review across journals. And based on our previous studies we thought, well, we hear early on in April in social media and true compacts, journal editors talking about a drop in the number of submissions coming through their journals by women. Maybe we can dig into Elsevier journal data and then see whether there is a clear evidence in statistically robust analysis on what's going on.
DR. BAHAR MEHMANI: And the result of the collaboration is now available on SSRN, and I can share the link to the preprint manuscript in the chat as well later on. Now what we did was to look into the submission metadata of 2,329 Elsevier journals across different disciplines in the month of February to May 2020, and the corresponding months in 2018 and 2019 for the baseline.
DR. BAHAR MEHMANI: And in order to find how women and men are performing we basically used dictionaries of names and gender APIs to assign gender to women and men, basically, names and email addresses that are collected in submission systems. And here I should emphasize that there is a limitation, these dictionaries of names and APIs are binary, so our study is blind to non-binary academics.
DR. BAHAR MEHMANI: Overall, what we see-- in terms of submissions, we see that it has grown quite, yeah, quite steep as you can see in comparison to 2018 and 2019, although there is a difference between the growth rate by men and women. And we looked also into renew acceptance rate of these journals, and you can see that in the graph on the right that it has increased slightly, but the difference between increase in the acceptance of the peer-review invitation rate for women and men is not that significant in comparison to submission.
DR. BAHAR MEHMANI: So we focused on the submission data. We thought that school closures and daycare closures potentially are putting a bigger burden on women academics. So in order to find out if that is correct-- no journal does collect any age information of people when they're submitting, so what we did was to look into the number of years between the latest submission of a person and their first Scopus publication data point.
DR. BAHAR MEHMANI: And I'm talking about more than four million individuals. So here you can see the cohorts that we have created. Based on previous studies, it seems that people start to publish their first paper when they are in their PhD or medical doctorate program. And the age group of those candidates, PhD candidates, MD candidates, are between 25 and 30. So based on the previous studies that I mentioned, we created these two cohorts, people who had in their first submission in 20 years in advance of their submission in 2020, and the ones who had more than 20 years of publication history.
DR. BAHAR MEHMANI: And you can see that, again, between men and women in the first cohort on the upper panel of the chart I'm showing to you, the difference between men and women is quite bigger than the other panel. So it basically shows or indicates that women in their 40s, early 50s are being hit harder, and their men can't afford potentially because they have families with small children.
DR. BAHAR MEHMANI: And, again, from previous studies, we know that household duties and care duties are not distributed equally between men and women, even in families where both of them are academics. Now, so the evidence is clear how statistically robust is this. So in order to check for the significance of these effects, we basically ran an estimate around a model, an estimated the mixed effect model using the age and gender, yeah, the age and the gender, and calculated in the model the submission difference in 2020 and the average of 2018 and 2019.
DR. BAHAR MEHMANI: And as you can see, again, we could clearly see a negative statistically significant negative effect for women in the first age cohort. Now to acknowledge that people are coming from different parts of the globe, and in different countries, the first pandemic lockdown has implemented in different time intervals.
DR. BAHAR MEHMANI: We also use a kind of controlled for this fact, using a random effect for the institution and country of the researchers. And, again, we could see a clear negative significant effect for women in all disciplines across the globe. We also use Google mobility data to see and check the results of the model. And, again, it was clear that this is the case. So what is going to stay with us?
DR. BAHAR MEHMANI: Well, the study that we started is going to continue, because we wanted to also, at the end of 2020, look back and look into the full year data. So this is an ongoing study. And hope this will hopefully will be disseminated later. We also wanted to check for the entire peer-review process. At the moment, for this study, we only looked at submission rate and the peer-review invitation accept rate.
DR. BAHAR MEHMANI: But once we have the full year data, we can also check for the entire submission to editorial decision men and women differences. But at least for the first wave of pandemic, what is clear is that the opportunity, a window of opportunity, got open for academics. And it seems that women didn't have enough chance to use that opportunity, and because eventually, every submission turns into publication, publications turning to citations.
DR. BAHAR MEHMANI: It is clear that that small window of opportunity that opened in the first few months of 2020 is going to widen the gap between women and men in all disciplines, as we could see from Elsevier journal data. So I think we all need to first acknowledge that, we all need to talk about it, we all need to be aware of it. And for journals, we need to stay on top of our journal data, look at it, check for it, and if it is something that we can disseminate further, we can talk about it, raise awareness, that's the least we could do.
DR. BAHAR MEHMANI: I think, also, universities institutions and funders have a role to play. Funders in hiring and promoting committees certainly, as both national and international level, can take this as an opportunity to widen the kind of metrics they use for all kinds of evaluations. This has been for quite some time a topic of discussion in the academic community. And I think this is the moment that they can take the chance and use all kind of other signals apart from publication and citation data.
DR. BAHAR MEHMANI: But they could also maybe ponder even more, and depending on the type of application, even disregard COVID-19 related publications and citations thereof, institutions and researchers themselves need to talk about the problems. We know, again, from previous studies that even in the academic community, the administrative, the teaching duties, they are not shared equally between men and women.
DR. BAHAR MEHMANI: And that is on top of all the other caring duties that women are facing, because this is also something that researchers need to talk about that. And I think with that, I'd like to stop my presentation. Thank you.
JASON POINTE: Great. Thank you, Bahar. Next up will be Shirley.
SHIRLEY DECKER-LUCKE: OK, great. Hopefully, you can see my screen.
JASON POINTE: We can.
SHIRLEY DECKER-LUCKE: And let just get it on. Great. Yeah, so I thought I talk to a little bit about Mythbusting Preprints in a Pandemic. I'm Shirley Decker-Lucke, I'm the Content Director of SSRN. We are a pretty vibrant preprint space, you can see from the numbers on the left that we have a lot of people who post and read papers on our site. We started out with a strong presence in social sciences and humanities.
SHIRLEY DECKER-LUCKE: And in recent years, we've been expanding out to offer our preprint services to multiple disciplines, including medicine. For just a little reminder, preprints are basically just early versions of research papers. So it's a paper that has not yet gone through the peer-review process. And, basically, advantages to researchers are, they get to share their content early, they get to get a DOI on that early on, they can get collaboration, feedback to refine, and improve their paper.
SHIRLEY DECKER-LUCKE: Sometimes, it's useful as a proof of productivity either to their university or to their funder. So mythbusters, I don't know if any of you have been to seen that show MythBusters, but I thought I'd kind of run through some myths that are out about preprints and COVID-19. So COVID-19 was cured because of preprints, well, busted. COVID-19 was not cured because of preprints. Preprints played a major role in helping share really cutting edge and important research, but, of course, had only a very small contribution to the global efforts of many people in lab coats around the world who have created the vaccine and are taking good steps to try to address the pandemic.
SHIRLEY DECKER-LUCKE: People died because of COVID-19 preprints. So this is also busted. There was a lot of fear that a bad preprint with bad science could lead to people gargling with Listerine or gargling with bleach and thinking that they would be cured, but not really. There have been very few problems with preprints that needed to be taken down. I'd say that it's very comparable to what happens with journal articles that are sometimes withdrawn, and the global community in those very few instances, where there was a paper that was posted that proved to not really be robust science, kind of brought it to everybody's attention and the paper got taken down promptly.
SHIRLEY DECKER-LUCKE: COVID-19 preprint are just like any other preprint, also busted, they're quite different. Part of that is because their medical preprints. And medical preprints need a much more robust revision, not revision, but screening and evaluation. And also, as has been said earlier with Miriam, COVID preprints had to really be prioritized, to be shared and screened as quickly as possible. You can believe everything you see in the media, also busted.
SHIRLEY DECKER-LUCKE: The media, I think, in the last year, and preprint servers and publishers have been in robust and much needed conversations about how do they contextualize and explain to their readers if they're quoting from something that's a preprint. And we've seen a variety, but, I think, also an improvement in the way that media now is much more likely to say, this citation is from a preprint that has not been peer-reviewed.
SHIRLEY DECKER-LUCKE: But in the beginning, that wasn't always made clear. So how have the pressures of the pandemic led to changes in how we do what we do? Well, it's been a real pressure cooker. The approaches that we've been doing since the '90s as far as screening and processing papers have pretty much held strong, and we're hopeful. In 2018, we worked with Preprint with The Lancet and started doing medical preprints.
SHIRLEY DECKER-LUCKE: And a lot of the evaluation and process and screening that we had in place then was really helpful when we got into the pandemic. But once we hit last year, we had huge operational challenges, where we had to be much faster and much more efficient, and prioritize COVID papers, and tracked them in a different way. We had a more extensive screening process and policies that we had to refine.
SHIRLEY DECKER-LUCKE: And we did a different approach to dissemination of the content by creating special hubs. And those approaches are ones that we will continue, and are also going to be useful, I think, as we look to non-medical areas that might have some ways that we could refine how we do those, based on what we've been doing with COVID. So the pandemic really put SSRN and all preprint servers in medical space in a pressure cooker, where you really see the extremes of the benefits and the extremes of the risks.
SHIRLEY DECKER-LUCKE: So of the benefit it's speed, because papers can just get posted there up, like two days later sometimes. And we were in a period of time, as you all know, back in February, nobody knew what was happening, nobody understood really what this pandemic was and what it was going to mean for us. And so the speed was such an essential element that preprint served. But the risks were also very extreme.
SHIRLEY DECKER-LUCKE: The potential for public harm, for public media and misunderstanding are much more significant with a medical COVID preprint than it is for a history preprint, for instance. This is a really nice research paper that looked at the way that preprints were a part of the general research productivity. And so one point you can just see here in May, for instance, of all of the research on COVID that was posted publicly.
SHIRLEY DECKER-LUCKE: A full 40% was in preprint. So researchers were really using these and finding them helpful way of getting their content out. And then on the other side, the downloads, were really high too. All of the preprint servers in the space saw a huge increase in the downloads per paper of COVID paper. So it was really embraced by the research community.
SHIRLEY DECKER-LUCKE: At SSRN, we created a hub. This was the first hub we did. So this was a new fun thing for us. And it was really just a place where we gathered and hosted and facilitated content around the coronavirus. We now have over 9,000 papers. We've seen 2.2 million downloads. And one of the things that we, because we're so interdisciplinary, do is that in addition to medical preprint on COVID, we also have a lot of other coronavirus related papers, so things around the economics of the pandemic, for instance.
SHIRLEY DECKER-LUCKE: And we saw a huge growth in those papers as well as the more scientific ones. We accelerated our processes, our basic processes to be able to handle the volume that were coming in. We saw across the board over 30% increase in our content submissions, which, of course, you can imagine put a lot of strain on our processing capacity. And you can see the screen to the right, the two boxes show our basic SSRN screening process, sort of a 2-step screening.
SHIRLEY DECKER-LUCKE: But for medical, we have these two additional steps that we established with Preprints for The Lancet, and we're able to use on our COVID. Like the journals, we prioritized COVID over other topics, which unfortunately meant some other papers had to sit and kind of wait their turn longer. And we are in the kind of fun situation, where we had groups like the WHO, the NIH, Europe PubMed Central reach out to us and ask us to help them when they were creating their own kind of libraries online collections of COVID research to contribute the relevant papers that we had, which was really fun, but it meant a whole new work process of identifying the papers, finding them, sending them, sending the update.
SHIRLEY DECKER-LUCKE: So a lot of new work for us. And then, of course, all of us are just dealing with the pandemic as individuals living with it, dealing with it, working from home, and all of that. This is a quick look at some of the screening we did. In the last year, we stepped up some of the screening. We added some cautionary language, and there's some examples of that. We actually hired a medical content editor.
SHIRLEY DECKER-LUCKE: We've been thinking about doing that for a while, but the pandemic kind of gave us the impetus to do it quickly. And we've been engaging in community dialogue around Preprints in the Public Eye, ASAPbio ran a great project, and just saying, how do we get publishers, preprint servers, the media to work together in a responsible way when reporting on science using preprints. So when we address the risks, I do feel like we found a fair balance.
SHIRLEY DECKER-LUCKE: That by doing these mitigation steps of screening, cautionary, language, withdrawals, looking for ways to communicate well, we did kind of balance out that risk in a way that I feel comfortable with, of course, always a little bit nerve racking. And also I think we did a nice-- this is an example of a way that the preprint can have some societal relevance. This is an example of a paper we did, it's a Preprint with The Lancet, and it had some really broad interest in media across the world.
SHIRLEY DECKER-LUCKE: You can see it got over almost 300,000 abstract views. And it just points to the way that preprints can play and have played a really helpful role in the broad understanding of the pandemic. So are these changes likely to continue post-pandemic? I would say, yes. I think basically the benefits far outweighed the risks. And we had SSRN, saw that there's really pushed us to innovate, to do what we've been doing, but do it better and faster, to do some things we've been thinking about doing that we'd wanted to do for a long time, and ultimately, I think made us a better preprint server overall.
SHIRLEY DECKER-LUCKE: So ending up with a couple more myths. Preprints played an important role in COVID-19 research. I would say, confirmed for sure. Medical and non-medical preprint saw accelerated growth in the past year. Absolutely confirmed. Events of the past year accelerated positive changes at SSRN for all of our preprints. Absolutely confirmed.
SHIRLEY DECKER-LUCKE: And question I get asked all the time, preprints are here to stay. I would say confirmed, but just for scientific integrity, to be actually super safe, I'll just say plausible. Thank you very much.
JASON POINTE: Thank you, Shirley. And now to our final panelist, Katy Funk from PubMed Central.
KATHRYN FUNK: All right, thanks Jason. I'm just bringing up my slides. Everything looks good?
SHIRLEY DECKER-LUCKE: Yup.
JASON POINTE: Yes--
KATHRYN FUNK: OK. Then I just get started. So I wanted to thank Shirley and Jason for the opportunity to be involved in this panel. I'm sort of representing the indexer perspective. And in this case that indexer is the US National Library of Medicine. I suspect each indexes has had its own way of working through the last year or so. And much of our response to NLM has really been about reframing how we view indexing the scholarly record.
KATHRYN FUNK: See it, not just as a version of record, but rather an emerging record of versions, which is not an idea I can take credit for, but it's one that has really resonated with me, particularly over the last 14 months. As you likely know, we operate two index archives sort of situations at the NLM, the PubMed, which is our citation and abstract database, and PMC or PubMed Central our full text archive. And today, I'm going to be focusing on two initiatives that we launched in response to COVID that directly impact what we index in PMC, and therefore what we include in PubMed as well.
KATHRYN FUNK: And these efforts, like those that Shirley just mentioned, were initially really geared at how can we speed discovery of this content during a public health emergency. Some things that have really emerged for me, and thinking about what we've been doing, are the value we place on transparency, and how that's sort of an ever evolving situation, as well as reuse.
KATHRYN FUNK: Open science is very important to NLM and that's been a priority for us. So the first of the efforts that we launched was the COVID-19 Initiative, and that was in March of last year. And that was really in response to a call that came from the National Science and Technology Advisors of many countries, to publishers, and scholarly societies to make their coronavirus. So we're talking SARS, MERS, past coronaviruses, as well as COVID-19 related research accessible and reusable through PMC and other open databases.
KATHRYN FUNK: And the degree to which the publishing community responded was incredible. And today, more than 50 publishers are collaborating with NLM. And the result has been 150,000 articles being made available to the public. And the interest in that collection just continues to grow. You'll see in April, we had one of our highest months to date of page views of these articles.
KATHRYN FUNK: So as the pandemic evolves and changes and becomes more of a situation, in new places, we're seeing continued interest in these paper. The second initiative we launched a few months later in June, and that's phase one of our preprint pilot. And this is sort of the opportunity that Shirley mentioned, where we had an opportunity to collaborate. In this first phase of the pilot, we committed to indexing all NIH preprint supporting COVID-19 related research to eligible preprint servers.
KATHRYN FUNK: Preprint have been an area of interest at NIH for several years now, I'd say five or six. We issued guidance in 2017 that encourage NIH investigators to post preprint, to report them on their progress reports for grants, to include them in their grant applications. And we saw this focus on not indexing all preprint on COVID, but instead looking at the NIH ones as being consistent with what we do with NIH public access, and the peer-reviewed manuscripts that we collect under the public access policy.
KATHRYN FUNK: So we've added more than 2,300 preprint records, as of this week. They've been viewed about 1.7 million times in PMC and about 1.5 billion in PubMed. Half of them have been posted to med archive, and that seems relatively consistent with how NIH researchers have been sort of approaching preprint policy throughout the pandemic. They also commonly post the bio archive we know.
KATHRYN FUNK: So with this initial focus on speed that I mentioned, there were two things that we did right away with these efforts. The first one was to start taking ahead of print content in PMC, which traditionally, as I mentioned, we've been very focused on that sort of classic version of record. And now we set aside concerns about duplicative processing and said as soon as you're willing to give it to us, we're willing to take it.
KATHRYN FUNK: As a full text archive, we also wanted to make sure that our preprint workflows didn't delay the discoverability of that content. So what we've done is set up a workflow that as soon as we curate preprint, they load to PMC. And a citation and abstract is made available there and in PubMed. , And at the same time we can currently have it tagged for the archival copy as well, which loads once it's available.
KATHRYN FUNK: In addition to speed, indexing preprint has really brought into focus for us the essential need for transparency kind of across the indexing process. We wanted to be as clear as possible in our databases about what sort of record is being viewed, what the current status of it is, and helping folks, not only when they were looking at a record itself, but when they're looking at search results to know exactly what they are seeing.
KATHRYN FUNK: And, I think, it's this version management that added additional challenges in transparency and made, particularly, the preprint pilot an interesting effort for us over the last year. We find ourselves frequently indexing a series of preprint versions, as well as a later published version. In all the versions of a preprint, we index under a single PMCID with the most current version of that preprint being discoverable in PubMed.
KATHRYN FUNK: And then if there is a published article linked to it later, that gets its own PMCID, its own PubMed ID, we just make sure that people can get someone and copy to the other. We also have automated searches for these preprint versions, and we are testing a really wide net for the peer-reviewed published article version. So we check preprint server API, process data, Europe PMC data, as well as our own internal tool.
KATHRYN FUNK: Additionally, as with all things COVID, we're doing regular checks the retraction of published articles and withdrawals of preprint. We've only had to deal with two preprint withdrawals so far, but it's something we're trying to actively stay on top of. To date, just over 40% of the prints that we've included in the pilot have been linked to journal articles. We find that on average, the preprint is available about 110 days prior to journal publication, but that's an incredibly wide spectrum of a few days to exactly a year that we've seen.
KATHRYN FUNK: So I think it will be interesting to see how that continues to evolve as well as what happens with that other 60%, whether those papers are really were intended for publication, or just the authors and researchers trying to get that data out as quickly as possible with no real intent to go further with it. Finally, the call from the OSPP and other science advisors, to publishers that led to the COVID initiative wasn't simply about indexing for discovery and access, but also for secondary analysis and reuse in standardized and machine readable formats.
KATHRYN FUNK: And that's a lot of what we do at PMC, and it's a huge priority for us. I would say it's been something we've had on our radar for several years, but COVID has really pushed it to the forefront. The COVID-19 initiative collection was actually sort of the initial foundation for the COVID-19 open research data set, or CORD-19, which was established by the Allen Institute for AI.
KATHRYN FUNK: As it currently stands, about 1/5 of the articles in that collection are available under a Creative Commons license. So we expect them to be available for this sort of reuse in perpetuity. The rest carry a license statement that allows for secondary reuse and analysis for the duration of the pandemic as determined by the WHO. This licensing bid is an area that NIH has identified as having the potential to maximize the impact of the research it supports.
KATHRYN FUNK: So as we've been indexing, we've been actively monitoring and ensuring that we're indexing the licenses of the preprint. We've been pleased actually to see that quarter by quarter throughout the pilot, there has been an increase in the use of NIH recommended CC-BY, or CC zero licenses, as well as the other Creative Commons licenses. So it's come a long way since the pilot launch, and we're curious to see how it continues to unfold.
KATHRYN FUNK: So looking ahead, we've been asked what we're going to continue, whether we think it's all good. And I would say right now, yes, although it's really hard to view the actions of the last year through a non-COVID lens. So we plan to continue the COVID-19 Initiative through the end of the pandemic. Should we ever have to deal with another pandemic? Knock on wood.
KATHRYN FUNK: I think we would consider how we could refine these engagements going forward, taking 100,000 articles at once, that's quite the thing. We're also looking to continue Phase 1 of the NIH Preprint Pilot, and considering a phase 2 that would stand the spectrum of NIH funded research to really get a sense for the use of preprint beyond COVID. We're going to continue to look for new ways to enable downstream reuse as far as licenses allow to support the AI and natural language processing communities.
KATHRYN FUNK: I think they're becoming more important all the time in dealing with the literature. And then we're also monitoring the emergence of new publication in particularly open peer review models, for preprint and journals, to increase the transparency of all this. So I'm going to leave it there. I did just want to take this moment to quickly thank all of the publishers who are here, who participated in the COVID-19 initiative.
KATHRYN FUNK: It was an absolute pleasure to be able to do something productive together, and probably what got us through and kept us sane for a while. So thank you.
JASON POINTE: Great. Thank you very much, Katy. All right. And Thank you to all of our speakers. We do have a couple of questions coming in now through the chat. But I want to start off with a question first, and I'd like to direct this first to Miriam. One of the things that none of us in scholarly publishing have direct control over, but which has been another effect of the pandemic, is the sort of misuse or misunderstanding, misinterpretation, deliberate or otherwise of some of the information that's been published in our journals.
JASON POINTE: And I'd just like to get the comments from all of the panelists, but, again, starting with you, Miriam, from Lancet on, do you think that we as a community in scholarly publishing have a responsibility to try and counter that, to try and affect that? And I know The Lancet has been the victim of this in some respects, where information that was published and then maybe retracted and still run with by some media outlets.
JASON POINTE: So it's another effect, another one of the changes, so I'd be interested in your thoughts on that.
DR. MIRIAM SABIN: Thank you very much for that. Yeah, I think we felt very strongly that we had a real responsibility in this as opposed to perhaps other years, and I'm not sure how particular this was to COVID. 2020 was also a very unusual year, because of the fact that in the United States, it was a presidential election going on.
DR. MIRIAM SABIN: And I think that the effect of that potentially cannot be underestimated when you've got all the media attention anyway focused on that, and everything going into it becomes so important, so vital to the debate. And so we realized very quickly that on the one hand, this could be really beneficial. For example, we had an editorial that I wrote, actually, looking at the CDC, and why the CDC was not as much in the public eye anymore, and calling for the CDC to be strengthened and so forth.
DR. MIRIAM SABIN: And that I think was very useful, was very helpful, it helped to push the debate. On the other hand, we had, as you're alluding to, a paper that we ended up having to express concern about, an expression of concern we put out, and then having to retract, because I think that, even though it was picked up for political reasons, of course, as was natural in these kinds of situations, it was an issue around databases.
DR. MIRIAM SABIN: And I mentioned in my presentation how we are addressing some of that. So it actually, I think, in the end, we felt we did a lot of discussion around it, at the Lancet internally. And we felt it was, in fact, very useful, because it helped us look at what are some of the vulnerabilities, in a general, for peer review research at any point in time.
DR. MIRIAM SABIN: And what can we do to kind of help improve that overall. And I think, I mean, the proof is in the pudding, I think we're trying to strengthen that. So overall, hopefully it's been positive.
JASON POINTE: And thank you. And without being specific about it, I do remember the particular paper that you're referring to, and having discussions just with laypeople, I mean, friends, neighbors in the community who were hearing about this on the news, and me trying to explain to them that, no, the paper wasn't retracted because of political or media pressure, that it was a question about the science and that this is the process. This is the way that things should work.
JASON POINTE: And then it's challenging sometimes, because we know how it works, explaining that to others is something, I think, we all need to participate in doing a better job in. Do any of the other panelists I want to touch on that subject? And if not, we do have some good questions coming in for you.
SHIRLEY DECKER-LUCKE: I was just going to throw in, I think that it just points to a real challenge. I mean, the media, everybody would like to have a clear answer. Do this, and you'll be safe. Don't do that, and it all will be good. I mean, of course, we want that. And science can't always give those answers. And I think it was a disconnect where science was scrambling to learn things, but science is iterative and experimental.
SHIRLEY DECKER-LUCKE: And it was being asked to give clear, do this, don't do this answers, and then people were interpreting that as, you can't trust science, which was a wrong interpretation. And I think we were all, I imagine, engaged in trying to help that in various ways.
JASON POINTE: Absolutely. Well, Shirley, the first question that we have from attendees is for you. And this is coming from Cammy Decker. She asks, are there any specific non-medical disciplines that could benefit from the additional processing steps that you've put in place for the medical COVID content on SSRN? Or do you plan to apply some of those new steps across all or many disciplines?
SHIRLEY DECKER-LUCKE: Yeah, we're looking at that. So like we have these four elements, things like conflict of interest. In our preprint server, we've always really tried to not be a publisher. We want to turn things around quickly, we don't want to do peer review, we really see that as an important difference between what we do. So we don't want to go down that slippery slope of trying to become a publisher.
SHIRLEY DECKER-LUCKE: But there are some things like perhaps a funding statement, which would be very helpful in multiple disciplines, a conflict of interest, and so forth. And I think right now, we still have our hands pretty full applying those to medical and COVID, but we definitely think there's some opportunity to start once we've really got it streamlined applying it to other disciplines where we see those as relevant. Great question.
SHIRLEY DECKER-LUCKE: Thanks Cammy.
JASON POINTE: Good. Question from Stephanie Pollack, and this is for Bahar, did the data set in your 2020 analysis include all co-authors or only the submitting, or corresponding offers? Stephanie says, she's curious how co-author, and/or first or last authorship rates may impact the disparities in research output during the pandemic.
DR. BAHAR MEHMANI: Yeah, that's a very good question. We looked into every individual to all authors of the manuscripts as individuals. And they can be the corresponding author of one manuscript, and then co-author of a different manuscript. So answering the first question, all authors. The second question is a great one. We didn't look at all the position in our study for the month of February to May.
DR. BAHAR MEHMANI: We are planning to do that when we are looking into the entire 2020 data and see. As we know, in different disciplines, all the positions have different meanings. So we are going to check for discipline women in prestigious authors positions, how that plays a role in manuscript submission and acceptance during the 2020 data.
SHIRLEY DECKER-LUCKE: Ooh, It looks like maybe Jason got frozen. He was warning us he might have that internet connection. All right. Do any of the panelist have a question for another panelist perhaps?
JASON POINTE: Sorry, I think I glitched.
SHIRLEY DECKER-LUCKE: Ooh, you back. You glitched, but you're back.
JASON POINTE: I was just asking, do you have questions for each other in the time we have remaining?
DR. BAHAR MEHMANI: I've got one for Shirley. So do you have any plans to run things like, commenting process or in community peer review process on SSRN and preprints?
SHIRLEY DECKER-LUCKE: We have commenting now in a number of places. And to be honest, we don't see a lot of use of it. We suspect that really a lot of the communication is happening offline, because we know people are looking for email addresses all the time on our site. So we think that people are finding the emails and having that happen offline. And we're not actively looking into adding a review element on SSRN.
SHIRLEY DECKER-LUCKE: A lot of other preprint servers are talking about it. And, I think, Katy, sort of alluded to some conversations about that or group review of preprints. I mean, I think we did really well during the coronavirus, but we're still catching our breath and regrouping a little bit probably before we take on a huge new something.
DR. BAHAR MEHMANI: Fair enough, thank you.
JASON POINTE: OK. Well, with that, I believe that we are at the conclusion of our time. I'd like to thank our panelists. I think, It's been a very good presentation of a lot of the diverse aspects of how our profession has been impacted by COVID. And I would just ask the attendees, we do have a three poll questions, they're the same questions that the presenters have been addressing during the presentation.
JASON POINTE: And we would invite you to please visit that and give us your answers. We'll be interested in seeing how you all have been affected, and how you think it's going to play out. Thank you very much for your time today.
DR. BAHAR MEHMANI: Thanks, everybody.
SHIRLEY DECKER-LUCKE: Thank you.
DR. MIRIAM SABIN: Thank you, bye.
KATHRYN FUNK: Thanks.
JASON POINTE: Bye.