Cadmore media player playing video Preprint review: addressing cultural barriers on the path for a more positive and inclusive review ecosystem
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
Preprint review: addressing cultural barriers on the path for a more positive and inclusive review ecosystem
Description:
Preprint review: addressing cultural barriers on the path for a more positive and inclusive review ecosystem
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/b5036aff-fcb5-439d-9728-8686608521e9/thumbnails/b5036aff-fcb5-439d-9728-8686608521e9.png?sv=2019-02-02&sr=c&sig=VnK%2Bw%2FlOvA2lDc51Orq5%2BQCp2KJ51hcr7xozdt9VvRg%3D&st=2024-11-21T15%3A25%3A02Z&se=2024-11-21T19%3A30%3A02Z&sp=r
Duration:
T00H46M06S
Embed URL:
https://stream.cadmore.media/player/b5036aff-fcb5-439d-9728-8686608521e9
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/b5036aff-fcb5-439d-9728-8686608521e9/Preprint review addressing cultural barriers on the path for.mp4?sv=2019-02-02&sr=c&sig=X6AczEtvX%2BZx0QToG3fIOhlc%2FCpC3hoNSgrLwfo1ye0%3D&st=2024-11-21T15%3A25%3A02Z&se=2024-11-21T17%3A30%3A02Z&sp=r
Upload Date:
2022-08-26T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
[MUSIC PLAYING]
IRATXE PUEBLA: Welcome, everyone. Thank you for participating in this session at NISO Plus 2022. We're here to discuss, prepare, and review more specifically a discussion around how we can potentially address cultural barriers towards a more positive and inclusive review ecosystem. I'm Iratxe Puebla, Associate Director at ASAPbio. So before we dive into the topic, I wanted to briefly introduce myself, as well as allow my colleague from PREreview to also introduce herself, because this is a session that we are organizing together.
IRATXE PUEBLA: So very briefly, ASAPbio is a nonprofit with a mission to make life sciences communication faster, more transparent. And as part of this, we support the productive use of preprints in the life sciences and transparency in peer review. So we also have an interest in all activities related to preprint review. We are, as I said, organizing this session with PREreview review.
IRATXE PUEBLA: So Daniela, would you like to say a few words about you and PREreview?
DANIELA SADERI: Sure. Thank you so much, Iratxe. And hello, everyone. My name is Daniela Saderi. And I'm the Co-Founder and Director of PREreview. And saying very briefly, at PREreview, we made it our mission to bring more equity and transparency to scholarly peer review. And specifically, with the goal of empowering and supporting researchers that have been traditionally marginalized in the peer review process.
DANIELA SADERI: And we do that by-- next slide, please-- by hosting an online platform that is an open source, in which any reviewers and researchers can post and contribute to the peer review of preprints. We organize training sessions, and collaborative discussions of emerging and new research published as preprints. So it is very much a pleasure for me to be here.
DANIELA SADERI: And thank you for inviting me. I'll pass it over back to Iratxe to introduce the speakers.
IRATXE PUEBLA: Thank you so much, Daniela. So the way we are going to run the session is, that we want this to be mostly a conversation, so that we can explore different perspectives. We are going to be hearing from three fantastic speakers, who bring the perspective of researchers, of organizations who are involved in review and preprint review, as well as the perspective of the journal in this context. We are going to be hearing more and introducing the speakers in a second.
IRATXE PUEBLA: But before that, I just wanted to give a little bit of an overview, and set the scene as to why are we having a conversation of preprint review. What we have seen in recent years-- and I'm sure many of you are familiar-- we've seen a strong adoption of preprints, and growth in the use of preprints in a number of disciplines, particularly the life sciences. And in parallel to this, we have also seen many new activities, projects, and communities appear that are facilitating, or even coordinating, the review of those preprints.
IRATXE PUEBLA: As you know, preprints and peer-- we felt any peer review on the part of the preprint server, but because they are publicly available, this means that we can organize, and contribute, react to those research findings that are being sought in the form of preprints. This also means that we have platforms that allow somewhat informal and unstructured commentary to be added, but also services that actually coordinate the review of preprints in a similar way to what we expect in the traditional journal review process.
IRATXE PUEBLA: In this context, we have also heard about the potential benefits of having those preprint reviews publicly available. And I wanted to mention a few of the benefits for the different members of the science communication community for all sorts of causes and opportunity of getting feedback, to help them improve their research, and also, because the interaction happens in the open.
IRATXE PUEBLA: It's also an opportunity to provide clarification, or response to any questions that may arise. From the perspective of the reviewers, the preprint review activity allows more researchers to participate, compared to the journal peer review, because as you know, in the journal system, we rely on specific invitations on the part of editors. This means that participating in preprint review is also a training opportunity for those who perhaps cannot take part in journal review, but are seeking to develop their reviewing skills.
IRATXE PUEBLA: Another potential advantage for reviewers is that by posting these comments on preprints, they open up the opportunity for those reviews to be reused. If, for example, that paper were to be submitted to a journal, then reacted, and there were to be further requests from the same reviewer to review the same paper, well, that means that review is already in the open. And it also means, again, from the journal perspective, that the preprint reviews allows them to complement their own peer review process, but also to have access to a new pool of potential experts whom they can consider as additions to the reviewer pool, or their editorial boards.
IRATXE PUEBLA: And then of course, importantly, given that preprints are posted without preprint review, having that feedback, that commentary provides additional context and expert evaluation on the preprints that is available to readers, not only to other scientists, but also, members of the public who can also get a glimpse then on how these conversations between scientists work. And the fact that science is an iterative process, and self-correcting process before we can establish some research findings as established evidence.
IRATXE PUEBLA: So there are a number of benefits to this, but what we are seeing is that the participation in public preprint review, at least so far, has been relatively low. And I'm sure there are a number of factors that fit into this. But I'm convinced that many of these are cultural issues. Elements related to perhaps the legacy of how we interpret peer reviews, and so we are very familiar with the review process of journals.
IRATXE PUEBLA: We are comfortable with that. How can we start trying to do it in a different space, in a different way. And also, potentially, the fact that there is a diversity, which is obviously positive, but this diversity of platforms and models around preprint review mean that it can be a little bit of a perhaps, confusing and fragmented ecosystem for those who come into this space.
IRATXE PUEBLA: We are going to be discussing many of these elements, and so with our speakers. But before I hand it over to them, I wanted to mention very briefly a recent initiative in this space, which is the first principles for preprint feedback that have been developed by ASAPbio with input from a working group that had representatives of the community in science communication.
IRATXE PUEBLA: They are a set of 14 principles that are encompassing the behaviors that we are expecting to see a positive and constructive preprint feedback being in the open. One of the things I want to mention is that we see the principles applying to anybody who participates in preprints, reviewers, of course. But also, the authors of preprints who may respond to any comments on their work, as well as members of the community who can play an important role in reinforcing positive behaviors, but also calling out those behaviors that are less positive.
IRATXE PUEBLA: So without further ado, I'm now going to be asking each of the three speakers to each introduce themselves, and also, so that they tell us a bit about the perspective that they bring to this discussion. So we are going to be starting with Gautum Dey, who is group leader at the European Molecular Biology Laboratory. So Gautum, do you want to introduce yourself, and share a bit of your perspective around preprint review?
GAUTUM DEY: Absolutely. Thanks so much, Iratxe. And thanks, Iratxe and Daniela for the invitation to be part of this discussion. Overall, the community attending this conference is not the community that I engage with on a daily basis, so I am also excited to hear about this perspective. So yeah, my day job is as the leader of a young research group at EMBL. We're cell biologists, and we study the organization of the nucleus.
GAUTUM DEY: And this is what occupies most of my time. But I also have some other hats related to my interest in publishing reform, advocating for preprints, and also, the publishing process in general. Because I believe that scientists should really participate in each of these steps, actively practicing scientists, contribute, give back to the community, but also find ways for the system to evolve in a changing world.
GAUTUM DEY: And so I sort of just put up some of these organizations here, organizations that I'm involved with, with bioRxiv, one of the big preprint platforms for the life sciences. I help screen preprints as part of a large group of volunteers. I also volunteer for ASAPbio. For example, I was part of this working group that Iratxe mentioned putting together fast principles for preprint review.
GAUTUM DEY: And I've been writing for some time for a preprint sort of highlight and discussion forum called preLights, that is one of the ways that the community is trying to use to encourage discussion around preprints. I also serve on some early career editorial boards, and editorial boards for a few community run journals, and community run initiatives, like review comments that incorporate preprint review into a sort of formalized format that's kind of like a bridge between, let's say, a traditional journal and a pure overlay.
GAUTUM DEY: And so in these roles, I've become conscious of the fact, just to echo something Iratxe said, that the infrastructure, and the broader kind of preprint, preprint and community, and preprint ecosystem is evolving rather rapidly, and is ready for this next phase of incorporating public review. But the scientists that are producing the research are always a little bit slow to follow, partly, because cultural change is never sudden, or at least has is never sudden until it reaches a certain critical mass.
GAUTUM DEY: And so for example, discussions about commenting on preprints in a formalized fashion in accordance with generalized principles, such as the ones Iratxe just mentioned is not a mainstream conversation amongst academics. And so my main interest here is to brainstorm with you ways for us to make that conversation more mainstream, and to bring more people into the discussion, and also, to simply educate or inform, and to let people know that these frameworks exist.
GAUTUM DEY: And instead of posting comments on Twitter, or writing emails to their colleagues when they see a preprint posted, which are forms of common practice at the moment in the life sciences, perhaps, we could move towards an ecosystem that is more streamlined, and allows faster, more efficient, and more transparent feedback on research. So yeah, so that's why I'm here today.
GAUTUM DEY: And I'm looking forward to the rest of the discussion. And Iratxe, please do move on to the next speaker.
IRATXE PUEBLA: Thank you so much, Gautum. Right, so we are now going to be hearing from Joy Owango. Joy is the Founding Director of the Training Center in Communication Africa, and is also a Board Member of the preprint server, Africa Archive. So Joy, would you like to say a few words about yourself, and again, your activities in preprint review?
JOY OWANGO: Thank you. Thank you so much, Iratxe. As you have mentioned, I'm the Executive Director of the Training Center and Communication. And we are based at the University of Nairobi. We are a research capacity trust that supports researchers, research institutes, and government on how they can improve their research output and increase their visibility. Over the last 15 years, we've supported over 10,000 researchers, and we've won a couple of awards.
JOY OWANGO: And what we've noted is that, the best way to support the researchers is also through mentorship, which is expected, but the mentorship is not as active with research who are self-funded. So we set up a mentorship group with over 900 researchers from Africa. And it's a hand-holding process in their research lifecycle from research idea to publish it. The next slide, please.
JOY OWANGO: So our premise is to train, support, and empower researchers. And also, when it comes to now the activities involved in supporting institutes, especially in the institutional capacity strengthening, our premise is on the research life cycle of institute, of organizations in higher education. So you're looking at granting councils, research councils, university commissions, and libraries.
JOY OWANGO: We need to build their capacities through education, facilitation, and partnership, so that they can effectively support early career researchers. And it is through such initiatives that we partnered with IDA Africa, PREreview, and eLife, and Africa Archive in coming up with a peer review program aimed at supporting early career African researchers, and with the objective of creating best practices, and innovative approaches in peer review.
JOY OWANGO: Now, the reason why we did this is because we needed to raise awareness around preprints, and foster the participation of African researchers in peer review, especially in open peer review of preprints. So the reality is, African researchers are quickly adopting preprints as a way as part of the academic publishing process, and as a way of increasing their research visibility. But we want to also involve them in the peer review process.
JOY OWANGO: There have been quite a bit of inequities when it comes to peer review from being done on researchers from Africa, due to existing biases. And through this training, we'll be building the capacity of early career researchers, or even mid-level researchers in understanding on what is expected in peer review, the best practice in peer review, and how they can also support future early career researchers in the process.
JOY OWANGO: So this is an ongoing process, and it also includes reviewing of preprints. So it's quite an exciting time for all of us and our partners in highlighting how we can support African researchers in adopting this process that they've already started getting involved in in preprints, and also the peer review of preprints. Thank you so much.
JOY OWANGO: Iratxe, you're on mute.
IRATXE PUEBLA: We would have expected by now I would be better at this with the microphones. Thank you so much, Joy, for sharing that perspective. Let me just pull up the slides again for our third speaker, who is Laurie Goodman. Laurie is the Policy Director for GigaScience Press, and is bringing more of a journal perspective to the conversation. So Laurie, would you like to say a few words about yourself, and again, what is GigaScience doing in this space?
LAURIE GOODMAN: Yeah, hi. And thank you very much for inviting me. I have been in publishing for now over 25 years, so I have perspective from journals trying to hide everything they do to journals now moving into first open access, and now, moving into trying to push everything open. And I think this is an essential part of publishing, because I feel like a lot of the hiding of information really held back researchers in understanding the whole process.
LAURIE GOODMAN: I don't want to say they were doing this for evil purposes. There was an idea that it was important to hide information as a means of protecting people, and also protecting the public for misinformation getting out there. Which I have a serious feeling that instead of putting out that scarves can solve arthritis, that we put out scientific information that the public can read. So anyway, we published two journals.
LAURIE GOODMAN: Both of them are something we call open science, because everything we do is completely open and transparent. The two journals we publish are GigaScience and Gigabyte. For GigaScience, we encourage the use of preprint servers, and that's sort of historical, and I know we're going to move toward you have to put it in a preprint server. I should mention that GigaScience, even though we don't mandate it, about 35% of the papers that people submit are already in preprint servers.
LAURIE GOODMAN: And then Gigabyte, 100% are in preprint servers, and it spans across all of the preprint servers, although, most of them are in bioRxiv, we have two that came from Africa Archive. And it's really exciting, because unfortunately, I knew it-- but until Joy was here, I realized, boy, we interact with the other preprint servers. And so I'm hoping to have conversations with them about what journals can do.
LAURIE GOODMAN: Because journals can bring a lot to the preprint area, especially if we are embracing that. And more and more journals are embracing that, and saying it's fine if you put it in a preprint server, which before, they were saying, don't do that. So I want to point out first of all, why you should use a preprint server. And a lot of it is because we use the preprint servers.
LAURIE GOODMAN: Right now, a lot of journals use preprint servers to find papers. Someone tweeted a couple of years ago, wow, I used to have to email all these editors to see if my paper was suitable. And now, editors are emailing me. So editors actually do use these to find papers that are suitable for their journal. When we launched 10 years ago-- it's our 10th anniversary-- the first thing we started doing was going through-- at that time, it was just archive for papers that we knew.
LAURIE GOODMAN: And we have done that consistently. The other thing is, we use preprint server is not only to look at authors of similar type papers, but also, if there are reviews there, to find suitable reviewers. If the preprint has a review on it, the authors really get us much faster review process, which everyone wants. Because we will use the reviews on a preprint server if we know the name, because we mandate that our peer reviews are named, so the authors know who they are.
LAURIE GOODMAN: And then we publish the peer reviews, so that the community knows who they are. Because we think this is an essential part of the scientific process. So if there's a review, which is what we're trying to encourage, you will speed authors. So you want to do reviews, and you want people to do reviews. The other thing is, is having reviews and a preprint server can provide training tools for teaching young researchers how to review.
LAURIE GOODMAN: They can read through it. They can try it. They can practice it. They can do it with other people. So if you're afraid to put a review on, have another person, your advisor, sign off on it, and put their name there, so that you feel supported. And you are supported. But mostly, it's to feel comfortable.
LAURIE GOODMAN: The other thing is that we find is that, having reviews in a preprint server, again, can aid authors if they're writing a similar type of paper. We find as editors, that specific papers that are in a similar genre almost always, the reviewers say, they need to do this special validation that for some reason nobody does. If you look in a preprint server, and there is a review, you can look at that and say, oh, every single reviewer asked for this.
LAURIE GOODMAN: I'm going to make sure that's in my paper. Having things ready for the reviewers already reduces the comments. So it's really a good way for avoiding the most common reviewer concerns. So again, that's why you should put reviews in, because you're helping other people. But it can also be done to help yourself. And then finally, when young people post reviews in a preprint server, it provides a record of expertise for job applications.
LAURIE GOODMAN: So I know people are anxious about doing it. But we found that the fear of reprisal is actually more of a boogeyman. There is no evidence that there is a huge reprisal of people who put their names in. And in fact, if there is a reprisal, and it's public, which it usually is, everyone can see that and comment on whether that's true or not.
LAURIE GOODMAN: And also say, hey, why are you inhibiting young people from doing it? Anyhow, we are heavily involved in trying to promote every area of preprint servers. And right now, we've been involved with ASAPbio in promoting reviews. We also, if we have a peer review, post it back onto preprint servers. Since our reviewers are named, they already know that everybody is going to see it.
LAURIE GOODMAN: And it's really fun in real time every time we post a review back on bioRxivs, they tweet it. There's this automatic robot tweet. So when we have one of our interns doing the posting of back posting of reviews and you're on Twitter, you can see just real time our interns working, so poor people. And then there's a new thing that's coming out called SCIETY, S- C- I- E- T- Y, which is a organization that's trying to funnel all of the peer reviews and all of the information from preprints into one area to help people navigate the big peer review process throughout to reduce the fragmentation, but also, make it much easier to find the reviews, find specific preprints.
LAURIE GOODMAN: So we're really excited to have started that. And as I mentioned, any time we see something new, sometimes, we're a little pushy. We want to be involved. We want to be involved, we think it's essential that journals participate, and become partners in growing this process. So thank you.
IRATXE PUEBLA: Yeah, thank you so much to all of you for sharing that perspective. I guess to get the conversation going, I wanted to come and touch back on a couple of things that you mentioned. I'm going to begin on Laurie's comment out the fear of reprisal, because I think that's something that tends to come up in conversation about posting comments publicly, and making ourselves vulnerable to exposing our potential critiques.
IRATXE PUEBLA: And I wanted to see what your thoughts were about-- essentially, do you think that this is a main driver for potentially, again, having researchers participate in this type of review activity? And if that is the case, whether you think there are any differences across career states, and we often hear or there is the power structure in place. It may be trickier for early career researchers to participate.
IRATXE PUEBLA: But there may be a lot of disparities, again, also depending on where are we based. We may have different review cultures. I think I want to start by asking Gautum for some thoughts on this from your perspective as a researcher. Is this something that, again, you may have encountered, or do you have some suggestions about?
GAUTUM DEY: Yeah, I mean, yeah, I think the debate about signing reviews in a way is just going to carry over. Because it's a long standing discussion within the context of traditional peer review. And I don't see those problems going away. I think here, are the perceptions are almost as important as the reality. So even if it turns out there are no documented reprisals, I think young, or early career researchers may still hesitate to lend their name to critique, even if it's constructive of a paper.
GAUTUM DEY: However, I think it's linked to this idea of separating, or of making peer review, or preprint review more objective. So if there are objective criteria that are being discussed, if it is constrained to constructive criticism of the research, plans, ideas, conclusions within this report or document, and it has no touch of either personal opinion, or value judgment, or any of the things that can be associated with peer review reports, then the potential threat to anybody of reprisal, or even be subject to accusations of inappropriate behavior does fall away.
GAUTUM DEY: So I think hopefully, these two will go hand in hand, that as it becomes normalized to review papers in this way, such that the review is a genuine piece of added scientific value, and not a social commentary on the authors of their work, then it will also become normalized for people to be able to post this at all stages of their careers, as long as they have the relevant expertise to comment on the subject matter.
GAUTUM DEY: So I guess, it's a hopeful thought. But I certainly agree that it is an issue that must be dealt with.
IRATXE PUEBLA: Joy, I wonder if you have any perspective on this. And again, you have been doing mentoring for researchers. I wonder, again, do you see these patterns around? Is this more of an issue for early career researchers? And I would also be interested in getting your perspective from working with African results. Again, are there any differences there in terms of culture that we may need to factor in as we try to normalize this type of activity, as Gautum was saying?
JOY OWANGO: OK, first of all, academic publishing is evolving. And the fact that African researchers are adopting these new technologies, like the use of preprints, of course, it comes with some of the issues that come with it, you know, if it means open peer review, and being part of that process. The reality is, we've had early career researchers involved in the peer review process, but then it has been closed.
JOY OWANGO: You know, because it's behind, you end up finding out much later. So what's the difference now? Because now, you can see the person, and now, it's in the open? What I'm trying to say, the fact, the mere fact that they are really accommodating these new technologies in academic publishing, and they've already been allowing early career researchers to be involved in the peer review process, albeit, closed, and closed in the closed peer review, I don't see any particular reason why they should be castigated to be involved in the open peer review system, particularly those who have already started getting the experience of peer review, whether it's in the in the closed peer review process.
JOY OWANGO: So that's my opinion. Then number two, the cultural aspect, yes. The average person with a PhD and above in Africa is in their 40s. It's getting lower, which is a good thing. And that means the early career researchers are in their 30s, or in their late 20s. The cultural dynamics in a continent that is still embedded in respecting your elders, regardless of the career path you're taking, and you're taking, means that, the poor early career researcher does not have a voice per se, unless the senior researcher is the one guiding them, and telling them it's OK to go through this process, or it's OK to present your view and say, this is wrong.
JOY OWANGO: All right. So the cultural dynamics need to be factored. And also, in regards to the cultural dynamics being factored, it means the capacity building, this is why we're doing this kind of training on understanding peer review systems, and how to peer review, how to train peer reviewers. Mainly, because at the end of the day, this early career researcher will end up working with a mature researcher.
JOY OWANGO: So it's also trying to break those barriers such that they are all on the same space in supporting the research life cycle, or in supporting their research life cycle. So yes, it is a problem. But we cannot ignore the fact that these early career researchers have already been involved in close peer review. What is the difference now? Because you can put a face to it, there's an open sign off to it?
JOY OWANGO: So for those who have been complaining, I always wonder where the issue is. Because you had already been allowing early career researchers to be involved in the close peer review process, yes.
IRATXE PUEBLA: Thank you so much for sharing that important perspective. I'm going to kind of move on to something slightly related to this potential reprisal element, but thinking a bit more about the structure. Something else that has come up in other conversations about this is that some may think all of this public commentary is all well and good if we are all angels who behave correctly. But the reality of things is that in the general process, we have a dedicated person who is tasked with policing this whole thing.
IRATXE PUEBLA: We have the editor. There is a framework, and kind of protected if something comes up. While there is not necessarily-- I mean, the moderation processes, I don't think are necessarily set across the different preprint review modalities. And I just wonder what your perspective is on this, whether essentially, they think this can operate without some form of basic standards of moderation?
IRATXE PUEBLA: Or do we start having a spectrum? I'm going to maybe perhaps ask Laurie to comment on this first from the journal perspective, and then Joy. Laurie?
LAURIE GOODMAN: Yeah, I just need to touch a little bit-- I won't go into it-- but on the previous thing. In terms of reprisal, if the peer review is hidden, you can still be reprised on it. I don't know how many people guessed who the reviewer was. And 95% of the time they were wrong. So the reprisal still happens, but unfairly. So if we're going to be fair, put your name out there. And they're always much more positive than the hidden review.
LAURIE GOODMAN: So I forgot your question, because I was still talking.
IRATXE PUEBLA: The potential moderation requirements for the new ecosystem, where we have a spectrum of you can comment and review.
LAURIE GOODMAN: I mean, with the editors making that process, each journal will have a set of biases. And a lot of these aren't well known, not to them even. I mean, there's always an unconscious bias. And I think that it's really important from the standpoint of both reviewers and editors to have some sort of guidance. A lot of institutions are putting in training programs to avoid bias.
LAURIE GOODMAN: And I feel like this is something that publishing needs, that editors should be taught and trained on how they might be in unknowing ways, biased, and that reviewers need to be trained. And that can start with young reviewers, especially. But if editors are aware of it, they have a better way of assessing whether they think that's happening. But I think that is probably one of the best ways of trying to remove bias, I think a training program.
LAURIE GOODMAN: It's a weird thing to say about editors. But it's a big institution. So instead of just being within a university, it's an institution. It should be worldwide. What are biases going in? It's something that maybe preprint servers could be involved in, because you're getting in the earliest reviews.
LAURIE GOODMAN: And so I'm not quite sure how that would be done, and you preprint server people should figure it out.
IRATXE PUEBLA: Joy, would you like to add to this?
JOY OWANGO: Yeah. And there is a need to have some basic standards and regulations when you're peer reviewing, whether it's open, so that you can make it more inclusive. The basis of peer review has always been really harsh. And that is why there's been also movement on peer review on being kind. Because you as much as you're critiquing someone's work, you do not know the time and effort they took in putting that project together, or that paper together.
JOY OWANGO: And the fact that today, we are much more aware of various social constructs, you know, like mental health. And we do not know what our insensitive point might do to this person. So we are much more aware. So what does that mean? And the fact that we are in an online environment, be kind. So we need to put in standards, basic standards that would also guide any open reviewer that any open reviewer will use when we work with when they're reviewing any preprints, or doing any open peer review.
JOY OWANGO: But then we also need to give publishers some credit. I never saw this five years ago. And this year alone, I have actually between late December and this month, we've been approached, TCC Africa has been approached by a number of publishers, because they want to train their audience on peer review. The editors are making an effort. Some of them have even set up programs just purposely focusing on peer review, what it entails, what their specific publishing houses expect.
JOY OWANGO: And also, which I find also quite fascinating is, they're not teaching their audience on peer review, but they're also teaching their audience to be peer reviewers. You see, they're becoming a bit proactive. And this has purely happened because everything is happening online. Now, the process is open. And people are getting more aware. And also, in terms of the bias, especially in Africa, now, we are getting a voice in raising concerns on the bias when it comes to peer review in the continent.
JOY OWANGO: And that is why training programs are being created. And just like Laurie said, yes, capacity building is what will help in mitigating this challenge. And all hope is not lost. What I'm saying is that with the rise of preprints, and preprint servers, and with us going online, up has created a snowball effect as a face in academic publishing, which was otherwise hidden. Now, it is open.
JOY OWANGO: And now, people are seeing the bias in it, and some of the challenges that were involved in it. And the fact that editors are coming to organizations to ask them, how can we help our audience in understanding peer review, or how can we train more peer reviewers? That's a step in the right direction. So I think it's, yes, it's a positive trend when it comes to peer review. It's been demystified.
JOY OWANGO: People are becoming more kind, because it's about critiquing. But people are becoming more kind, and becoming more conscious, especially when it comes to sharing their view or opinion on the work that they're reviewing.
IRATXE PUEBLA: Thank you so much. An important reminder that at the end of the day, it's the same researchers who will comment on review preprints and journal articles. Essentially, there is a continuum here. And I wanted to ask one more question. And here, I'm going to ask all of you to contribute, which is, as I mentioned at the beginning, we have a number of platforms and projects available to comment on preprints, which I think is great to see the diversity.
IRATXE PUEBLA: You know, it's a new experimentation in different ways. But it means that if I put a researcher hat on, and I want to contribute, I may be slightly daunted by the diversity of options, and I may say, where do I go? What should I post my comments? What's going to happen? How is it going to be used? So I wanted to ask each of you to briefly share your thoughts as to what do you recommend in terms of steps to make this perhaps a bit more clear for researchers, to perhaps add some clarity, so that it is also an easier path to participation for researchers who want to contribute preprint research?
IRATXE PUEBLA: Gautum, starting from you.
GAUTUM DEY: Yeah, sure. I mean, yeah, I mean, it's tricky in the sense that it's one of those things where maybe it's also important to see what sticks. So it may be that some approaches become more intuitive to people to use than others, and that's not obvious from the get go. I mean, having said that, I found that one of the formats that seemed intuitively most accessible to people, because it's already very close to something that people do regularly is, some form of being able to transfer journal club for discussion format to the page, right?
GAUTUM DEY: So this is in a way a plug for a peer review type approach, where we're doing this all the time anyway in our research groups. We read preprints. We discuss them. And what will often happen, especially with maybe let's say, slightly more traditional research groups, as they might collate the feedback, and email the authors, right?
GAUTUM DEY: So I mean, this is just a very, very small step from there to posting it formally. And so for me, my intuition would be that that's an easy entry point. And maybe that's one of the things that can be-- I mean, it is already being promoted, but it's maybe one of the things that can be a part of a kind of global effort to spearhead more open journal clubs, basically.
GAUTUM DEY:
IRATXE PUEBLA: Great. So yes, I'm a great fan of journal clubs. Thank you for that, Gautum. Laurie, what would be your suggestion if we wanted to, again, try to help researchers navigate this space, anything we can add to add clarity?
LAURIE GOODMAN: Yeah, I mean, there are a lot of preprint servers. One of the ways to do it is to actually do a Google search, or a Google Scholar search, because preprints are in the search. I guess, I would do a Google Scholar search, because preprints are recorded in that. So you can do that and find papers across the board of those different preprint servers. It's a great way.
LAURIE GOODMAN: And I do need to comment, because I always comment on everything. I love the idea of having journal club comment and be part of the review process. And now, I'm trying to think about how a journal could engage that, of course. I mean, I remember when we did our journal clubs when I was a graduate student. We talked really in detail through all the papers, what we liked, what we were concerned about.
LAURIE GOODMAN: And it was sort of a brainstorming section. So now, I'm going to be thinking about that. But for finding preprints, I suggest Google Scholar. And hopefully, SCIETY will be one of the hubs to bring it together. I like the fact that there are preprint servers across the world. It takes care of biases. They know specifically what the problems are.
LAURIE GOODMAN: So I like the fact. I wouldn't say we all want one preprint server, especially since there are ways to do it. I think it's essential that preprint servers are all around the world.
IRATXE PUEBLA: Joy, any further suggestions from you?
JOY OWANGO: Let's accommodate what-- to go back to what I was saying, let's give credit where it's due. Because with publishers today, they are more conscious about their audience, because the audience has a voice. OK. So what does that mean? We can see that there is that effort being-- so for potential partners who are involved in research capacity building, especially in the research lifecycle, either early career researchers, or mid-level researchers, don't hold back to reach out to publishers, especially when it comes to supporting researchers s in understanding peer review, in understanding how preprint servers work.
JOY OWANGO: Because they are now reaching out. They are now aggressively reaching out to organizations to see how they can support their audience. So gone are the days when they would say, OK, yes, we are going to train you on academic writing. They are going through the entire research lifecycle. So yes, you can get the preprints, preprints going through any sites. Today, we have more bibliographic and citation databases than we had five years ago.
JOY OWANGO: And one of the filters that you can get publication types on is preprints. So it's slowly becoming the norm, and that is exciting. So we shouldn't be afraid of it. But then for organizations, I'll just say, reach out as well. For research capacity organizations, reach out to publishers, because publishers are looking for capacity building partners who they can work with in supporting their audience in understanding best practice in peer review, understanding in mentorship, and peer review, so that things are changing.
JOY OWANGO: Basically, things are changing for the best.
IRATXE PUEBLA: Thank you so much, Joy, and all of you for sharing your perspective and ideas. And we are going to now close this part of the discussion. Obviously, many other things to discuss about preprint review. But I'm hoping that there will be plenty of opportunities through the questions with the audience. So thank you so much, and I look forward to the discussion with the audience as well. [MUSIC PLAYING]
Segment:0 .
[MUSIC PLAYING]
IRATXE PUEBLA: Welcome, everyone. Thank you for participating in this session at NISO Plus 2022. We're here to discuss, prepare, and review more specifically a discussion around how we can potentially address cultural barriers towards a more positive and inclusive review ecosystem. I'm Iratxe Puebla, Associate Director at ASAPbio. So before we dive into the topic, I wanted to briefly introduce myself, as well as allow my colleague from PREreview to also introduce herself, because this is a session that we are organizing together.
IRATXE PUEBLA: So very briefly, ASAPbio is a nonprofit with a mission to make life sciences communication faster, more transparent. And as part of this, we support the productive use of preprints in the life sciences and transparency in peer review. So we also have an interest in all activities related to preprint review. We are, as I said, organizing this session with PREreview review.
IRATXE PUEBLA: So Daniela, would you like to say a few words about you and PREreview?
DANIELA SADERI: Sure. Thank you so much, Iratxe. And hello, everyone. My name is Daniela Saderi. And I'm the Co-Founder and Director of PREreview. And saying very briefly, at PREreview, we made it our mission to bring more equity and transparency to scholarly peer review. And specifically, with the goal of empowering and supporting researchers that have been traditionally marginalized in the peer review process.
DANIELA SADERI: And we do that by-- next slide, please-- by hosting an online platform that is an open source, in which any reviewers and researchers can post and contribute to the peer review of preprints. We organize training sessions, and collaborative discussions of emerging and new research published as preprints. So it is very much a pleasure for me to be here.
DANIELA SADERI: And thank you for inviting me. I'll pass it over back to Iratxe to introduce the speakers.
IRATXE PUEBLA: Thank you so much, Daniela. So the way we are going to run the session is, that we want this to be mostly a conversation, so that we can explore different perspectives. We are going to be hearing from three fantastic speakers, who bring the perspective of researchers, of organizations who are involved in review and preprint review, as well as the perspective of the journal in this context. We are going to be hearing more and introducing the speakers in a second.
IRATXE PUEBLA: But before that, I just wanted to give a little bit of an overview, and set the scene as to why are we having a conversation of preprint review. What we have seen in recent years-- and I'm sure many of you are familiar-- we've seen a strong adoption of preprints, and growth in the use of preprints in a number of disciplines, particularly the life sciences. And in parallel to this, we have also seen many new activities, projects, and communities appear that are facilitating, or even coordinating, the review of those preprints.
IRATXE PUEBLA: As you know, preprints and peer-- we felt any peer review on the part of the preprint server, but because they are publicly available, this means that we can organize, and contribute, react to those research findings that are being sought in the form of preprints. This also means that we have platforms that allow somewhat informal and unstructured commentary to be added, but also services that actually coordinate the review of preprints in a similar way to what we expect in the traditional journal review process.
IRATXE PUEBLA: In this context, we have also heard about the potential benefits of having those preprint reviews publicly available. And I wanted to mention a few of the benefits for the different members of the science communication community for all sorts of causes and opportunity of getting feedback, to help them improve their research, and also, because the interaction happens in the open.
IRATXE PUEBLA: It's also an opportunity to provide clarification, or response to any questions that may arise. From the perspective of the reviewers, the preprint review activity allows more researchers to participate, compared to the journal peer review, because as you know, in the journal system, we rely on specific invitations on the part of editors. This means that participating in preprint review is also a training opportunity for those who perhaps cannot take part in journal review, but are seeking to develop their reviewing skills.
IRATXE PUEBLA: Another potential advantage for reviewers is that by posting these comments on preprints, they open up the opportunity for those reviews to be reused. If, for example, that paper were to be submitted to a journal, then reacted, and there were to be further requests from the same reviewer to review the same paper, well, that means that review is already in the open. And it also means, again, from the journal perspective, that the preprint reviews allows them to complement their own peer review process, but also to have access to a new pool of potential experts whom they can consider as additions to the reviewer pool, or their editorial boards.
IRATXE PUEBLA: And then of course, importantly, given that preprints are posted without preprint review, having that feedback, that commentary provides additional context and expert evaluation on the preprints that is available to readers, not only to other scientists, but also, members of the public who can also get a glimpse then on how these conversations between scientists work. And the fact that science is an iterative process, and self-correcting process before we can establish some research findings as established evidence.
IRATXE PUEBLA: So there are a number of benefits to this, but what we are seeing is that the participation in public preprint review, at least so far, has been relatively low. And I'm sure there are a number of factors that fit into this. But I'm convinced that many of these are cultural issues. Elements related to perhaps the legacy of how we interpret peer reviews, and so we are very familiar with the review process of journals.
IRATXE PUEBLA: We are comfortable with that. How can we start trying to do it in a different space, in a different way. And also, potentially, the fact that there is a diversity, which is obviously positive, but this diversity of platforms and models around preprint review mean that it can be a little bit of a perhaps, confusing and fragmented ecosystem for those who come into this space.
IRATXE PUEBLA: We are going to be discussing many of these elements, and so with our speakers. But before I hand it over to them, I wanted to mention very briefly a recent initiative in this space, which is the first principles for preprint feedback that have been developed by ASAPbio with input from a working group that had representatives of the community in science communication.
IRATXE PUEBLA: They are a set of 14 principles that are encompassing the behaviors that we are expecting to see a positive and constructive preprint feedback being in the open. One of the things I want to mention is that we see the principles applying to anybody who participates in preprints, reviewers, of course. But also, the authors of preprints who may respond to any comments on their work, as well as members of the community who can play an important role in reinforcing positive behaviors, but also calling out those behaviors that are less positive.
IRATXE PUEBLA: So without further ado, I'm now going to be asking each of the three speakers to each introduce themselves, and also, so that they tell us a bit about the perspective that they bring to this discussion. So we are going to be starting with Gautum Dey, who is group leader at the European Molecular Biology Laboratory. So Gautum, do you want to introduce yourself, and share a bit of your perspective around preprint review?
GAUTUM DEY: Absolutely. Thanks so much, Iratxe. And thanks, Iratxe and Daniela for the invitation to be part of this discussion. Overall, the community attending this conference is not the community that I engage with on a daily basis, so I am also excited to hear about this perspective. So yeah, my day job is as the leader of a young research group at EMBL. We're cell biologists, and we study the organization of the nucleus.
GAUTUM DEY: And this is what occupies most of my time. But I also have some other hats related to my interest in publishing reform, advocating for preprints, and also, the publishing process in general. Because I believe that scientists should really participate in each of these steps, actively practicing scientists, contribute, give back to the community, but also find ways for the system to evolve in a changing world.
GAUTUM DEY: And so I sort of just put up some of these organizations here, organizations that I'm involved with, with bioRxiv, one of the big preprint platforms for the life sciences. I help screen preprints as part of a large group of volunteers. I also volunteer for ASAPbio. For example, I was part of this working group that Iratxe mentioned putting together fast principles for preprint review.
GAUTUM DEY: And I've been writing for some time for a preprint sort of highlight and discussion forum called preLights, that is one of the ways that the community is trying to use to encourage discussion around preprints. I also serve on some early career editorial boards, and editorial boards for a few community run journals, and community run initiatives, like review comments that incorporate preprint review into a sort of formalized format that's kind of like a bridge between, let's say, a traditional journal and a pure overlay.
GAUTUM DEY: And so in these roles, I've become conscious of the fact, just to echo something Iratxe said, that the infrastructure, and the broader kind of preprint, preprint and community, and preprint ecosystem is evolving rather rapidly, and is ready for this next phase of incorporating public review. But the scientists that are producing the research are always a little bit slow to follow, partly, because cultural change is never sudden, or at least has is never sudden until it reaches a certain critical mass.
GAUTUM DEY: And so for example, discussions about commenting on preprints in a formalized fashion in accordance with generalized principles, such as the ones Iratxe just mentioned is not a mainstream conversation amongst academics. And so my main interest here is to brainstorm with you ways for us to make that conversation more mainstream, and to bring more people into the discussion, and also, to simply educate or inform, and to let people know that these frameworks exist.
GAUTUM DEY: And instead of posting comments on Twitter, or writing emails to their colleagues when they see a preprint posted, which are forms of common practice at the moment in the life sciences, perhaps, we could move towards an ecosystem that is more streamlined, and allows faster, more efficient, and more transparent feedback on research. So yeah, so that's why I'm here today.
GAUTUM DEY: And I'm looking forward to the rest of the discussion. And Iratxe, please do move on to the next speaker.
IRATXE PUEBLA: Thank you so much, Gautum. Right, so we are now going to be hearing from Joy Owango. Joy is the Founding Director of the Training Center in Communication Africa, and is also a Board Member of the preprint server, Africa Archive. So Joy, would you like to say a few words about yourself, and again, your activities in preprint review?
JOY OWANGO: Thank you. Thank you so much, Iratxe. As you have mentioned, I'm the Executive Director of the Training Center and Communication. And we are based at the University of Nairobi. We are a research capacity trust that supports researchers, research institutes, and government on how they can improve their research output and increase their visibility. Over the last 15 years, we've supported over 10,000 researchers, and we've won a couple of awards.
JOY OWANGO: And what we've noted is that, the best way to support the researchers is also through mentorship, which is expected, but the mentorship is not as active with research who are self-funded. So we set up a mentorship group with over 900 researchers from Africa. And it's a hand-holding process in their research lifecycle from research idea to publish it. The next slide, please.
JOY OWANGO: So our premise is to train, support, and empower researchers. And also, when it comes to now the activities involved in supporting institutes, especially in the institutional capacity strengthening, our premise is on the research life cycle of institute, of organizations in higher education. So you're looking at granting councils, research councils, university commissions, and libraries.
JOY OWANGO: We need to build their capacities through education, facilitation, and partnership, so that they can effectively support early career researchers. And it is through such initiatives that we partnered with IDA Africa, PREreview, and eLife, and Africa Archive in coming up with a peer review program aimed at supporting early career African researchers, and with the objective of creating best practices, and innovative approaches in peer review.
JOY OWANGO: Now, the reason why we did this is because we needed to raise awareness around preprints, and foster the participation of African researchers in peer review, especially in open peer review of preprints. So the reality is, African researchers are quickly adopting preprints as a way as part of the academic publishing process, and as a way of increasing their research visibility. But we want to also involve them in the peer review process.
JOY OWANGO: There have been quite a bit of inequities when it comes to peer review from being done on researchers from Africa, due to existing biases. And through this training, we'll be building the capacity of early career researchers, or even mid-level researchers in understanding on what is expected in peer review, the best practice in peer review, and how they can also support future early career researchers in the process.
JOY OWANGO: So this is an ongoing process, and it also includes reviewing of preprints. So it's quite an exciting time for all of us and our partners in highlighting how we can support African researchers in adopting this process that they've already started getting involved in in preprints, and also the peer review of preprints. Thank you so much.
JOY OWANGO: Iratxe, you're on mute.
IRATXE PUEBLA: We would have expected by now I would be better at this with the microphones. Thank you so much, Joy, for sharing that perspective. Let me just pull up the slides again for our third speaker, who is Laurie Goodman. Laurie is the Policy Director for GigaScience Press, and is bringing more of a journal perspective to the conversation. So Laurie, would you like to say a few words about yourself, and again, what is GigaScience doing in this space?
LAURIE GOODMAN: Yeah, hi. And thank you very much for inviting me. I have been in publishing for now over 25 years, so I have perspective from journals trying to hide everything they do to journals now moving into first open access, and now, moving into trying to push everything open. And I think this is an essential part of publishing, because I feel like a lot of the hiding of information really held back researchers in understanding the whole process.
LAURIE GOODMAN: I don't want to say they were doing this for evil purposes. There was an idea that it was important to hide information as a means of protecting people, and also protecting the public for misinformation getting out there. Which I have a serious feeling that instead of putting out that scarves can solve arthritis, that we put out scientific information that the public can read. So anyway, we published two journals.
LAURIE GOODMAN: Both of them are something we call open science, because everything we do is completely open and transparent. The two journals we publish are GigaScience and Gigabyte. For GigaScience, we encourage the use of preprint servers, and that's sort of historical, and I know we're going to move toward you have to put it in a preprint server. I should mention that GigaScience, even though we don't mandate it, about 35% of the papers that people submit are already in preprint servers.
LAURIE GOODMAN: And then Gigabyte, 100% are in preprint servers, and it spans across all of the preprint servers, although, most of them are in bioRxiv, we have two that came from Africa Archive. And it's really exciting, because unfortunately, I knew it-- but until Joy was here, I realized, boy, we interact with the other preprint servers. And so I'm hoping to have conversations with them about what journals can do.
LAURIE GOODMAN: Because journals can bring a lot to the preprint area, especially if we are embracing that. And more and more journals are embracing that, and saying it's fine if you put it in a preprint server, which before, they were saying, don't do that. So I want to point out first of all, why you should use a preprint server. And a lot of it is because we use the preprint servers.
LAURIE GOODMAN: Right now, a lot of journals use preprint servers to find papers. Someone tweeted a couple of years ago, wow, I used to have to email all these editors to see if my paper was suitable. And now, editors are emailing me. So editors actually do use these to find papers that are suitable for their journal. When we launched 10 years ago-- it's our 10th anniversary-- the first thing we started doing was going through-- at that time, it was just archive for papers that we knew.
LAURIE GOODMAN: And we have done that consistently. The other thing is, we use preprint server is not only to look at authors of similar type papers, but also, if there are reviews there, to find suitable reviewers. If the preprint has a review on it, the authors really get us much faster review process, which everyone wants. Because we will use the reviews on a preprint server if we know the name, because we mandate that our peer reviews are named, so the authors know who they are.
LAURIE GOODMAN: And then we publish the peer reviews, so that the community knows who they are. Because we think this is an essential part of the scientific process. So if there's a review, which is what we're trying to encourage, you will speed authors. So you want to do reviews, and you want people to do reviews. The other thing is, is having reviews and a preprint server can provide training tools for teaching young researchers how to review.
LAURIE GOODMAN: They can read through it. They can try it. They can practice it. They can do it with other people. So if you're afraid to put a review on, have another person, your advisor, sign off on it, and put their name there, so that you feel supported. And you are supported. But mostly, it's to feel comfortable.
LAURIE GOODMAN: The other thing is that we find is that, having reviews in a preprint server, again, can aid authors if they're writing a similar type of paper. We find as editors, that specific papers that are in a similar genre almost always, the reviewers say, they need to do this special validation that for some reason nobody does. If you look in a preprint server, and there is a review, you can look at that and say, oh, every single reviewer asked for this.
LAURIE GOODMAN: I'm going to make sure that's in my paper. Having things ready for the reviewers already reduces the comments. So it's really a good way for avoiding the most common reviewer concerns. So again, that's why you should put reviews in, because you're helping other people. But it can also be done to help yourself. And then finally, when young people post reviews in a preprint server, it provides a record of expertise for job applications.
LAURIE GOODMAN: So I know people are anxious about doing it. But we found that the fear of reprisal is actually more of a boogeyman. There is no evidence that there is a huge reprisal of people who put their names in. And in fact, if there is a reprisal, and it's public, which it usually is, everyone can see that and comment on whether that's true or not.
LAURIE GOODMAN: And also say, hey, why are you inhibiting young people from doing it? Anyhow, we are heavily involved in trying to promote every area of preprint servers. And right now, we've been involved with ASAPbio in promoting reviews. We also, if we have a peer review, post it back onto preprint servers. Since our reviewers are named, they already know that everybody is going to see it.
LAURIE GOODMAN: And it's really fun in real time every time we post a review back on bioRxivs, they tweet it. There's this automatic robot tweet. So when we have one of our interns doing the posting of back posting of reviews and you're on Twitter, you can see just real time our interns working, so poor people. And then there's a new thing that's coming out called SCIETY, S- C- I- E- T- Y, which is a organization that's trying to funnel all of the peer reviews and all of the information from preprints into one area to help people navigate the big peer review process throughout to reduce the fragmentation, but also, make it much easier to find the reviews, find specific preprints.
LAURIE GOODMAN: So we're really excited to have started that. And as I mentioned, any time we see something new, sometimes, we're a little pushy. We want to be involved. We want to be involved, we think it's essential that journals participate, and become partners in growing this process. So thank you.
IRATXE PUEBLA: Yeah, thank you so much to all of you for sharing that perspective. I guess to get the conversation going, I wanted to come and touch back on a couple of things that you mentioned. I'm going to begin on Laurie's comment out the fear of reprisal, because I think that's something that tends to come up in conversation about posting comments publicly, and making ourselves vulnerable to exposing our potential critiques.
IRATXE PUEBLA: And I wanted to see what your thoughts were about-- essentially, do you think that this is a main driver for potentially, again, having researchers participate in this type of review activity? And if that is the case, whether you think there are any differences across career states, and we often hear or there is the power structure in place. It may be trickier for early career researchers to participate.
IRATXE PUEBLA: But there may be a lot of disparities, again, also depending on where are we based. We may have different review cultures. I think I want to start by asking Gautum for some thoughts on this from your perspective as a researcher. Is this something that, again, you may have encountered, or do you have some suggestions about?
GAUTUM DEY: Yeah, I mean, yeah, I think the debate about signing reviews in a way is just going to carry over. Because it's a long standing discussion within the context of traditional peer review. And I don't see those problems going away. I think here, are the perceptions are almost as important as the reality. So even if it turns out there are no documented reprisals, I think young, or early career researchers may still hesitate to lend their name to critique, even if it's constructive of a paper.
GAUTUM DEY: However, I think it's linked to this idea of separating, or of making peer review, or preprint review more objective. So if there are objective criteria that are being discussed, if it is constrained to constructive criticism of the research, plans, ideas, conclusions within this report or document, and it has no touch of either personal opinion, or value judgment, or any of the things that can be associated with peer review reports, then the potential threat to anybody of reprisal, or even be subject to accusations of inappropriate behavior does fall away.
GAUTUM DEY: So I think hopefully, these two will go hand in hand, that as it becomes normalized to review papers in this way, such that the review is a genuine piece of added scientific value, and not a social commentary on the authors of their work, then it will also become normalized for people to be able to post this at all stages of their careers, as long as they have the relevant expertise to comment on the subject matter.
GAUTUM DEY: So I guess, it's a hopeful thought. But I certainly agree that it is an issue that must be dealt with.
IRATXE PUEBLA: Joy, I wonder if you have any perspective on this. And again, you have been doing mentoring for researchers. I wonder, again, do you see these patterns around? Is this more of an issue for early career researchers? And I would also be interested in getting your perspective from working with African results. Again, are there any differences there in terms of culture that we may need to factor in as we try to normalize this type of activity, as Gautum was saying?
JOY OWANGO: OK, first of all, academic publishing is evolving. And the fact that African researchers are adopting these new technologies, like the use of preprints, of course, it comes with some of the issues that come with it, you know, if it means open peer review, and being part of that process. The reality is, we've had early career researchers involved in the peer review process, but then it has been closed.
JOY OWANGO: You know, because it's behind, you end up finding out much later. So what's the difference now? Because now, you can see the person, and now, it's in the open? What I'm trying to say, the fact, the mere fact that they are really accommodating these new technologies in academic publishing, and they've already been allowing early career researchers to be involved in the peer review process, albeit, closed, and closed in the closed peer review, I don't see any particular reason why they should be castigated to be involved in the open peer review system, particularly those who have already started getting the experience of peer review, whether it's in the in the closed peer review process.
JOY OWANGO: So that's my opinion. Then number two, the cultural aspect, yes. The average person with a PhD and above in Africa is in their 40s. It's getting lower, which is a good thing. And that means the early career researchers are in their 30s, or in their late 20s. The cultural dynamics in a continent that is still embedded in respecting your elders, regardless of the career path you're taking, and you're taking, means that, the poor early career researcher does not have a voice per se, unless the senior researcher is the one guiding them, and telling them it's OK to go through this process, or it's OK to present your view and say, this is wrong.
JOY OWANGO: All right. So the cultural dynamics need to be factored. And also, in regards to the cultural dynamics being factored, it means the capacity building, this is why we're doing this kind of training on understanding peer review systems, and how to peer review, how to train peer reviewers. Mainly, because at the end of the day, this early career researcher will end up working with a mature researcher.
JOY OWANGO: So it's also trying to break those barriers such that they are all on the same space in supporting the research life cycle, or in supporting their research life cycle. So yes, it is a problem. But we cannot ignore the fact that these early career researchers have already been involved in close peer review. What is the difference now? Because you can put a face to it, there's an open sign off to it?
JOY OWANGO: So for those who have been complaining, I always wonder where the issue is. Because you had already been allowing early career researchers to be involved in the close peer review process, yes.
IRATXE PUEBLA: Thank you so much for sharing that important perspective. I'm going to kind of move on to something slightly related to this potential reprisal element, but thinking a bit more about the structure. Something else that has come up in other conversations about this is that some may think all of this public commentary is all well and good if we are all angels who behave correctly. But the reality of things is that in the general process, we have a dedicated person who is tasked with policing this whole thing.
IRATXE PUEBLA: We have the editor. There is a framework, and kind of protected if something comes up. While there is not necessarily-- I mean, the moderation processes, I don't think are necessarily set across the different preprint review modalities. And I just wonder what your perspective is on this, whether essentially, they think this can operate without some form of basic standards of moderation?
IRATXE PUEBLA: Or do we start having a spectrum? I'm going to maybe perhaps ask Laurie to comment on this first from the journal perspective, and then Joy. Laurie?
LAURIE GOODMAN: Yeah, I just need to touch a little bit-- I won't go into it-- but on the previous thing. In terms of reprisal, if the peer review is hidden, you can still be reprised on it. I don't know how many people guessed who the reviewer was. And 95% of the time they were wrong. So the reprisal still happens, but unfairly. So if we're going to be fair, put your name out there. And they're always much more positive than the hidden review.
LAURIE GOODMAN: So I forgot your question, because I was still talking.
IRATXE PUEBLA: The potential moderation requirements for the new ecosystem, where we have a spectrum of you can comment and review.
LAURIE GOODMAN: I mean, with the editors making that process, each journal will have a set of biases. And a lot of these aren't well known, not to them even. I mean, there's always an unconscious bias. And I think that it's really important from the standpoint of both reviewers and editors to have some sort of guidance. A lot of institutions are putting in training programs to avoid bias.
LAURIE GOODMAN: And I feel like this is something that publishing needs, that editors should be taught and trained on how they might be in unknowing ways, biased, and that reviewers need to be trained. And that can start with young reviewers, especially. But if editors are aware of it, they have a better way of assessing whether they think that's happening. But I think that is probably one of the best ways of trying to remove bias, I think a training program.
LAURIE GOODMAN: It's a weird thing to say about editors. But it's a big institution. So instead of just being within a university, it's an institution. It should be worldwide. What are biases going in? It's something that maybe preprint servers could be involved in, because you're getting in the earliest reviews.
LAURIE GOODMAN: And so I'm not quite sure how that would be done, and you preprint server people should figure it out.
IRATXE PUEBLA: Joy, would you like to add to this?
JOY OWANGO: Yeah. And there is a need to have some basic standards and regulations when you're peer reviewing, whether it's open, so that you can make it more inclusive. The basis of peer review has always been really harsh. And that is why there's been also movement on peer review on being kind. Because you as much as you're critiquing someone's work, you do not know the time and effort they took in putting that project together, or that paper together.
JOY OWANGO: And the fact that today, we are much more aware of various social constructs, you know, like mental health. And we do not know what our insensitive point might do to this person. So we are much more aware. So what does that mean? And the fact that we are in an online environment, be kind. So we need to put in standards, basic standards that would also guide any open reviewer that any open reviewer will use when we work with when they're reviewing any preprints, or doing any open peer review.
JOY OWANGO: But then we also need to give publishers some credit. I never saw this five years ago. And this year alone, I have actually between late December and this month, we've been approached, TCC Africa has been approached by a number of publishers, because they want to train their audience on peer review. The editors are making an effort. Some of them have even set up programs just purposely focusing on peer review, what it entails, what their specific publishing houses expect.
JOY OWANGO: And also, which I find also quite fascinating is, they're not teaching their audience on peer review, but they're also teaching their audience to be peer reviewers. You see, they're becoming a bit proactive. And this has purely happened because everything is happening online. Now, the process is open. And people are getting more aware. And also, in terms of the bias, especially in Africa, now, we are getting a voice in raising concerns on the bias when it comes to peer review in the continent.
JOY OWANGO: And that is why training programs are being created. And just like Laurie said, yes, capacity building is what will help in mitigating this challenge. And all hope is not lost. What I'm saying is that with the rise of preprints, and preprint servers, and with us going online, up has created a snowball effect as a face in academic publishing, which was otherwise hidden. Now, it is open.
JOY OWANGO: And now, people are seeing the bias in it, and some of the challenges that were involved in it. And the fact that editors are coming to organizations to ask them, how can we help our audience in understanding peer review, or how can we train more peer reviewers? That's a step in the right direction. So I think it's, yes, it's a positive trend when it comes to peer review. It's been demystified.
JOY OWANGO: People are becoming more kind, because it's about critiquing. But people are becoming more kind, and becoming more conscious, especially when it comes to sharing their view or opinion on the work that they're reviewing.
IRATXE PUEBLA: Thank you so much. An important reminder that at the end of the day, it's the same researchers who will comment on review preprints and journal articles. Essentially, there is a continuum here. And I wanted to ask one more question. And here, I'm going to ask all of you to contribute, which is, as I mentioned at the beginning, we have a number of platforms and projects available to comment on preprints, which I think is great to see the diversity.
IRATXE PUEBLA: You know, it's a new experimentation in different ways. But it means that if I put a researcher hat on, and I want to contribute, I may be slightly daunted by the diversity of options, and I may say, where do I go? What should I post my comments? What's going to happen? How is it going to be used? So I wanted to ask each of you to briefly share your thoughts as to what do you recommend in terms of steps to make this perhaps a bit more clear for researchers, to perhaps add some clarity, so that it is also an easier path to participation for researchers who want to contribute preprint research?
IRATXE PUEBLA: Gautum, starting from you.
GAUTUM DEY: Yeah, sure. I mean, yeah, I mean, it's tricky in the sense that it's one of those things where maybe it's also important to see what sticks. So it may be that some approaches become more intuitive to people to use than others, and that's not obvious from the get go. I mean, having said that, I found that one of the formats that seemed intuitively most accessible to people, because it's already very close to something that people do regularly is, some form of being able to transfer journal club for discussion format to the page, right?
GAUTUM DEY: So this is in a way a plug for a peer review type approach, where we're doing this all the time anyway in our research groups. We read preprints. We discuss them. And what will often happen, especially with maybe let's say, slightly more traditional research groups, as they might collate the feedback, and email the authors, right?
GAUTUM DEY: So I mean, this is just a very, very small step from there to posting it formally. And so for me, my intuition would be that that's an easy entry point. And maybe that's one of the things that can be-- I mean, it is already being promoted, but it's maybe one of the things that can be a part of a kind of global effort to spearhead more open journal clubs, basically.
GAUTUM DEY:
IRATXE PUEBLA: Great. So yes, I'm a great fan of journal clubs. Thank you for that, Gautum. Laurie, what would be your suggestion if we wanted to, again, try to help researchers navigate this space, anything we can add to add clarity?
LAURIE GOODMAN: Yeah, I mean, there are a lot of preprint servers. One of the ways to do it is to actually do a Google search, or a Google Scholar search, because preprints are in the search. I guess, I would do a Google Scholar search, because preprints are recorded in that. So you can do that and find papers across the board of those different preprint servers. It's a great way.
LAURIE GOODMAN: And I do need to comment, because I always comment on everything. I love the idea of having journal club comment and be part of the review process. And now, I'm trying to think about how a journal could engage that, of course. I mean, I remember when we did our journal clubs when I was a graduate student. We talked really in detail through all the papers, what we liked, what we were concerned about.
LAURIE GOODMAN: And it was sort of a brainstorming section. So now, I'm going to be thinking about that. But for finding preprints, I suggest Google Scholar. And hopefully, SCIETY will be one of the hubs to bring it together. I like the fact that there are preprint servers across the world. It takes care of biases. They know specifically what the problems are.
LAURIE GOODMAN: So I like the fact. I wouldn't say we all want one preprint server, especially since there are ways to do it. I think it's essential that preprint servers are all around the world.
IRATXE PUEBLA: Joy, any further suggestions from you?
JOY OWANGO: Let's accommodate what-- to go back to what I was saying, let's give credit where it's due. Because with publishers today, they are more conscious about their audience, because the audience has a voice. OK. So what does that mean? We can see that there is that effort being-- so for potential partners who are involved in research capacity building, especially in the research lifecycle, either early career researchers, or mid-level researchers, don't hold back to reach out to publishers, especially when it comes to supporting researchers s in understanding peer review, in understanding how preprint servers work.
JOY OWANGO: Because they are now reaching out. They are now aggressively reaching out to organizations to see how they can support their audience. So gone are the days when they would say, OK, yes, we are going to train you on academic writing. They are going through the entire research lifecycle. So yes, you can get the preprints, preprints going through any sites. Today, we have more bibliographic and citation databases than we had five years ago.
JOY OWANGO: And one of the filters that you can get publication types on is preprints. So it's slowly becoming the norm, and that is exciting. So we shouldn't be afraid of it. But then for organizations, I'll just say, reach out as well. For research capacity organizations, reach out to publishers, because publishers are looking for capacity building partners who they can work with in supporting their audience in understanding best practice in peer review, understanding in mentorship, and peer review, so that things are changing.
JOY OWANGO: Basically, things are changing for the best.
IRATXE PUEBLA: Thank you so much, Joy, and all of you for sharing your perspective and ideas. And we are going to now close this part of the discussion. Obviously, many other things to discuss about preprint review. But I'm hoping that there will be plenty of opportunities through the questions with the audience. So thank you so much, and I look forward to the discussion with the audience as well. [MUSIC PLAYING]
Form Title Download
Video
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode