Cadmore media player playing video From Pencils to Processors: How AI is Cracking the Peer Review Puzzle
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
From Pencils to Processors: How AI is Cracking the Peer Review Puzzle
Description:
From Pencils to Processors: How AI is Cracking the Peer Review Puzzle
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/202de7c3-d985-4f21-aead-0cb47110cdfd/videoscrubberimages/Scrubber_1.jpg
Duration:
T00H29M30S
Embed URL:
https://stream.cadmore.media/player/202de7c3-d985-4f21-aead-0cb47110cdfd
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/202de7c3-d985-4f21-aead-0cb47110cdfd/SSP2025 5-28 1200 - Industry Breakout - MDPI.mp4?sv=2019-02-02&sr=c&sig=vMtxtwkGVWIVlyHyDofsl%2FE4igi%2F4kty9PY65ktxr8U%3D&st=2025-06-15T20%3A07%3A54Z&se=2025-06-15T22%3A12%3A54Z&sp=r
Upload Date:
2025-06-05T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
So hello everyone. My name is milosh and today, together with my colleague Henrik from MDPI, I will present to you on how AI is cracking the peer review puzzle. So we will go through a couple of slides that presents roughly the tech innovation effort we do and have at MDPI. So before we start with the slides, a few words about myself.
I said, my name is milosh. I'm holding a PhD in computer science, majored in AI and software engineering. If someone is willing to connect to hesitate to shoot the QR code for LinkedIn or just get in touch over email with myself. So I am currently acting as head of tech innovation at MDPI, and I do work with a team of AI engineers and data scientists on tech innovation projects. And who says tech innovation says AI currently, which is a bit hot topic of most of the conferences related or not to academic publishing.
So I will hand the laser and the mic to Henrik. Thank you. OK, so. Hello, everybody. I'm Henrik and I'm working on. I started working on MDPI eight years ago and I'm based in the Barcelona office. I started as an assistant editor, so I mainly focused on editorial matters.
And recently I transferred to the AI team as a business analyst, where I use my editorial expertise in order to help the AI team define and the processes and the issues that we want to address using this new technology. So briefly, our talk will be divided in these five parts. We start doing a brief introduction about MDPI. Then we will discuss the current developments of artificial intelligence and how it is impacting our industry.
Then we would like to present one of the several challenges that are present in the peer review process and how we think we can alleviate this challenge using this new technology. And finally, we would like to briefly discuss some other MDPI initiatives. So MDPI, as you might imagine, is a scholarly publishing publisher, and we are a pioneer in scholarly open access publishing.
We are the leading pure open access, and it was founded in 1996. Our business model is Gordon open access, so our publications are free to read and access as soon as they are published, and we collect fees from the authors once the manuscripts are accepted. So why we use this model. So basically, we believe that open access allows for faster scientific progress and dissemination.
Since the manuscripts are really available, as soon as they are accepted, they provide more visibility to this research and it helps bridging gaps such as geographical or financial gaps in order to equalize the fields so any research institution can get access easier and faster to the latest research. Some other data about MDPI. We currently have more than 400 journals Among those, 237 have an impact factor, and I think 2/3 of them are placed either in Q1 or Q2.
90 journals are indexed in PubMed central or PubMed because they deal with medical research. And I also like to point out this. Let me see the laser. This information about another survey that we conducted last year, in which we observed that 92% of our authors rated their overall experience regarding the peer review process as good or very good, and 95% of those also rated their overall experience with the paper processing as also excellent or good.
Just to finalize the introduction with MDPI, just to point out that we are an international team of more than 66,000 employees, distributed about around 21 offices in 12 different countries. 2/3 of our staff are dedicated to editorial matters, so their function is to collaborate with the authors, with reviewers and with academic editors in order to ensure a smooth paper processing and to make this process as easy and efficient as possible to the authors, to the reviewers, and to everyone involved.
So now mirrors. I'll read them to you. Thank you. Thank you Eric. So we are presenting AI today, but this is an example of how AI doesn't work well with this translation tool. Sorry for that.
Good so what is the current state of AI in academic publishing at least. How do we see it from the publisher side, from mdpi? We all know AI is revolutionizing the world with especially generative AI, so adopting AI is no longer an option. It's like a must for most of the medium to large sized publishers also included MDPI. So we started a couple of years ago with a team of mainly data scientists working on different AI projects that will help us with increasing the quality, the speed and also the quality of service we give to our authors.
We have divided the AI use cases in two categories. On the left for scientists, researchers, and also on the right for publishers as MDPI. So researchers use AI to usually Polish English language improve the tone of the manuscript. If they're not English native speakers, they use AI to generate code, generate a different content faster. Summarized research literature.
Brainstorm about research topics and ideas and so on and so forth. On the other side, the publishers, we use AI to find peer reviewers. We use it to check for ethical issues and breaches in submissions we receive. We use AI to assist peer reviewers during the review process. Also to find potential authors, promote their journals, write presentations, emails, and also provide customer support.
Chatbots so those are some of the use cases that we see as interesting. When I came to the stage, however, there is also an always a dark side of mostly every new technology, and I would like to highlight some of them here. First of all, there is or there was a big authorship attribution debate when we talk about AI. If you search on Google Scholar for articles once ChatGPT went out in November 2022, if I recall it correctly, you'll find a couple of papers where one of the co-authors was ChatGPT or any other chatbots.
And back in time. So this is a debate that is hopefully closed now. But it was present at the beginning. It then makes plagiarism easier and harder to detect, which is a big issue for us as publishers. It brings mistakes or inaccuracies into papers. It allows creating fake papers, but also more important, fake peer review reports. And last but not least, it Gen AI currently has the inability to draw reliable scientific conclusions.
So that's something that is very important for scientists whenever they use AI to write their papers. So MDPI and most of the large publishers nowadays they follow code standards. Whenever we talk about the usage of AI in academic publishing. So authors are not allowed to list those Gen AI chatbots as co-authors, they can use AI without any disclosure. If we talk about English language polishing about refinement of manuscript content.
However, they need to disclose whenever they use AI for any other purposes, which means they used AI to generate maybe a data set to help running an experiments, and so on and so forth. So that's something that we as a publisher do require authors to dissolve whenever they use AI within their manuscripts. On the other side, we have the peer reviewers and the review process.
So AI is strictly forbidden, at least at MDPI in the peer review process itself. So it comes mainly from the fact that the authorship or the copyright of the paper belongs to the author before the paper gets published, which means that the peer reviewers are not allowed to upload the manuscript on any of those third party AI chatbots. So I will hand it back to Henrik now to describe you. What is the preview puzzle in academic publishing.
OK, so as the peer review is an integral part of our business of processing and publishing manuscripts. There are several issues going on regarding the peer review, and we could discuss about it for hours. In particular, I'd like to focus on this issue that is summarized in this graph over here. So the upper line, the darker blue one represents the number of researchers over the years. As expected, it's increasing.
And the light blue line represents the number of papers published during those same years. And as you can see, the number of papers published is increasing faster than the number of researchers in actually, it's increasing nearly twice as fast. Now if we translate this into peer review reports needed to publish these papers. We get this graph here. We are not only taking into account the peer review reports needed for covering all the published papers, but also those papers that are rejected after peer review because those also generated peer review reports of their own.
And this right over here represents that, assuming that all the researchers are engaged in this peer review process, which is not true, we see that the number of peer review reports yearly that these researchers should do in order to cover and generate all these peer reviewed reports, is also increasing over time. Just to summarize the number of reviews per researcher has increased from 2.6 to 3.3, nearly one third during this time period.
So the issue here is how can we use these kind of technologies in order to alleviate this extra strain that we are putting into the research community. So this is what we are going to discuss now. Thanks good. So now that we saw what is the peer review puzzle in academic publishing, I will try to describe you as much as possible how we do try to crack that puzzle at MDPI.
So we do believe that if not solve the puzzle, at least help. It's solving it through tech innovation efforts. So I would like to present you quickly. What is the purpose and the mission of the tech innovation team within our company, but also then go more into details of some of the projects and products we have developed. So first of all, our main mission is to increase the quality and the speed of services we do provide to researchers.
So this is the North Star is the main goal. And we try to do this through leveraging AI and data driven mindset. Also doing two additional things. One is to train our internal staff, our assistant editors, by empowering their efficiency and quality work, but also empower researchers, which means authors, editorial board members and reviewers to be more efficient.
Again, using the tools we are developing there on the right side of the screen, you have, it's probably too small if you're in the back, but just very briefly, it's something we call the innovation project life cycle, and it's divided in four categories. We first try to identify the problem external impacting problems or internal processes improvement. We try to stay tech savvy.
We then need to identify and prioritize different ideas and projects. Nowadays, when people talk about AI, they do have a lot of ideas, unfortunately, and we cannot fulfill all of them. And finally, we go to the implementation phase where we run a POC. We do try to first test the tools internally with a smaller group of people through pilots, and then go and scale up and go into production.
So something we'll mention together, me and Henrik, during this presentation, we always pay attention on the human in the loop approach, which means that any of the tools that we present to you next are not meant to do any final decisions. So it's always raising flags, proposing the next steps. But we have people, humans inside a company that will then have to judge how truth or how good that AI proposal was made.
This is the editorial process at MDPI. It's not diverse from many of the other publishers, so we go from the pre-submission stage until the submission through all those technical pre-checks, peer review, internal decision, and then if the paper gets accepted, we move to the production and finally to publish the paper. So if not every but most of the tools we have developed using AI, they are stick to the tightly stick to this process here.
So we start with the tool that we call journal finder that allows our customers to authors to select the right journal they can publish their paper in. So it's something that is freely available on apple.com website, where our future customers can upload the title and abstract of their manuscript, and then they get a list of proposed journals that they can submit into it. Mdpi has over 450 journals, so it's an interesting tool for them to select the right one that matches on one side the article, but also on the other side, the criteria regarding the impact factor, the price of the gold open access, et cetera within the same technology, we have something we call in scope checker.
So in scope checker is run on our site on every single submission to make sure that the scope of the submission matches what the scope of the journal. So that way also we ensure not to publish out of scope papers or journal out of scope papers. Then we have the finder. Reviewer finder project that is currently in a late POC phase, so it allows our internal editors to match the best experts in the field to be invited to do peer review for a given manuscript for us.
So again, I will propose a list of people. Then a human will have to decide who to invite after some more detailed checks there. If you ask me, it's probably one of our most important products at the moment. Maybe Henrik will disagree. It's called ethicality. So ethicality.
It's used to check for a list of potential data breaches that we may have within a given manuscript. So it raises flags, and our editors then need to go through solve them internally if possible. If it's not solvable internally, they will go out and approach authors directly or eventually at the end of the journey, the editor in chief. If this needs to be scaled up, we have another project that is called researcher profiles, where we do build our internal researcher profile database.
That is quite also important for that ethicality tool XML conversion I assisted. We are in a very early stage of implementation in there, so there is nothing yet tangible. And finally, last but not least, we are also working on developing customer support chatbots. So we are running now through a proof of concept for an internal chatbot. And we will move forward hopefully by the end of this year with an external chatbot that will be used by our customers to go through all sorts of FAQs or eventually ask customer support.
So in the next couple of slides, Henrik will introduce you to the reviewer finder more into details what this product does, and then I will continue with the totality so that you can have a bit more of insights. How those two products can crack that peer review puzzle that was introduced by Henrik before. OK, so now focusing on the reviewer finder. Here you have a brief diagram about how it works.
Basically we start from the submitted paper here. And we are able to there are several ways to do it. How to extract features. Out of these papers you can check the citation analysis network or in our case, you can extract information out of title and abstract using a natural language processing and tool that we have. Then we can cross-check this information in our database, which is called cited where.
Here we not only have MDPI papers, but we have information from papers from other publishers. We do a similar extract, feature extraction from these papers and using a technique called semantic search, we are able to identify those papers that are more similar to the target paper, the submitted paper. Then the next step is to extract author information out of the retrieved papers. And then we apply some business and logics in order to rank them and select the most suitable ones.
We also cross-referenced this with our internal reviewer database, just to make sure that there are no conflicts of interest that they were not recently contacted. We also check if they did the recent reviews, how good they were, and all this information is collected, processed, and the final output is a ranked list of potential reviewers for a given paper. This is the information that receives the assistant editor, what we call with which we do the final decision.
Again, as mentioned here, we do not aim to automate this process. The final decision will always be done by the assistant editor who has enough of expertise to check this list and select the most suitable reviewers for this paper. So what we aim to achieve with this tool is basically to have better targeted invitations, and this way reduce the amount of invites required to secure two or three peer review reports, and therefore reduce the strain of inviting and trying to secure these reviewers by contacting to the research community.
And of course, just to highlight again that the suggestions are always validated by the assistant editor. And now we will talk about the ethicality project. Good so I will move forward with the second project. We'll do more of an in-depth overview of how this works. So it's called ethicality. In the last years at MDPI, we have built an internal team of ethical or ethics experts working mainly on research integrity.
So this team of probably like 10 plus people now there are in charge of dealing with complaints on published papers, which is some sort of a curing mode. Once a paper gets published, they need to jump into it, spend literally months of work and understand if this paper needs to be retracted or not, depending on how critical or severe the breach the ethical breach is on the paper. At one point, we decided to move to the curing mode, which means that we do try to prevent as much as possible.
Of course, it would be nice, but I don't think we will be able to reach 100% But to prevent as much as possible of those papers going through the peer review process and being published with potential ethical issues. So doing this by hand, it's really impossible. Or we need to hire another 100 or 200 people to do it. So we decided to work on a tool that can help on raising flags and then spotting different content elements of the paper, where experts are not depending on how severe disease they can go through, and then not spend their whole time on reading the full paper, really going into details everywhere, but concentrate on those flagged content elements.
So what ethicality is meant to do at the end. It's not. We're not there yet. Unfortunately it's to spot for plagiarism paper Mills, fake papers, citation misconduct, image manipulation, and also authorship manipulation practices that we find in academic publishing. So here is what ethicality can do. For now, we start on the bottom of this page with a new submission manuscript coming into the system.
We go through an extraction phase where we do extract the key elements from the manuscript like Title abstract authors references the body of the paper with all the sections, and then we run given specific contents to different kind of smaller tool sets to detect all kinds of ethical breaches. So first of all, on the left, we do. Citation misconduct analysis, which means we try to spot self-citation percentage.
We try to spot out of scope, but also out of context citations. Then we try to check the authorship, which means we try to analyze and detect purchase authorship. Also fictitious authors. But last but not least, like citation cartels that may erase on author and citation manipulation. We then go through plagiarism detection by using a third party tool. I mean, we use iThenticate as many of the other publishers image manipulation, again, using a third party provider to detect duplicates, cross publisher duplicates, but also to detect in paper image manipulations in there.
We then go to the AI text detection. So we don't only try to isolate or to detect if there is AI used within the manuscript, because you saw it before. This is allowed and legit, but we try to isolate English polishing AI usage versus generated text detection. So that's quite tricky technically. And I think we have a tool that does it pretty well for now.
And finally, we also try to detect duplicate submission cross journals, but also to detect similar submissions across all journals in order to avoid salami slicing and other practices that are not allowed in the publishing business. Mdpi is also part of the SDM integrity hub since last year, so we do also benefit from all their checks they do on the other side. Cross publisher duplicate submissions.
We also try to use their platform to detect bad actors in their bad actors watchlist, do all sorts of DUI analysis, checking, Retraction Watch, and so on and so forth. Again, I would like to stress this for the fifth time in this presentation. Human in the loop is also something we try to stick to. It's not because the tool says something, and there is a proof here that it doesn't really work well, that we take decision purely based on that.
So those are flag raised. We go into details, we try to analyze, and then our editors are trained to take specific decisions on those parts there. I will let Henrik present you the last couple of slides. Just some initiatives from MDPI. So to end, we just wanted to show you some of other MDPI initiatives that we are working with right now.
The most important one for us is jumps. And in MDPI, most all the paper processing from the paper submission to its publication is done using an internal tool. It's called submission system. Susi in short, and this is a commercial platform that is essentially used for third party users. That encompasses all the processes, again from submission to publication.
It's a modular tool that has different functionalities like the submission system, production service, like production until the final version of the paper is ready website hosting, building infrastructure and editorial guidance. So if anyone is interested in this, they can just select the modules that are needed in their case. And we also wanted to highlight that all these different AI tools that we have discussed in this presentation will be also available in jams.
So we think this will be very useful for potential users. And also to finish, just briefly mentioned that we have several other initiatives. The one that I think is most interesting here is that as we have discussed, this is our scholarly literature database and this is the database that our reviewer finder tool queries in order to find potential reviewers to our submitted papers.
But we have also a preprint server, side profiles, which is an author database proceedings series MDPI books SCI forum, which is a website to organize conferences and other forms of exchange and collaboration, and also professional languages and editing services. So that's all from our side. Thank you very much for your attention. And now I think we have five minutes for a Q&A session. If you have any questions for us.
If you do, we would appreciate if you can use the mic in the middle because this is a recorded session. So in order for the questions to get record. We would appreciate if you go there, state your name and affiliation and ask the question. Are there questions.
I'm Erin flock, I'm coming from Paris. And regarding your willingness to empower the reviewers, I would like to know if you ever considered to pay them. Because we are suffering from a lot of journalists telling us, sorry, we haven't found the reviewers. Maybe reviews are always the same. I mean, it's one thing to identify them, it's another thing to push them to review in time.
And so waiting for three months or four months to get the answer from the journal. Because the editor. I mean, that could be an help to pay them. Not a big amount, but at least to show some consideration to them. So that's my question. Yeah thank you very much. That's a very interesting question.
And actually it's a tricky one also because we need to find ways to reward reviewers for their free effort because this is at the beginning, it. There's not enough recognition given to reviewers either monetarily or any other way. Now the problem if we start discussing about paying reviewers, we need to also discuss about potential conflict of interest. Because maybe you by paying them, you are incentivizing them to give you good review reports.
And we would need also to avoid this. Now regarding what we do at MDPI, we do not pay reviewers, but we do provide vouchers discounts that they can use later if they wish to submit to MDPI. We are also investigating other ways of rewarding the reviewers. One thing that one way that we are it's on the table right now, and I think it's interesting, is that we can give this voucher not only to the reviewer, but to the institution in which this reviewer works.
And this institution can get a pool of vouchers that anyone from that institution that would like to publish with MDPI could eventually use. Of course, there are many other ways in which we could try to reward reviewers, and this is a very interesting discussion to be had. Any other question related to the presentation to mdpi? The publishing topic.
OK, so let's start. Thank you very much for your attention and hope this was interesting.
Segment:0 .
So hello everyone. My name is milosh and today, together with my colleague Henrik from MDPI, I will present to you on how AI is cracking the peer review puzzle. So we will go through a couple of slides that presents roughly the tech innovation effort we do and have at MDPI. So before we start with the slides, a few words about myself.
I said, my name is milosh. I'm holding a PhD in computer science, majored in AI and software engineering. If someone is willing to connect to hesitate to shoot the QR code for LinkedIn or just get in touch over email with myself. So I am currently acting as head of tech innovation at MDPI, and I do work with a team of AI engineers and data scientists on tech innovation projects. And who says tech innovation says AI currently, which is a bit hot topic of most of the conferences related or not to academic publishing.
So I will hand the laser and the mic to Henrik. Thank you. OK, so. Hello, everybody. I'm Henrik and I'm working on. I started working on MDPI eight years ago and I'm based in the Barcelona office. I started as an assistant editor, so I mainly focused on editorial matters.
And recently I transferred to the AI team as a business analyst, where I use my editorial expertise in order to help the AI team define and the processes and the issues that we want to address using this new technology. So briefly, our talk will be divided in these five parts. We start doing a brief introduction about MDPI. Then we will discuss the current developments of artificial intelligence and how it is impacting our industry.
Then we would like to present one of the several challenges that are present in the peer review process and how we think we can alleviate this challenge using this new technology. And finally, we would like to briefly discuss some other MDPI initiatives. So MDPI, as you might imagine, is a scholarly publishing publisher, and we are a pioneer in scholarly open access publishing.
We are the leading pure open access, and it was founded in 1996. Our business model is Gordon open access, so our publications are free to read and access as soon as they are published, and we collect fees from the authors once the manuscripts are accepted. So why we use this model. So basically, we believe that open access allows for faster scientific progress and dissemination.
Since the manuscripts are really available, as soon as they are accepted, they provide more visibility to this research and it helps bridging gaps such as geographical or financial gaps in order to equalize the fields so any research institution can get access easier and faster to the latest research. Some other data about MDPI. We currently have more than 400 journals Among those, 237 have an impact factor, and I think 2/3 of them are placed either in Q1 or Q2.
90 journals are indexed in PubMed central or PubMed because they deal with medical research. And I also like to point out this. Let me see the laser. This information about another survey that we conducted last year, in which we observed that 92% of our authors rated their overall experience regarding the peer review process as good or very good, and 95% of those also rated their overall experience with the paper processing as also excellent or good.
Just to finalize the introduction with MDPI, just to point out that we are an international team of more than 66,000 employees, distributed about around 21 offices in 12 different countries. 2/3 of our staff are dedicated to editorial matters, so their function is to collaborate with the authors, with reviewers and with academic editors in order to ensure a smooth paper processing and to make this process as easy and efficient as possible to the authors, to the reviewers, and to everyone involved.
So now mirrors. I'll read them to you. Thank you. Thank you Eric. So we are presenting AI today, but this is an example of how AI doesn't work well with this translation tool. Sorry for that.
Good so what is the current state of AI in academic publishing at least. How do we see it from the publisher side, from mdpi? We all know AI is revolutionizing the world with especially generative AI, so adopting AI is no longer an option. It's like a must for most of the medium to large sized publishers also included MDPI. So we started a couple of years ago with a team of mainly data scientists working on different AI projects that will help us with increasing the quality, the speed and also the quality of service we give to our authors.
We have divided the AI use cases in two categories. On the left for scientists, researchers, and also on the right for publishers as MDPI. So researchers use AI to usually Polish English language improve the tone of the manuscript. If they're not English native speakers, they use AI to generate code, generate a different content faster. Summarized research literature.
Brainstorm about research topics and ideas and so on and so forth. On the other side, the publishers, we use AI to find peer reviewers. We use it to check for ethical issues and breaches in submissions we receive. We use AI to assist peer reviewers during the review process. Also to find potential authors, promote their journals, write presentations, emails, and also provide customer support.
Chatbots so those are some of the use cases that we see as interesting. When I came to the stage, however, there is also an always a dark side of mostly every new technology, and I would like to highlight some of them here. First of all, there is or there was a big authorship attribution debate when we talk about AI. If you search on Google Scholar for articles once ChatGPT went out in November 2022, if I recall it correctly, you'll find a couple of papers where one of the co-authors was ChatGPT or any other chatbots.
And back in time. So this is a debate that is hopefully closed now. But it was present at the beginning. It then makes plagiarism easier and harder to detect, which is a big issue for us as publishers. It brings mistakes or inaccuracies into papers. It allows creating fake papers, but also more important, fake peer review reports. And last but not least, it Gen AI currently has the inability to draw reliable scientific conclusions.
So that's something that is very important for scientists whenever they use AI to write their papers. So MDPI and most of the large publishers nowadays they follow code standards. Whenever we talk about the usage of AI in academic publishing. So authors are not allowed to list those Gen AI chatbots as co-authors, they can use AI without any disclosure. If we talk about English language polishing about refinement of manuscript content.
However, they need to disclose whenever they use AI for any other purposes, which means they used AI to generate maybe a data set to help running an experiments, and so on and so forth. So that's something that we as a publisher do require authors to dissolve whenever they use AI within their manuscripts. On the other side, we have the peer reviewers and the review process.
So AI is strictly forbidden, at least at MDPI in the peer review process itself. So it comes mainly from the fact that the authorship or the copyright of the paper belongs to the author before the paper gets published, which means that the peer reviewers are not allowed to upload the manuscript on any of those third party AI chatbots. So I will hand it back to Henrik now to describe you. What is the preview puzzle in academic publishing.
OK, so as the peer review is an integral part of our business of processing and publishing manuscripts. There are several issues going on regarding the peer review, and we could discuss about it for hours. In particular, I'd like to focus on this issue that is summarized in this graph over here. So the upper line, the darker blue one represents the number of researchers over the years. As expected, it's increasing.
And the light blue line represents the number of papers published during those same years. And as you can see, the number of papers published is increasing faster than the number of researchers in actually, it's increasing nearly twice as fast. Now if we translate this into peer review reports needed to publish these papers. We get this graph here. We are not only taking into account the peer review reports needed for covering all the published papers, but also those papers that are rejected after peer review because those also generated peer review reports of their own.
And this right over here represents that, assuming that all the researchers are engaged in this peer review process, which is not true, we see that the number of peer review reports yearly that these researchers should do in order to cover and generate all these peer reviewed reports, is also increasing over time. Just to summarize the number of reviews per researcher has increased from 2.6 to 3.3, nearly one third during this time period.
So the issue here is how can we use these kind of technologies in order to alleviate this extra strain that we are putting into the research community. So this is what we are going to discuss now. Thanks good. So now that we saw what is the peer review puzzle in academic publishing, I will try to describe you as much as possible how we do try to crack that puzzle at MDPI.
So we do believe that if not solve the puzzle, at least help. It's solving it through tech innovation efforts. So I would like to present you quickly. What is the purpose and the mission of the tech innovation team within our company, but also then go more into details of some of the projects and products we have developed. So first of all, our main mission is to increase the quality and the speed of services we do provide to researchers.
So this is the North Star is the main goal. And we try to do this through leveraging AI and data driven mindset. Also doing two additional things. One is to train our internal staff, our assistant editors, by empowering their efficiency and quality work, but also empower researchers, which means authors, editorial board members and reviewers to be more efficient.
Again, using the tools we are developing there on the right side of the screen, you have, it's probably too small if you're in the back, but just very briefly, it's something we call the innovation project life cycle, and it's divided in four categories. We first try to identify the problem external impacting problems or internal processes improvement. We try to stay tech savvy.
We then need to identify and prioritize different ideas and projects. Nowadays, when people talk about AI, they do have a lot of ideas, unfortunately, and we cannot fulfill all of them. And finally, we go to the implementation phase where we run a POC. We do try to first test the tools internally with a smaller group of people through pilots, and then go and scale up and go into production.
So something we'll mention together, me and Henrik, during this presentation, we always pay attention on the human in the loop approach, which means that any of the tools that we present to you next are not meant to do any final decisions. So it's always raising flags, proposing the next steps. But we have people, humans inside a company that will then have to judge how truth or how good that AI proposal was made.
This is the editorial process at MDPI. It's not diverse from many of the other publishers, so we go from the pre-submission stage until the submission through all those technical pre-checks, peer review, internal decision, and then if the paper gets accepted, we move to the production and finally to publish the paper. So if not every but most of the tools we have developed using AI, they are stick to the tightly stick to this process here.
So we start with the tool that we call journal finder that allows our customers to authors to select the right journal they can publish their paper in. So it's something that is freely available on apple.com website, where our future customers can upload the title and abstract of their manuscript, and then they get a list of proposed journals that they can submit into it. Mdpi has over 450 journals, so it's an interesting tool for them to select the right one that matches on one side the article, but also on the other side, the criteria regarding the impact factor, the price of the gold open access, et cetera within the same technology, we have something we call in scope checker.
So in scope checker is run on our site on every single submission to make sure that the scope of the submission matches what the scope of the journal. So that way also we ensure not to publish out of scope papers or journal out of scope papers. Then we have the finder. Reviewer finder project that is currently in a late POC phase, so it allows our internal editors to match the best experts in the field to be invited to do peer review for a given manuscript for us.
So again, I will propose a list of people. Then a human will have to decide who to invite after some more detailed checks there. If you ask me, it's probably one of our most important products at the moment. Maybe Henrik will disagree. It's called ethicality. So ethicality.
It's used to check for a list of potential data breaches that we may have within a given manuscript. So it raises flags, and our editors then need to go through solve them internally if possible. If it's not solvable internally, they will go out and approach authors directly or eventually at the end of the journey, the editor in chief. If this needs to be scaled up, we have another project that is called researcher profiles, where we do build our internal researcher profile database.
That is quite also important for that ethicality tool XML conversion I assisted. We are in a very early stage of implementation in there, so there is nothing yet tangible. And finally, last but not least, we are also working on developing customer support chatbots. So we are running now through a proof of concept for an internal chatbot. And we will move forward hopefully by the end of this year with an external chatbot that will be used by our customers to go through all sorts of FAQs or eventually ask customer support.
So in the next couple of slides, Henrik will introduce you to the reviewer finder more into details what this product does, and then I will continue with the totality so that you can have a bit more of insights. How those two products can crack that peer review puzzle that was introduced by Henrik before. OK, so now focusing on the reviewer finder. Here you have a brief diagram about how it works.
Basically we start from the submitted paper here. And we are able to there are several ways to do it. How to extract features. Out of these papers you can check the citation analysis network or in our case, you can extract information out of title and abstract using a natural language processing and tool that we have. Then we can cross-check this information in our database, which is called cited where.
Here we not only have MDPI papers, but we have information from papers from other publishers. We do a similar extract, feature extraction from these papers and using a technique called semantic search, we are able to identify those papers that are more similar to the target paper, the submitted paper. Then the next step is to extract author information out of the retrieved papers. And then we apply some business and logics in order to rank them and select the most suitable ones.
We also cross-referenced this with our internal reviewer database, just to make sure that there are no conflicts of interest that they were not recently contacted. We also check if they did the recent reviews, how good they were, and all this information is collected, processed, and the final output is a ranked list of potential reviewers for a given paper. This is the information that receives the assistant editor, what we call with which we do the final decision.
Again, as mentioned here, we do not aim to automate this process. The final decision will always be done by the assistant editor who has enough of expertise to check this list and select the most suitable reviewers for this paper. So what we aim to achieve with this tool is basically to have better targeted invitations, and this way reduce the amount of invites required to secure two or three peer review reports, and therefore reduce the strain of inviting and trying to secure these reviewers by contacting to the research community.
And of course, just to highlight again that the suggestions are always validated by the assistant editor. And now we will talk about the ethicality project. Good so I will move forward with the second project. We'll do more of an in-depth overview of how this works. So it's called ethicality. In the last years at MDPI, we have built an internal team of ethical or ethics experts working mainly on research integrity.
So this team of probably like 10 plus people now there are in charge of dealing with complaints on published papers, which is some sort of a curing mode. Once a paper gets published, they need to jump into it, spend literally months of work and understand if this paper needs to be retracted or not, depending on how critical or severe the breach the ethical breach is on the paper. At one point, we decided to move to the curing mode, which means that we do try to prevent as much as possible.
Of course, it would be nice, but I don't think we will be able to reach 100% But to prevent as much as possible of those papers going through the peer review process and being published with potential ethical issues. So doing this by hand, it's really impossible. Or we need to hire another 100 or 200 people to do it. So we decided to work on a tool that can help on raising flags and then spotting different content elements of the paper, where experts are not depending on how severe disease they can go through, and then not spend their whole time on reading the full paper, really going into details everywhere, but concentrate on those flagged content elements.
So what ethicality is meant to do at the end. It's not. We're not there yet. Unfortunately it's to spot for plagiarism paper Mills, fake papers, citation misconduct, image manipulation, and also authorship manipulation practices that we find in academic publishing. So here is what ethicality can do. For now, we start on the bottom of this page with a new submission manuscript coming into the system.
We go through an extraction phase where we do extract the key elements from the manuscript like Title abstract authors references the body of the paper with all the sections, and then we run given specific contents to different kind of smaller tool sets to detect all kinds of ethical breaches. So first of all, on the left, we do. Citation misconduct analysis, which means we try to spot self-citation percentage.
We try to spot out of scope, but also out of context citations. Then we try to check the authorship, which means we try to analyze and detect purchase authorship. Also fictitious authors. But last but not least, like citation cartels that may erase on author and citation manipulation. We then go through plagiarism detection by using a third party tool. I mean, we use iThenticate as many of the other publishers image manipulation, again, using a third party provider to detect duplicates, cross publisher duplicates, but also to detect in paper image manipulations in there.
We then go to the AI text detection. So we don't only try to isolate or to detect if there is AI used within the manuscript, because you saw it before. This is allowed and legit, but we try to isolate English polishing AI usage versus generated text detection. So that's quite tricky technically. And I think we have a tool that does it pretty well for now.
And finally, we also try to detect duplicate submission cross journals, but also to detect similar submissions across all journals in order to avoid salami slicing and other practices that are not allowed in the publishing business. Mdpi is also part of the SDM integrity hub since last year, so we do also benefit from all their checks they do on the other side. Cross publisher duplicate submissions.
We also try to use their platform to detect bad actors in their bad actors watchlist, do all sorts of DUI analysis, checking, Retraction Watch, and so on and so forth. Again, I would like to stress this for the fifth time in this presentation. Human in the loop is also something we try to stick to. It's not because the tool says something, and there is a proof here that it doesn't really work well, that we take decision purely based on that.
So those are flag raised. We go into details, we try to analyze, and then our editors are trained to take specific decisions on those parts there. I will let Henrik present you the last couple of slides. Just some initiatives from MDPI. So to end, we just wanted to show you some of other MDPI initiatives that we are working with right now.
The most important one for us is jumps. And in MDPI, most all the paper processing from the paper submission to its publication is done using an internal tool. It's called submission system. Susi in short, and this is a commercial platform that is essentially used for third party users. That encompasses all the processes, again from submission to publication.
It's a modular tool that has different functionalities like the submission system, production service, like production until the final version of the paper is ready website hosting, building infrastructure and editorial guidance. So if anyone is interested in this, they can just select the modules that are needed in their case. And we also wanted to highlight that all these different AI tools that we have discussed in this presentation will be also available in jams.
So we think this will be very useful for potential users. And also to finish, just briefly mentioned that we have several other initiatives. The one that I think is most interesting here is that as we have discussed, this is our scholarly literature database and this is the database that our reviewer finder tool queries in order to find potential reviewers to our submitted papers.
But we have also a preprint server, side profiles, which is an author database proceedings series MDPI books SCI forum, which is a website to organize conferences and other forms of exchange and collaboration, and also professional languages and editing services. So that's all from our side. Thank you very much for your attention. And now I think we have five minutes for a Q&A session. If you have any questions for us.
If you do, we would appreciate if you can use the mic in the middle because this is a recorded session. So in order for the questions to get record. We would appreciate if you go there, state your name and affiliation and ask the question. Are there questions.
I'm Erin flock, I'm coming from Paris. And regarding your willingness to empower the reviewers, I would like to know if you ever considered to pay them. Because we are suffering from a lot of journalists telling us, sorry, we haven't found the reviewers. Maybe reviews are always the same. I mean, it's one thing to identify them, it's another thing to push them to review in time.
And so waiting for three months or four months to get the answer from the journal. Because the editor. I mean, that could be an help to pay them. Not a big amount, but at least to show some consideration to them. So that's my question. Yeah thank you very much. That's a very interesting question.
And actually it's a tricky one also because we need to find ways to reward reviewers for their free effort because this is at the beginning, it. There's not enough recognition given to reviewers either monetarily or any other way. Now the problem if we start discussing about paying reviewers, we need to also discuss about potential conflict of interest. Because maybe you by paying them, you are incentivizing them to give you good review reports.
And we would need also to avoid this. Now regarding what we do at MDPI, we do not pay reviewers, but we do provide vouchers discounts that they can use later if they wish to submit to MDPI. We are also investigating other ways of rewarding the reviewers. One thing that one way that we are it's on the table right now, and I think it's interesting, is that we can give this voucher not only to the reviewer, but to the institution in which this reviewer works.
And this institution can get a pool of vouchers that anyone from that institution that would like to publish with MDPI could eventually use. Of course, there are many other ways in which we could try to reward reviewers, and this is a very interesting discussion to be had. Any other question related to the presentation to mdpi? The publishing topic.
OK, so let's start. Thank you very much for your attention and hope this was interesting.
Form Title Download
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode