Cadmore media player playing video Leveraging Bibliometric Data for Strategic Growth: Publisher Use Cases
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
Leveraging Bibliometric Data for Strategic Growth: Publisher Use Cases
Description:
Leveraging Bibliometric Data for Strategic Growth: Publisher Use Cases
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/131aab31-c98c-406f-abdb-3ec45de42587/videoscrubberimages/Scrubber_1.jpg
Duration:
T00H28M33S
Embed URL:
https://stream.cadmore.media/player/131aab31-c98c-406f-abdb-3ec45de42587
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/131aab31-c98c-406f-abdb-3ec45de42587/industry_breakout__knowledgeworks_2024-05-29 (1080p).mp4?sv=2019-02-02&sr=c&sig=GoCzRzl4amCF1BrQSfotPF0CFiVBFFSivR8jVYmHqn8%3D&st=2025-04-29T19%3A38%3A01Z&se=2025-04-29T21%3A43%3A01Z&sp=r
Upload Date:
2024-12-03T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
All right. We are going to go ahead and get started. Welcome and Thank you for joining us this afternoon. My name is Annette Hager. I'm a senior consultant with AGL consulting and I'm hosting the session today. I'm very honored to have two colleagues with me today who are going to speak to you in a few minutes.
Vaishali Danley is here from proceedings of the IEEE, and Chris Reed is with us from triple A's or the American Association for the Advancement of Science. Before I turn it over to Vaishali and Chris, I want to spend just a couple of minutes going over the objectives of the session and a little bit of background. In this session, we're going to talk about practical applications of citation data in shaping editorial strategies.
Now, we're not planning to get into the different data or analyses that are available that would take all afternoon. We still wouldn't get to the bottom of it. We're also not going to discuss the pros and cons of those. Suffice it to say, there's a lot of data that's available. Translating all that data into actionable strategies can be really challenging, and that's where we want to focus our attention on harnessing citation data to inform effective editorial strategies with the ultimate goal of enhancing the quality and quantity of submissions.
And that's really important. Just a little bit of background about that. So we're all starting at this from a similar perspective. You all scholarly publishing is undergoing a prolonged period of transformation. On this slide, I've listed just a few of the most influential factors that are shaping the market today. It's a complex market to simplify very much in this environment.
Journals or portfolios that publish more content from a diverse author pool are much more likely to succeed than journals or portfolios that publish less. Now, please hear me clearly. I'm not advocating for just publishing anything and everything. Absolutely not. The idea is to find and publish high quality content that's going to enhance and move forward the reputation and influence of your journal portfolio organization to your community.
It's very easy when you're thinking about citation data to get lost in the sea of data that's available or spend a lot of time studying something that's really fascinating. And in the end, you realize it really doesn't hold much value for you. In order to help avoid those two eventualities, it's important to articulate what you want to know before you start in on a citation or bibliometric analysis.
So on this slide, I've listed some of the questions that journals and societies can use to frame their thinking around strategy, development and bibliometric analysis can be used to inform any of these questions. So having given that little bit of background, I'm going to turn it over to Chris and Vaishali, and they're going to tell you how their organizations have actually done this, how they've used citation and bibliometric analysis to inform strategy development.
There is going to be time for questions and answers at our questions and discussion afterwards. So jot down your questions or thoughts as Chris and Vaishali are speaking and we will talk in a few minutes. Chris is going to talk to you first from triple A's. You, sir. Thank you very much.
So, Yes. So my name is Chris Reed. I'm the director of my director of product and publishing development at triple science and alongside Scott Turner, my colleague who's just here, our lead, our science partner journal program. And so today, I want to speak to you, obviously, about biometrics. But before I start, I just want to provide a little bit of background about the program and our work with kagl as well.
So the program started in 2017. It's a relatively new program. I'm not going to read through all the words here, but essentially its purpose is to provide a nonprofit solution to work with other societies, other institutions for a high quality gold open access program. And so at the currently we have about 16 to 17 titles are currently signing a couple of contracts. So it's growing very quickly and each of the titles in the program editorially independent, and they're responsible for the content published in that title.
And so the question obviously we're here to address today is why bibliometric analysis. Why is this important. And the background here for the science partner journal program and again, working with kagl is looking at the growth. So in order I look at bibliometric analysis as a foundation piece in order to achieve the aims of your journals, the aims of your program, you really need to have that strong foundation to allow you to do so.
And so the reasons why do this and you'll hear probably a lot about this over our talk is looking at that growth in high quality submissions, making sure that the journal content is representative of the aims and scopes we've seen over a couple of years. Journals get into trouble where that's happened, where it's not been representative, and also looking at the editorial board and are they representative of both the community they're trying to serve of the journal and actually of wider diversity in the world as well.
Now, Annette mentioned this as well. I think one of the challenges and great enjoyments I think of bibliometric analysis is how much there is. There are so many things you can do. There's hours and hours and hours and hours of work, and that's something I'm going to come back to in one of my final slides. So how do you break that down. How do you approach it in a kind of more in a way, you can actually eat the elephant, so to speak.
And so what we do with each title, so whether that's a launch title or whether that's a journal that's recently joined us, is looking at benchmarking. So benchmark the titles to look at their areas of strength or weakness, then looking at established titles. So this is really important in terms of competitors. This is really important for launch titles. We've got a brand new journal. You have no data, so where do you look to get that data.
Where do you look at the strengths and weaknesses. Where is the top side of papers coming from, for example. And then you have the opportunity to really go into depth. And so that's looking at that granular citation data available from the more established competitor titles to identify things like top cited authors, all the classics, top cited institutions and labs. And so what I just want to address on this slide really is looking at an application.
So again, Annette spoke to this. I think it's very easy to have this huge amount of data, these beautifully produced reports, color coded, saying your most citations are from x institution or y institution. So what do you actually do about that. And I think that's really important. So one example here I think is on the left hand side for you is looking at, OK, this journal I keep an anonymous for the sake of today therefore with the Chinese institution is there an editorial board represent the constituents it should be does it represent the highly cited institutions, the individuals in those situations.
And kind of the answer is no, not really. Actually, you're looking at this here. China is well represented, as you'd expect with a Chinese institution, but other areas of the world are not. Now, I think it's very easy to say to journals what you need to do is go and expand your editorial board and then leave it at that. That's not actually a very helpful message to tell to people.
Of course. And I think this is where the application of bibliometric data is so important to provide targeted approaches to people and to say, expand your editorial board. By the way, here is a number of individuals at the Helmholtz association who are highly cited in the area. When you're next at a conference, seek them out, bring them on, do that editorial work to really bring them on and providing them with the data and the evidence to do that.
I think the other part of this is blind spots is to address everybody. We all have blind spots. Every editorial has blind spots. Editorial boards traditionally have been recruited from people who sound and look like other people on those boards. But this data can really help bring that diversity up to make sure those editorial boards are not just more representative of their communities at large, but of the world at large as well.
And so again, it's this foundation piece of data. Now, something to come back to just for my final slide is the work involved. And I think this is something that's very both interesting and challenge. So challenging. Rather so you can really go down a rabbit hole. There is so much work and as Annette said, having the right goals in place, the right questions, it really helps with that.
But if you are a small publisher like us or a small publisher, I'm sure lots of you represent in the room today, you're very busy doing operational work. How can you leverage new tools to help you with this. And of course, no talk of SSP in this year would be complete without mentioning AI. And this will not be an exception to that. So how can we leverage AI tools like ChatGPT to help us with this.
And I will say, the clear word here is experimentation very much in the early days of this, trying to work out what to do. And so what I would do quickly is look through these. So the first is looking at ChatGPT VS Code interpreter. It's a great tool for reformatting data. So a good example of this would be downloading huge swathes of data from websites, collecting it all together, beating the maximum download limit all of that thing and then saying brilliant, I've assembled my data set.
What do I do now. ChatGPT is fantastic at resourcing that, reinterpreting it and I'll come into an example later. The next is visualizations, and I think this is an underappreciated piece. Actually what we're finding more and more is the experiment. It can produce great graphs, it can produce really interesting ways of demonstrating the data in a more consumable fashion.
But I think, again, it goes back to this point that I think we say a lot today is it's not just about having the data, but it's communicating the data and making concrete actions out of it. It's making someone understand and reason why. And the last one, which is extremely nascent. But I think quite interesting is this area of external tools. So this is looking at something like a burgeoning services like system, pro or illicit or insightful.
And what these do is they take huge corpuses of data and they try and map connections between those in a way no human can do it already. What we're finding with through various focus groups, through various surveys and anecdotal data is that researchers are taking advantage of this to find new areas of research, new areas of discovery, where they think traditional links may not exist. And actually, I think the same can be done for journals.
Frankly, we're able to find whether that's identifying a new journal or a new topic or a new special issue within a journal, actually a better way of serving a community that otherwise you would not be able to do. So to wrap up, I do. I think this is very small. I apologize for that. But the kind of two very practical examples, I wanted to look at a really easy one and a little bit more medium easy.
So the first one is and I think this is where again, AI is kind of underused, actually, frankly, is this kind of Copilot approach. So what we'll do is here in the top left is looking at editorial board and saying, OK, what are the constituents on this editorial board. Where are they from. And the traditional way of doing that is to go to the editorial board page, count them.
It's a manual job. It takes you 25 minutes, half an hour, maybe a bit longer. ChatGPT can do this in about three minutes. Two minutes. Yeah, much, much faster. And it allows that shortcut to do it again. It's just finding ways of being able to do things with lower sets of resources that you otherwise would not be able to do.
The next one, I think is a little more advanced. It's saying, OK, we have this huge amount of data from web of science, but let's try and classify this a little bit. How can we look at something. Can we classify whether an article is basic science. Can we classify it as clinical subset of GP, a number of rules to work from, interpret it and say in this example, they wanted to know which was cited more.
And so we're able to break that up again, a task that a human can do. But it takes a lot of time. This took frankly five minutes. And the other part of that is really purposeful. Once you've done it, once you can do it again. So I think there's a lot of opportunity. And when we get to the discussion stage, I'd love to hear any questions, any ideas, any comments, has I will probably wasn't going to show this slide, but it's just for fun slides.
So I will there are limits. So this is where we ask ChatGPT to do map the editorial board and a nice visualization. It's not there. It's very fun. It's mad, utterly mad. But yeah, it is getting better. But a note of caution there. And with that, I'll hand over.
I'm glad that you talked about AI. So I don't have to. So Hello, everybody. What I will be talking about today is just giving you a little bit of background about the publication that I deal with, which is the proceedings of the IEEE, and then give you a few reasons of why we looked at the bibliometric data and the citation analysis.
I will also touch upon what we looked at specifically when looking at the journals content versus what we looked at when we looked at competitor journal content because it was a little different. And then talk about the insights gained. For those of you who are not familiar, the IEEE, which is the Institute for electrical and Electronics Engineers, is a leading authority and a trusted voice in a wide variety of areas, ranging from electrical engineering, telecom, computing, aerospace and more.
And it is a not for profit with a simple mission, which is advancing technology for humanity. It also happens to be one of the largest technical professional organizations with over 450,000 members globally. And where does the proceedings come into the picture. It is not a conference proceeding, which is what I wanted to make very clear. It is a publication, it is a journal, and it has the name proceedings in the title because it has a very rich and long history and back, back, back in the day, more than 100 years ago, things that societies published were called the proceedings, and that's where the name comes from.
So it originated in the 1909 as the proceedings of the wireless Institute and now this year we are publishing our 112th issue and it's a journal again. What is the mission of the proceedings of the ieee? The mission really is to offer reviews of broad significance and long range interest, and it also aims to serve as a tutorial and technical information source with an emphasis on applications driven technologies.
And this is important because it also guides the kind of citation analysis that we have done in partnership with and also the kind we do in general for the journal. The way the proceedings also packages its content is interesting because we have special issues, which is mostly a collection of solicited articles on a particular area that are led by distinguished guest editors and with contributions from leading authors around the world.
We also do regular paper issues which are mostly unsolicited papers. So the journal is open for submissions and we process them the same as we would other content. And we do special sections which are something we've reintroduced in the recent years, which are smaller, a little smaller package, special issues, so fewer articles than a special issue, but a similar concept there.
And the papers in the journal are primarily in two categories tutorial papers, like I mentioned, because it wants to be a tutorial source and also review and survey papers. So why bibliometric data and citation analysis. If you missed it earlier, I said that we are a society of engineers and when you talk to engineers, they always say, show me the data. Why should we do something.
And they always want data to lead their decision making. So definitely that is one of the main if not main, but one of the motivating reasons to look at data. And also what do we not know because we never done. If you never do an extensive analysis, there could be things that you have not considered before. It's also always good to know what you are doing well. Because we should keep doing those things and look for where there are gaps so that we can make improvements to the strategy.
More specifically, if we look at the analysis that we did with AGL, what topics and Article types have the highest impact in our field. Some of the questions that can be answered include that and also in what area should our journal consider expanding, looking at low hanging fruit that we could easily implement in our strategy quickly. So what did we look at when it comes to the journals content. We looked at the obvious things like the market share and metrics trends over time.
We also looked at the editorial board, so we did a bit of a thorough editorial board analysis looking at board geographic distribution versus our submission and publication trends. And when it came to the citation analysis, we looked at the obvious things like top cited articles, top authors, top institutions, but also looked at least cited articles and top cited issues and sections, which is something specific to the way the proceedings are structured.
We came up with a list of competitor journals and looked at their metrics and of course, also their citation analysis that was led with top cited articles, authors, countries, institutions, and keywords. And we gained a lot of insights. I don't want to go over all of these necessarily in much detail. I will touch upon a few of them, but we tried to categorize them into buckets of what that helped with.
So some LED us to modify our content strategy. Some suggested that there was great stuff out there that could be source of future content for us. There was some insight into how we could better. Improve the diversity of our editorial board and more specifically, at three cases that I wanted to point out how that we implemented that into our strategy looking at solicited versus unsolicited content and least cited articles.
I think those two were insightful. Looking at the data, we realized that our unsolicited content sometimes performs much better than our solicited content. And this is not a big surprise to us because of the way the content is structured. And there's value for both. So what could we do. We could perhaps increase the percentage of the unsolicited content coming into the journal and also look at the solicited content and see if it's being too sliced and diced into many, many niche articles.
When we look at special issue contents and special section contents, maybe instead of doing a special issue on a topic, we could lead with a special section, which is fewer articles but more meatier articles that would lead to higher impact. Thinking of high performing issues. This is also, again, more specific to journals that have that kind of structure where they do issues, special issues.
But don't forget to look at content that you published in the past. That's one insight that's gained because you can always see that research cycles where topics come back and have traction again because of certain movements and discoveries. And just for an instance, space related missions, we did something seven or eight years ago on that topic and it seems to be gaining traction again.
So that could be a good topic for us to revisit and reconsider for future content. And another topic. High performing countries and journals and competitors definitely could be sources of more top articles. So looking at those high performing authors and trying to see how we can tap those for future articles for the proceedings is another insight that we gained. And I would say that citation analysis is something that is not a do it once and done kind of thing.
It's something that should be integrated into the strategy for the long run. And definitely looking back and measuring how successful your analysis was in bringing content to the journal in the future is key. And with that, I'm done. Thank you. Kristin Vaishali.
That was wonderful. Hopefully it gave you some good ideas about how you can use citation analysis in your organization. We do have a few minutes for questions or comments. Yes, Thank you. I'm wondering how you ensure good data quality. Both with methods that use.
Don't say that. Nope OK. Yeah, it's a very good question. So I think so in terms of the data we use, we use high quality data to start with, so it's not introducing brand new. So the gold standard, well, the semi gold standard is webofscience. I think the challenge websites increasingly refining some data out of date.
There And actually, that's an issue for another day. But that to answer your question very directly, that's the prime source because it is a gold standard. And so building off that, I think what would be very interesting is if we were to be building in other data sources. And so we Altmetric Explorer is a good example of that and be able to combine data sets there to provide brand new insights.
Because I think obviously we've spoken a lot today about citation analysis, but this isn't just about citation analysis, this is bibliometric analysis as well and actually does usage of mentions. And I know altmetrics has all kinds of flaws in it, but it can be used as a proxy for attention as well. So sorry, there's a longer answer, but Yeah. Yes excellent. And I'll just add, just.
Keeping in mind every data source has its. Shall we say, quirks. Keeping those in mind and contextualizing your analysis based on your data source is helpful. Also, just to piggyback off of Chris. What other questions. There's one in the back. Thank you very much. Wonderful two questions, actually.
Did you have any copyright issues. You rejection. No Well, so. So we don't. So we have a professional subscription to ChatGPT. So it doesn't go back into their training model. So make sure that that's Yeah, we don't want to. It's a good question because we don't want to fill ChatGPT with Weber science data and tread any copyright restrictions there.
So we make sure it's not it's not training on it. Yeah quick question for you in terms say is this a dashboard, is this a data lake. And then you expect questions that you want from this or other lenses. So there are various use cases for this data. So how will you actually use the entire data dashboard that. Do it for free environment.
No so this song. I don't know. So for this doesn't want me to answer this question, so. Now it's too loud. OK, sorry. So for this particular case, this was something that was run custom for the journal. So there wasn't platform.
The IEEE does have some platforms where we can extract data and analytics that we can turn every time you want to. Because you said it is also I think once you integrate it into strategy, you would have to think of ways to get it more contiguous continuously, not do a custom approach. Yeah OK. Excellent.
Thank you. Anything else. Any other questions. Go ahead. Following my question was, do you use any other. Citations to. And my other question is for topical.
He worked over there and other. I will answer. I will answer the part about sorry about keywords. Yeah so we used keywords to look at topical areas and that also meant that we had to have good data to begin with the keywords. And that was also a learning lesson for us in the way keywords are put into the system and which ones we are looking at, because it has to be comparable.
It has to be. It can't be apples and oranges when we look at competitor data and ours. So yeah, we use keywords, but they were comparable to that degree. Yeah Yeah I can. So I'll take the risk. And I think. I think this is working.
OK here we go. Your question was about using other sets of data. And so it's a really good question. And maybe I can answer that with an example. So one of the things that we've been looking at is saying, OK, usage data, can we usage data indicate something prior citation citation data is, of course delayed. And so an example there is across the science partner general program, we're seeing a lot of usage coming from India in particular, but actually we don't see a lot of that in citation data necessarily, but it's an audience we want to serve if there's a lot of interest in readership.
And so how do we take that data and interpret it into action. So one of the things we've done there is we actually worked with ResearchGate to target potential editorial board members in India to attract them to the journal, because I think speaking, as I was before about diversity and approaching that in editorial boards, citation data is helpful, but it can exacerbate existing networks. And so trying to find new networks, new ways of approaching people with an eye on the future, it's not just about your citations two years ago, three years ago, but also the future as well.
So that's something we're experimenting more with. We have time for a couple more questions, if anybody has any. When you mentioned one of the insights that you took away was the difference between solicited and unsolicited content. That almost seemed like the reverse of what I would expect that be solicited.
Content would be more highly cited, and that's often a strategy that's recommended. I'm just curious what you took away from that and why you think. Yeah so frankly, it wasn't a complete surprise to us. If you believe it or not. But we have to cover all areas of interest to the Tripoli. And it's a broad scope. Journal of course, we are doing the hot topics that people are interested in, but it does provide some insight into how we can better structure the special issue content because we are covering state of the art in a specific area.
But maybe we are going to narrow with some of the articles and maybe that's where that special section idea is. Great We can get the coverage on a specific topic, but we don't have to necessarily do the whole issue. And I think Yeah, that can guide your strategy a little bit more in balance of content too, of like, what should you do more of. But it doesn't mean that we're going to revolutionize it and just do unsolicited content because that's not the mission and the scope of the journal.
We want to stay true to who we are, but we want to do something slightly differently. So yeah. Excellent OK. Well, that takes us just about to the end of the session. So Chris and Vaishali, Thank you very much. And Yes, definitely.
If I may, can I volunteer you to answer additional questions afterwards. OK, excellent. And I'll be up here or I'll be out back, I guess, also. So feel free to ask more questions. And Thanks again for joining us this afternoon.
Segment:0 .
All right. We are going to go ahead and get started. Welcome and Thank you for joining us this afternoon. My name is Annette Hager. I'm a senior consultant with AGL consulting and I'm hosting the session today. I'm very honored to have two colleagues with me today who are going to speak to you in a few minutes.
Vaishali Danley is here from proceedings of the IEEE, and Chris Reed is with us from triple A's or the American Association for the Advancement of Science. Before I turn it over to Vaishali and Chris, I want to spend just a couple of minutes going over the objectives of the session and a little bit of background. In this session, we're going to talk about practical applications of citation data in shaping editorial strategies.
Now, we're not planning to get into the different data or analyses that are available that would take all afternoon. We still wouldn't get to the bottom of it. We're also not going to discuss the pros and cons of those. Suffice it to say, there's a lot of data that's available. Translating all that data into actionable strategies can be really challenging, and that's where we want to focus our attention on harnessing citation data to inform effective editorial strategies with the ultimate goal of enhancing the quality and quantity of submissions.
And that's really important. Just a little bit of background about that. So we're all starting at this from a similar perspective. You all scholarly publishing is undergoing a prolonged period of transformation. On this slide, I've listed just a few of the most influential factors that are shaping the market today. It's a complex market to simplify very much in this environment.
Journals or portfolios that publish more content from a diverse author pool are much more likely to succeed than journals or portfolios that publish less. Now, please hear me clearly. I'm not advocating for just publishing anything and everything. Absolutely not. The idea is to find and publish high quality content that's going to enhance and move forward the reputation and influence of your journal portfolio organization to your community.
It's very easy when you're thinking about citation data to get lost in the sea of data that's available or spend a lot of time studying something that's really fascinating. And in the end, you realize it really doesn't hold much value for you. In order to help avoid those two eventualities, it's important to articulate what you want to know before you start in on a citation or bibliometric analysis.
So on this slide, I've listed some of the questions that journals and societies can use to frame their thinking around strategy, development and bibliometric analysis can be used to inform any of these questions. So having given that little bit of background, I'm going to turn it over to Chris and Vaishali, and they're going to tell you how their organizations have actually done this, how they've used citation and bibliometric analysis to inform strategy development.
There is going to be time for questions and answers at our questions and discussion afterwards. So jot down your questions or thoughts as Chris and Vaishali are speaking and we will talk in a few minutes. Chris is going to talk to you first from triple A's. You, sir. Thank you very much.
So, Yes. So my name is Chris Reed. I'm the director of my director of product and publishing development at triple science and alongside Scott Turner, my colleague who's just here, our lead, our science partner journal program. And so today, I want to speak to you, obviously, about biometrics. But before I start, I just want to provide a little bit of background about the program and our work with kagl as well.
So the program started in 2017. It's a relatively new program. I'm not going to read through all the words here, but essentially its purpose is to provide a nonprofit solution to work with other societies, other institutions for a high quality gold open access program. And so at the currently we have about 16 to 17 titles are currently signing a couple of contracts. So it's growing very quickly and each of the titles in the program editorially independent, and they're responsible for the content published in that title.
And so the question obviously we're here to address today is why bibliometric analysis. Why is this important. And the background here for the science partner journal program and again, working with kagl is looking at the growth. So in order I look at bibliometric analysis as a foundation piece in order to achieve the aims of your journals, the aims of your program, you really need to have that strong foundation to allow you to do so.
And so the reasons why do this and you'll hear probably a lot about this over our talk is looking at that growth in high quality submissions, making sure that the journal content is representative of the aims and scopes we've seen over a couple of years. Journals get into trouble where that's happened, where it's not been representative, and also looking at the editorial board and are they representative of both the community they're trying to serve of the journal and actually of wider diversity in the world as well.
Now, Annette mentioned this as well. I think one of the challenges and great enjoyments I think of bibliometric analysis is how much there is. There are so many things you can do. There's hours and hours and hours and hours of work, and that's something I'm going to come back to in one of my final slides. So how do you break that down. How do you approach it in a kind of more in a way, you can actually eat the elephant, so to speak.
And so what we do with each title, so whether that's a launch title or whether that's a journal that's recently joined us, is looking at benchmarking. So benchmark the titles to look at their areas of strength or weakness, then looking at established titles. So this is really important in terms of competitors. This is really important for launch titles. We've got a brand new journal. You have no data, so where do you look to get that data.
Where do you look at the strengths and weaknesses. Where is the top side of papers coming from, for example. And then you have the opportunity to really go into depth. And so that's looking at that granular citation data available from the more established competitor titles to identify things like top cited authors, all the classics, top cited institutions and labs. And so what I just want to address on this slide really is looking at an application.
So again, Annette spoke to this. I think it's very easy to have this huge amount of data, these beautifully produced reports, color coded, saying your most citations are from x institution or y institution. So what do you actually do about that. And I think that's really important. So one example here I think is on the left hand side for you is looking at, OK, this journal I keep an anonymous for the sake of today therefore with the Chinese institution is there an editorial board represent the constituents it should be does it represent the highly cited institutions, the individuals in those situations.
And kind of the answer is no, not really. Actually, you're looking at this here. China is well represented, as you'd expect with a Chinese institution, but other areas of the world are not. Now, I think it's very easy to say to journals what you need to do is go and expand your editorial board and then leave it at that. That's not actually a very helpful message to tell to people.
Of course. And I think this is where the application of bibliometric data is so important to provide targeted approaches to people and to say, expand your editorial board. By the way, here is a number of individuals at the Helmholtz association who are highly cited in the area. When you're next at a conference, seek them out, bring them on, do that editorial work to really bring them on and providing them with the data and the evidence to do that.
I think the other part of this is blind spots is to address everybody. We all have blind spots. Every editorial has blind spots. Editorial boards traditionally have been recruited from people who sound and look like other people on those boards. But this data can really help bring that diversity up to make sure those editorial boards are not just more representative of their communities at large, but of the world at large as well.
And so again, it's this foundation piece of data. Now, something to come back to just for my final slide is the work involved. And I think this is something that's very both interesting and challenge. So challenging. Rather so you can really go down a rabbit hole. There is so much work and as Annette said, having the right goals in place, the right questions, it really helps with that.
But if you are a small publisher like us or a small publisher, I'm sure lots of you represent in the room today, you're very busy doing operational work. How can you leverage new tools to help you with this. And of course, no talk of SSP in this year would be complete without mentioning AI. And this will not be an exception to that. So how can we leverage AI tools like ChatGPT to help us with this.
And I will say, the clear word here is experimentation very much in the early days of this, trying to work out what to do. And so what I would do quickly is look through these. So the first is looking at ChatGPT VS Code interpreter. It's a great tool for reformatting data. So a good example of this would be downloading huge swathes of data from websites, collecting it all together, beating the maximum download limit all of that thing and then saying brilliant, I've assembled my data set.
What do I do now. ChatGPT is fantastic at resourcing that, reinterpreting it and I'll come into an example later. The next is visualizations, and I think this is an underappreciated piece. Actually what we're finding more and more is the experiment. It can produce great graphs, it can produce really interesting ways of demonstrating the data in a more consumable fashion.
But I think, again, it goes back to this point that I think we say a lot today is it's not just about having the data, but it's communicating the data and making concrete actions out of it. It's making someone understand and reason why. And the last one, which is extremely nascent. But I think quite interesting is this area of external tools. So this is looking at something like a burgeoning services like system, pro or illicit or insightful.
And what these do is they take huge corpuses of data and they try and map connections between those in a way no human can do it already. What we're finding with through various focus groups, through various surveys and anecdotal data is that researchers are taking advantage of this to find new areas of research, new areas of discovery, where they think traditional links may not exist. And actually, I think the same can be done for journals.
Frankly, we're able to find whether that's identifying a new journal or a new topic or a new special issue within a journal, actually a better way of serving a community that otherwise you would not be able to do. So to wrap up, I do. I think this is very small. I apologize for that. But the kind of two very practical examples, I wanted to look at a really easy one and a little bit more medium easy.
So the first one is and I think this is where again, AI is kind of underused, actually, frankly, is this kind of Copilot approach. So what we'll do is here in the top left is looking at editorial board and saying, OK, what are the constituents on this editorial board. Where are they from. And the traditional way of doing that is to go to the editorial board page, count them.
It's a manual job. It takes you 25 minutes, half an hour, maybe a bit longer. ChatGPT can do this in about three minutes. Two minutes. Yeah, much, much faster. And it allows that shortcut to do it again. It's just finding ways of being able to do things with lower sets of resources that you otherwise would not be able to do.
The next one, I think is a little more advanced. It's saying, OK, we have this huge amount of data from web of science, but let's try and classify this a little bit. How can we look at something. Can we classify whether an article is basic science. Can we classify it as clinical subset of GP, a number of rules to work from, interpret it and say in this example, they wanted to know which was cited more.
And so we're able to break that up again, a task that a human can do. But it takes a lot of time. This took frankly five minutes. And the other part of that is really purposeful. Once you've done it, once you can do it again. So I think there's a lot of opportunity. And when we get to the discussion stage, I'd love to hear any questions, any ideas, any comments, has I will probably wasn't going to show this slide, but it's just for fun slides.
So I will there are limits. So this is where we ask ChatGPT to do map the editorial board and a nice visualization. It's not there. It's very fun. It's mad, utterly mad. But yeah, it is getting better. But a note of caution there. And with that, I'll hand over.
I'm glad that you talked about AI. So I don't have to. So Hello, everybody. What I will be talking about today is just giving you a little bit of background about the publication that I deal with, which is the proceedings of the IEEE, and then give you a few reasons of why we looked at the bibliometric data and the citation analysis.
I will also touch upon what we looked at specifically when looking at the journals content versus what we looked at when we looked at competitor journal content because it was a little different. And then talk about the insights gained. For those of you who are not familiar, the IEEE, which is the Institute for electrical and Electronics Engineers, is a leading authority and a trusted voice in a wide variety of areas, ranging from electrical engineering, telecom, computing, aerospace and more.
And it is a not for profit with a simple mission, which is advancing technology for humanity. It also happens to be one of the largest technical professional organizations with over 450,000 members globally. And where does the proceedings come into the picture. It is not a conference proceeding, which is what I wanted to make very clear. It is a publication, it is a journal, and it has the name proceedings in the title because it has a very rich and long history and back, back, back in the day, more than 100 years ago, things that societies published were called the proceedings, and that's where the name comes from.
So it originated in the 1909 as the proceedings of the wireless Institute and now this year we are publishing our 112th issue and it's a journal again. What is the mission of the proceedings of the ieee? The mission really is to offer reviews of broad significance and long range interest, and it also aims to serve as a tutorial and technical information source with an emphasis on applications driven technologies.
And this is important because it also guides the kind of citation analysis that we have done in partnership with and also the kind we do in general for the journal. The way the proceedings also packages its content is interesting because we have special issues, which is mostly a collection of solicited articles on a particular area that are led by distinguished guest editors and with contributions from leading authors around the world.
We also do regular paper issues which are mostly unsolicited papers. So the journal is open for submissions and we process them the same as we would other content. And we do special sections which are something we've reintroduced in the recent years, which are smaller, a little smaller package, special issues, so fewer articles than a special issue, but a similar concept there.
And the papers in the journal are primarily in two categories tutorial papers, like I mentioned, because it wants to be a tutorial source and also review and survey papers. So why bibliometric data and citation analysis. If you missed it earlier, I said that we are a society of engineers and when you talk to engineers, they always say, show me the data. Why should we do something.
And they always want data to lead their decision making. So definitely that is one of the main if not main, but one of the motivating reasons to look at data. And also what do we not know because we never done. If you never do an extensive analysis, there could be things that you have not considered before. It's also always good to know what you are doing well. Because we should keep doing those things and look for where there are gaps so that we can make improvements to the strategy.
More specifically, if we look at the analysis that we did with AGL, what topics and Article types have the highest impact in our field. Some of the questions that can be answered include that and also in what area should our journal consider expanding, looking at low hanging fruit that we could easily implement in our strategy quickly. So what did we look at when it comes to the journals content. We looked at the obvious things like the market share and metrics trends over time.
We also looked at the editorial board, so we did a bit of a thorough editorial board analysis looking at board geographic distribution versus our submission and publication trends. And when it came to the citation analysis, we looked at the obvious things like top cited articles, top authors, top institutions, but also looked at least cited articles and top cited issues and sections, which is something specific to the way the proceedings are structured.
We came up with a list of competitor journals and looked at their metrics and of course, also their citation analysis that was led with top cited articles, authors, countries, institutions, and keywords. And we gained a lot of insights. I don't want to go over all of these necessarily in much detail. I will touch upon a few of them, but we tried to categorize them into buckets of what that helped with.
So some LED us to modify our content strategy. Some suggested that there was great stuff out there that could be source of future content for us. There was some insight into how we could better. Improve the diversity of our editorial board and more specifically, at three cases that I wanted to point out how that we implemented that into our strategy looking at solicited versus unsolicited content and least cited articles.
I think those two were insightful. Looking at the data, we realized that our unsolicited content sometimes performs much better than our solicited content. And this is not a big surprise to us because of the way the content is structured. And there's value for both. So what could we do. We could perhaps increase the percentage of the unsolicited content coming into the journal and also look at the solicited content and see if it's being too sliced and diced into many, many niche articles.
When we look at special issue contents and special section contents, maybe instead of doing a special issue on a topic, we could lead with a special section, which is fewer articles but more meatier articles that would lead to higher impact. Thinking of high performing issues. This is also, again, more specific to journals that have that kind of structure where they do issues, special issues.
But don't forget to look at content that you published in the past. That's one insight that's gained because you can always see that research cycles where topics come back and have traction again because of certain movements and discoveries. And just for an instance, space related missions, we did something seven or eight years ago on that topic and it seems to be gaining traction again.
So that could be a good topic for us to revisit and reconsider for future content. And another topic. High performing countries and journals and competitors definitely could be sources of more top articles. So looking at those high performing authors and trying to see how we can tap those for future articles for the proceedings is another insight that we gained. And I would say that citation analysis is something that is not a do it once and done kind of thing.
It's something that should be integrated into the strategy for the long run. And definitely looking back and measuring how successful your analysis was in bringing content to the journal in the future is key. And with that, I'm done. Thank you. Kristin Vaishali.
That was wonderful. Hopefully it gave you some good ideas about how you can use citation analysis in your organization. We do have a few minutes for questions or comments. Yes, Thank you. I'm wondering how you ensure good data quality. Both with methods that use.
Don't say that. Nope OK. Yeah, it's a very good question. So I think so in terms of the data we use, we use high quality data to start with, so it's not introducing brand new. So the gold standard, well, the semi gold standard is webofscience. I think the challenge websites increasingly refining some data out of date.
There And actually, that's an issue for another day. But that to answer your question very directly, that's the prime source because it is a gold standard. And so building off that, I think what would be very interesting is if we were to be building in other data sources. And so we Altmetric Explorer is a good example of that and be able to combine data sets there to provide brand new insights.
Because I think obviously we've spoken a lot today about citation analysis, but this isn't just about citation analysis, this is bibliometric analysis as well and actually does usage of mentions. And I know altmetrics has all kinds of flaws in it, but it can be used as a proxy for attention as well. So sorry, there's a longer answer, but Yeah. Yes excellent. And I'll just add, just.
Keeping in mind every data source has its. Shall we say, quirks. Keeping those in mind and contextualizing your analysis based on your data source is helpful. Also, just to piggyback off of Chris. What other questions. There's one in the back. Thank you very much. Wonderful two questions, actually.
Did you have any copyright issues. You rejection. No Well, so. So we don't. So we have a professional subscription to ChatGPT. So it doesn't go back into their training model. So make sure that that's Yeah, we don't want to. It's a good question because we don't want to fill ChatGPT with Weber science data and tread any copyright restrictions there.
So we make sure it's not it's not training on it. Yeah quick question for you in terms say is this a dashboard, is this a data lake. And then you expect questions that you want from this or other lenses. So there are various use cases for this data. So how will you actually use the entire data dashboard that. Do it for free environment.
No so this song. I don't know. So for this doesn't want me to answer this question, so. Now it's too loud. OK, sorry. So for this particular case, this was something that was run custom for the journal. So there wasn't platform.
The IEEE does have some platforms where we can extract data and analytics that we can turn every time you want to. Because you said it is also I think once you integrate it into strategy, you would have to think of ways to get it more contiguous continuously, not do a custom approach. Yeah OK. Excellent.
Thank you. Anything else. Any other questions. Go ahead. Following my question was, do you use any other. Citations to. And my other question is for topical.
He worked over there and other. I will answer. I will answer the part about sorry about keywords. Yeah so we used keywords to look at topical areas and that also meant that we had to have good data to begin with the keywords. And that was also a learning lesson for us in the way keywords are put into the system and which ones we are looking at, because it has to be comparable.
It has to be. It can't be apples and oranges when we look at competitor data and ours. So yeah, we use keywords, but they were comparable to that degree. Yeah Yeah I can. So I'll take the risk. And I think. I think this is working.
OK here we go. Your question was about using other sets of data. And so it's a really good question. And maybe I can answer that with an example. So one of the things that we've been looking at is saying, OK, usage data, can we usage data indicate something prior citation citation data is, of course delayed. And so an example there is across the science partner general program, we're seeing a lot of usage coming from India in particular, but actually we don't see a lot of that in citation data necessarily, but it's an audience we want to serve if there's a lot of interest in readership.
And so how do we take that data and interpret it into action. So one of the things we've done there is we actually worked with ResearchGate to target potential editorial board members in India to attract them to the journal, because I think speaking, as I was before about diversity and approaching that in editorial boards, citation data is helpful, but it can exacerbate existing networks. And so trying to find new networks, new ways of approaching people with an eye on the future, it's not just about your citations two years ago, three years ago, but also the future as well.
So that's something we're experimenting more with. We have time for a couple more questions, if anybody has any. When you mentioned one of the insights that you took away was the difference between solicited and unsolicited content. That almost seemed like the reverse of what I would expect that be solicited.
Content would be more highly cited, and that's often a strategy that's recommended. I'm just curious what you took away from that and why you think. Yeah so frankly, it wasn't a complete surprise to us. If you believe it or not. But we have to cover all areas of interest to the Tripoli. And it's a broad scope. Journal of course, we are doing the hot topics that people are interested in, but it does provide some insight into how we can better structure the special issue content because we are covering state of the art in a specific area.
But maybe we are going to narrow with some of the articles and maybe that's where that special section idea is. Great We can get the coverage on a specific topic, but we don't have to necessarily do the whole issue. And I think Yeah, that can guide your strategy a little bit more in balance of content too, of like, what should you do more of. But it doesn't mean that we're going to revolutionize it and just do unsolicited content because that's not the mission and the scope of the journal.
We want to stay true to who we are, but we want to do something slightly differently. So yeah. Excellent OK. Well, that takes us just about to the end of the session. So Chris and Vaishali, Thank you very much. And Yes, definitely.
If I may, can I volunteer you to answer additional questions afterwards. OK, excellent. And I'll be up here or I'll be out back, I guess, also. So feel free to ask more questions. And Thanks again for joining us this afternoon.
Form Title Download
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode