Cadmore media player playing video Usage metrics - open access and book metrics-NISO Plus
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
Usage metrics - open access and book metrics-NISO Plus
Description:
Usage metrics - open access and book metrics-NISO Plus
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/0b6e5636-9f0d-4495-8fb8-3ab23906f33a/thumbnails/0b6e5636-9f0d-4495-8fb8-3ab23906f33a.png?sv=2019-02-02&sr=c&sig=KsUwEF6dzMdJ9YwsSIIE3ML9z3Tpdg%2B%2FEJd%2Fo2k92Cg%3D&st=2024-12-22T03%3A22%3A40Z&se=2024-12-22T07%3A27%3A40Z&sp=r
Duration:
T00H42M39S
Embed URL:
https://stream.cadmore.media/player/0b6e5636-9f0d-4495-8fb8-3ab23906f33a
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/0b6e5636-9f0d-4495-8fb8-3ab23906f33a/Usage metrics - open access and book metrics-NISO Plus.mp4?sv=2019-02-02&sr=c&sig=1cKjACe174GsMObXQtYaZq3R%2FPdD4gd5L6m%2Bgj%2FGKeI%3D&st=2024-12-22T03%3A22%3A42Z&se=2024-12-22T05%3A27%3A42Z&sp=r
Upload Date:
2022-08-26T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
[MUSIC PLAYING]
SPEAKER: Hello, everyone, and welcome to the session about usage metrics, or more specific, about usage data reporting in exchange for open access material and books. My name is [INAUDIBLE],, and I'm the moderator for this session. We have, in this session, two presentations and three presenters who approach the topic from different angles. In the first presentation, titled Consistent Usage Metrics in an Open Access World, we will hear about how culture, the standard of corporate practice, can help to facilitate usage reports for open access material and why this is important.
SPEAKER: The session aims to answer questions such as how usage research metrics might help demonstrate return on open access investment, whose usage is relevant for open access, and how publishers might benefit from providing item-level content usage as part of a suite of metrics. The presenters are Tasha Mellins-Cohen and Lorraine Estelle. Tasha is an independent publishing consultant and a member of the COUNTER executive committee, and Lorraine is the COUNTER project director.
SPEAKER: The second presentation has the title From Distributed Data Hovering to Upstream Data Quality, How International Data Space Frameworks and Standards May Transform Usage Metrics. It will focus on an open access equal usage reporting and discuss how an infrastructure supporting the exchange and accessibility of trusted usage data may look like. It will address questions such as how an existing infrastructure maps to an international data space model for such usage data, what active research is going on to develop policy guardrails, and how international data space approach can support and reinforce norms for the ethical use of usage data across diverse stakeholders.
SPEAKER: The presenter is Christina Drummond. Christina is the data trust program officer of Educopia Institute. This was my short introduction, and with that, we start with the first presentation, Consistent Usage Metrics in an Open Access World.
TASHA MELLINS-COHEN: Good afternoon, everybody. This is Tasha Mellins-Cohen, presenting on behalf of COUNTER and speaking about evolving COUNTER statistics for open access. So let's kick off. People tend to think about COUNTER reports as one of the pieces of information, one of the inputs, that information librarians use to evaluate subscription content. In our increasingly open world, COUNTER really needs to evolve to address new needs.
TASHA MELLINS-COHEN: So in this session, we will outline the ways in which consistent, credible, comparable usage data remains relevant in a-- sorry, remains relevant in evaluating the investment that libraries are making in open access journal and book content. We will discuss what COUNTER has been doing to facilitate open access reporting within the bounds of the existing release 5.
TASHA MELLINS-COHEN: And we will also explore some future options for enhanced and extended item-level open access reporting in the upcoming release 5.1. So why is COUNTER usage data relevant today if it's historically helped people to evaluate subscriptions? As we've already said, libraries have traditionally relied on COUNTER usage reports for measuring subscription content.
TASHA MELLINS-COHEN: That could include total usage or cost per use of the material to which they subscribe or the number of turnaways, that is, how many times their users were unable to access content to which the library does not subscribe. As an increasingly large component of scholarly output is being made open access, libraries are now becoming increasingly interested in the usage of that OA content.
TASHA MELLINS-COHEN: That might be the usage of the content published by authors affiliated with that institution or the cost per use of the OA materials that the library has funded or the wider institution has funded. There are a lot of claims made about open access providing better value for money as well as its ability to expand the reach of scholarly outputs and help authors attract more readers. By making use of COUNTER reports, libraries and publishers and funders can really start to validate the truth of those claims.
TASHA MELLINS-COHEN: Just before we move on, a quick word on cost per use. This is a really interesting concept that should be calculated consistency with a small variation of input. So to calculate cost per use for a subscription journal, you need to know the price or cost of the subscription that was paid by your institution, you need to know the number of times that people associated with your institution have made use of content covered by the subscription-- and we recommend using COUNTER's unique item requests metric for this-- and then you divide the first by the second to get your cost per use for your subscription.
TASHA MELLINS-COHEN: For open access concept, the same concept applies, but the calculation is usually run at the article level, not the subscription level. So for this, you need to know the cost paid to make the article OA. That might be an article processing charge, it might be a share of your publishing retail, if you have one. You need to know the total number of times the article has been used by anyone in the world, and that's still using the unique item requests.
TASHA MELLINS-COHEN: And then you divide the first by the second to get your cost per use for your open access spend. So please, do shout if you've got questions about that. I will be answering questions later. So we know about cost per use. How does COUNTER actually facilitate OA reporting? Back in 2016, when we were starting to develop release 5 of the COUNTER code of practice, we incorporated a mechanism for publishers to easily report on OA content, and it is written like this.
TASHA MELLINS-COHEN: The COUNTER code of practice currently states that where you've got the institution name in the report header, that for open access publishers and repositories where it is not possible to identify usage by individual institutions, the usage should be attributed to the world. And that's section 3.2.1, table 3F. And I know that's a lot to remember. When the rule was added, the focus was on fully open access publishers.
TASHA MELLINS-COHEN: The expectation we had was that those fully OA'd publishers would not try to attribute usage to an institution. So a report to the world was intended to include all global usage, whether it was attributed to an institution or not. To explain that a little, I need to go back to authentication and authorization.
TASHA MELLINS-COHEN: And if you've read my article on the subject, you know I quite enjoy this. Authentication and authorization start with a user, in our case, Sam, visiting a publisher platform. The first thing that platform will do, usually invisibly, is check to see whether Sam can be authenticated. That is, does their IP match an institution's records? Have they already logged in with Shibboleth? Have they come through GetFTR, whatever?
TASHA MELLINS-COHEN: If Sam can be authenticated as belonging to an institution, all of their usage will then be attributed to that institution. If Sam cannot be authenticated, they may still be able to use content if it's open access, if it's free to read. And the platform will still track that usage, but the usage cannot be attributed to a particular organization. Now, we know that fully open access publishers can make use databases of IP details for institutions all over the world to help them attribute usage, but this isn't necessary.
TASHA MELLINS-COHEN: Even when a publisher does use such a database of IP ranges, so much usage happens outside of institutional domains outside of campus, it cannot be attributed to an institution. Some publishers have reported up to 80% of their content use is not within an institutional agreement. So now we think about attribution. Total usage of publisher platforms is composed of non-attributed and attributed usage, though most open access platforms will only have non-attributed usage.
TASHA MELLINS-COHEN: Within that split, whether the content is paywalled or free-to-read or open access is entirely secondary. By reporting total usage, that is, by reporting to the world, open access publishers can easily provide COUNTER reports to libraries and funders and even make them openly available if they choose to do so.
TASHA MELLINS-COHEN: The latest release of the code of practice, that's release 5.0.2, which came out in summer last year, includes some optional elements that allow people to provide more granular usage reporting. One of those is attributed. So this element simply shows whether or not the content provider was able to attribute the usage to an institution. So the valid values are yes or no.
TASHA MELLINS-COHEN: So you could have a world report that says attributed, not attributed. You can also-- or publishers and report providers could also include country name and country code, which makes use of ISO 3166-1 to show global usage broken down by country. And they could also implement region name and region code, which makes use of the complementary ISO 3166-2 to show that global usage broken down by country subdivision.
TASHA MELLINS-COHEN: So for example, that would be New South Wales within Australia. These new elements mean that in addition to reporting total usage, the world reports could be broken down to show attribution types or geographic locations. And we worked with librarians during 2021 to understand what they needed to be able to do with our total usage reports, the world reports, to make them really valuable for OA reporting.
TASHA MELLINS-COHEN: And the four things that I've listed on this slide came through as priorities. The absolute top one was that a librarian might want to retrieve a report that showed their institutional usage plus all other usage so they can contextualize what their institution is doing. The second one was that of their institutional usage, they would want to compare that with all other attributed usage, so all other institutional usage and with all non-attributed usage.
TASHA MELLINS-COHEN: And the reason that they wanted that was to be able to see whether publisher claims of most usage coming from outside of library bounds, outside of institutional affiliation, was accurate. The third place request from librarians was an interest in breaking down usage by country of user, so a geographic breakdown where unknown would indicate users whose location can't be placed. And then fourthly, seeing their institutional usage within the context of their country.
TASHA MELLINS-COHEN: Now, a lot of the librarians we spoke to were very aware that IP ranges are occasionally reallocated and that with the use of VPNs and similar workarounds, geographic breakdowns may not be entirely reliable. And because of that, while country usage was of interest, regional usage really was not, particularly. We did, of course, speak to publishers as well. They weren't unanimously in favor of the new elements, but they could see the value of them.
TASHA MELLINS-COHEN: And because these are optional elements, none of the publishers that we spoke to raised any objections to their inclusion in the code of practice. So believe me, we know that everyone, publishers, COUNTER report providers, librarians, they all want COUNTER to be as stable as possible so that the whole community can feel secure that reports from different providers are consistent and credible and comparable.
TASHA MELLINS-COHEN: So just to reassure everybody, I will briefly describe the process behind a COUNTER release. It starts with a lot of internal development work undertaken by our really dedicated volunteers from the technical group, the communications group, and in this case, the open access board. These people evaluate all the requests we've received, pick up on potential areas of confusion in the code, and assess what's going on more broadly within scholarly communications to create a list of target areas for code revision.
TASHA MELLINS-COHEN: That list is then taken to the executive committee for approval before returning to the working groups for development. Once the working groups are satisfied they've adequately addressed everything, all the proposed amendments go to the executive committee and then to COUNTER's board for approval. And those approved items form the basis for our community consultation process, which is open to everyone interested in COUNTER.
TASHA MELLINS-COHEN: We aim to collect as much feedback as possible over a period of a couple of months with concerns, questions, comments, and all the rest of it returned to the working groups so they can revise their amendments before another round of sign-off and so on. Even once the updated code is released, because we're aware that development can take time, we're careful to ensure that there's at least a 12-month lead time before compliance with the new code becomes mandatory.
TASHA MELLINS-COHEN: Right now, we're at the very first part of the internal development for release 5.1, so it's the very first starting point of the process. So our objectives for release 5.1 are to make the process and reliability of COUNTER audits more simple and more straightforward, to encourage more content providers, more publishers to become compliant, because we feel that that means that more libraries can trust COUNTER reports and makes them more useful.
TASHA MELLINS-COHEN: We are also looking at changing the primary report output. So at the moment, report output is both tabular in CSV and in JSON. We're considering suggesting that JSON should be the primary output. Again, this will help make the auditing process easier. We are carefully considering the terminologies used in the code of practice, including report-naming conventions.
TASHA MELLINS-COHEN: For example, master report and standard view maybe not the most sensible naming conventions for how the reports work. So we're considering just calling them reports and derived views rather than most reporting standard view. We are looking at the use and the value of search metrics in light of changing technologies to see whether they're still reliable and helpful.
TASHA MELLINS-COHEN: Of course, we're trying to resolve any issues that we've identified during the development of earlier versions of the code of practice. And we will be collaborating with the community to encourage more transparency about COUNTER metrics while respecting commercial confidentiality. Now, not all of our objectives for 5.1 relate solely to open access, so for the rest of this talk, I'm going to be focusing in very strongly on those bits which are applicable to OA.
TASHA MELLINS-COHEN: You'll have noticed I didn't specifically mention OA in that objectives list. That's because it's touched on in so many other areas, starting with the reports themselves. So currently, platforms that deliver journal and/or book content only need to provide platform, title, and database reports. They don't need to provide item reports that specify the usage of each article or book chapter.
TASHA MELLINS-COHEN: And particularly, when it comes to evaluating the use of OA materials, item-level reporting is important. So consider the case of an article in hybrid journal, published open access after an APC was paid by the library. Under the terms of the current release of the code of practice, the content provider must offer both platform and title reports that allow the library to assess usage of the journal as a whole within the context of the institution.
TASHA MELLINS-COHEN: However, the content provider does not have to offer the world reporting to assess total usage of the channel globally, though we do recommend it. And the content provider does not need to offer an item report that would allow the library to assess usage of specific articles. So we are now considering whether to introduce the item report as a requirement for those host types that deliver journals and books.
TASHA MELLINS-COHEN: Library feedback has been that they would find this valuable, while providers have implied that those additional mandatory and potentially very large reports do add additional barriers to compliance. Resolving this is a priority, and solutions under discussion include making the item report as showed rather than a must and considering size limits on the number of items requested as part of an item report. And as I said, the other aspect of reports that we're looking at is nomenclature.
TASHA MELLINS-COHEN: A second aspect of our terminology work is focused on access types, or for those of you less intimately familiar with the code of practice, whether or not something is OA. In release 5, we introduced four access types, only two of which are in widespread use, which are controlled. So at the time of usage, the content item was behind a paywall.
TASHA MELLINS-COHEN: So that would include access to content because of trial subscriptions, anything made free temporarily for marketing purposes, and also includes delayed open access, so content made available after an embargo. The other item in common use is OA gold, which is specifically content that is immediately and permanently available as open access from the day of publication. OA delayed we introduced with the intention of implementing it at a later point.
TASHA MELLINS-COHEN: It has never been implemented. And other free to read is only for institutional repositories. We know that our definitions are not fit for purpose, so we are discussing alternatives. And the current state of play, which is absolutely subject to change, is that controlled would be content that is behind a paywall. No change to that. Open would indicate any content that is openly available, whether it was born open under a Creative Commons license or has been made openly available after an embargo, for example, those publishers who make their archives freely available, provided that there is no intention for the content to return behind a paywall in the future.
TASHA MELLINS-COHEN: So this is permanently available from the day of use. And the third item would be not controlled. So this would be material that is free for marketing purposes, for example, where it should be paywalled or it was born paywalled, and it will return behind a paywall in the future, but it is not currently controlled. I hope that makes sense. Please do ask me questions if you have any.
TASHA MELLINS-COHEN: Finally, on the open access front, we have questions of transparency. One aspect of this is the idea of making the world reports openly available. We would love to see usage take its place among the suite of article-level metrics or general-level metrics that publishers make available. And a second aspect relates to the very widespread nature of usage.
TASHA MELLINS-COHEN: At the moment, publishers don't know how much usage their content receives on abstracting and indexing platforms and scholarly social media sites. So we would love to encourage sharing of such information to make the world reports more representative of actual total usage. Both of these aspects of transparency do raise questions of commercial confidentiality, so we're treading very carefully.
TASHA MELLINS-COHEN: And with apologies if I'm starting to sound like a broken record. We will be consulting in detail with all of those who could be affected by any changes. So when we released version 5.02, the code of practice last year, we also announced that a larger release, 5.1, will be coming up for consultation in 2022. I've given you a sneak peek as to what's going to be coming. Please do contact me, contact Lorraine, and get involved in that consultation.
TASHA MELLINS-COHEN: We would love to have you. Bye.
CHRISTINA DRUMMOND: I'm Christina Drummond, and over the next 20 minutes, I'm going to introduce you to the concept of international data spaces and describe how global team is aiming to leverage this framework to support community-governed sharing of high-quality, interoperable open access book usage data. To get started, I'm going to define some terminology. You're likely familiar with the evolution from flat data to relational databases and data warehouses and lakes to the link data bridges that connect them.
CHRISTINA DRUMMOND: As these data storage structures have evolved, so has the structured metadata and query standards required to navigate them. We've witnessed an evolution in siloed data sets to multi-dimensional link data environments. Yet we're on the cusp of another evolution as we shift to data spaces as a way to manage the data agreements, secure data exchange, and audit allowable use of sovereign data at scale, or in other words, across the stakeholders within a given data ecosystem.
CHRISTINA DRUMMOND: Our work in the OA ebook data trust, or the open access ebook usage data trust effort, has been occurring on this leading edge of innovation. So we're gleaning directions from emerging standards and frameworks in Europe so that we can improve the state of open access usage analytics for scholarly communications, research, and innovation policy around the world. This is vital because to tell the story of an author's open access publication impacts, we need to understand usage across the myriad of internet-based platform services and websites, some of which provision usage data in the open, but others which treat this usage data as proprietary or sensitive.
CHRISTINA DRUMMOND: And currently, individual libraries, presses, publishers, and research offices are all working to curate, aggregate, and analyze such multiple streams of data so they can leverage the results for strategic decision-making. Yet such data stewardship is no simple feat. This brings me to the heart of our OA ebook data trust effort. We grew from a shared interest among partners to ease the burdens associated with usage data reporting by exploring shared infrastructure to generate economies of scale.
CHRISTINA DRUMMOND: Last year, our board members clarified this into a mission and vision for our data trust, focusing in on fostering the exchange of reliable usage data in a trusted, equitable, and community-governed way. The challenge we took on is figuring out how to do this in a trusted and transparent way that works at scale. Luckily for us, we're not the only sector industry exploring how to do this, and we were able to learn from others.
CHRISTINA DRUMMOND: Over the next 15 minutes, I'm going to describe the international data space model that has most inspired our path forward, while sharing the journey and lessons learned regarding open access book usage analytics. Then I'm going to ask, actually, for your input and help considering which organizations are positioned to play key roles in this data space for open access conversion. With that, you should note as we begin that organizational networks to exchange, process, and steward data are not new.
CHRISTINA DRUMMOND: There are many wonderful resources out there to guide the development of such efforts from the minds at Sage Bionetworks, the Open Data Institute, the GovLabs Data Stewards Network, Mozilla's Data Futures Lab, and others. What's important to note here is that the governance mechanisms, sustainability models, and technical infrastructure requirements vary based on the type of data institution and the terms for the data cooperatives.
CHRISTINA DRUMMOND: But common across many of these organizational networks is their ability to clarify participation obligations related to how that shared data is to be governed, stewarded, secured, and processed by those who are participating. These data institutions can span everything from open data commons with minimal obligations to complex public-private partnership agreements. And models are emerging to inform how this data exchange can be controlled.
CHRISTINA DRUMMOND: One of the phrases I love is as open as possible, but as controlled as necessary. The EU-based data spaces model appears to be the farthest along, but it's similar in many ways to Canada's developing Zero-Copy Integration standard and MIT's open analytics paradigm. For this talk, I'm going to speak to the IDS model, or International Data Spaces model.
CHRISTINA DRUMMOND: Analogous to how institutions set access and data use policies for staff as required for security, privacy, and regulatory compliance, the IDS approach sets forth tailored data rules as well as the technical and operational mechanisms to ensure compliance with such rules. This may feel like a lot of formality and complexity, and in fact, we're still learning how much of this model is required for scholarly communications data. Over the course of this talk, I'm going to share my personal opinions on why I feel such complexity may be required for book usage, even though we have efforts like IRIS and standards such as COUNTER now are in existence.
CHRISTINA DRUMMOND: And to kick us off, I just want to have us all think about a document and tracked changes for a moment. So over the past few decades, we've seen word processing evolve from flat files distributed for edits and laboriously reaggregated by aggregators to today's dynamic real-time collaborative processing online. If you think of where we're at with usage data, open data is freely shared, but data that must be closed cannot be represented in the same platform in the same way.
CHRISTINA DRUMMOND: We have issues with teams, users not being able to collaborate in GDocs. So if you think about this for a second, we're passing around lots of copies of usage data to someone downstream to innovate with and repackage. The problem with this, though, is beyond the labor expended. The data quality and interoperability degrades as things are passed around without provenance and transformation tracking.
CHRISTINA DRUMMOND: It can leave us with a lot of versions, a few version numbers, and unclear authorities downstream. So if we want to prepare for a future with automated data exchange, contract management, and compliance, the community of stakeholders needs to agree upon shared policies and have the ability to audit and certify compliance with those policies. This is why for all these data intermediary platforms, you have to have diverse community stakeholder governance and engagement.
CHRISTINA DRUMMOND: This is a good time to note that within our effort, we continue to find ways to diversify and engage partners in the development of a shared infrastructure. Since 2020, I have been humbled to work alongside members of this community with experts in open access book publishing and usage data through the support of the Andrew W. Mellon Foundation. I've engaged thought leaders across five continents in this process and continue to have an open door to others who want to join.
CHRISTINA DRUMMOND: So how does a community of scholarly communication stakeholders interested in usage data leverage the IDS model? What you see here is our internal model that informed a beta open source environment for a dashboard pilot that we operated over the past two years. Our technical team, based at Curtin University, pulled together public and private sources of usage data to then secure that data, process it, and share it back the aggregated views of the information in a controlled fashion.
CHRISTINA DRUMMOND: What we've learned in piloting our model is that while the open source software is relatively straightforward from a technical perspective, one must still manage the legal and regulatory data governance issues related to bringing together open and sensitive data. Data aggregators and brokers must consider constantly the shifting landscape of data use terms, service transition, such as when MAG sunsetted and OpenAlex came up, as well as those institutional data sharing agreements with the data owners, in addition to national data regulatory frameworks.
CHRISTINA DRUMMOND: So as this work spans library information science, it's not just those offices. It's legal offices, it's privacy offices, data governance teams that are all involved at each institution. And so one of things we're trying to do is investigate, what would it take to support this data curation aggregation and benchmarking? And here, we piloted it with six institutions and confirmed that to think about how to do this at scale, we have to address sustainability and operational integrity and while streamlining that data sharing agreement process and the compliance process for all involved so people will trust us with their protected data.
CHRISTINA DRUMMOND: This is where the IDS model helps us to think forward. At the core, the IDS model aims to operationally maintain trust across stakeholders, which raises a number of questions. How do you do this? How do you help partners trust that they can share data for a specific purpose and that the data will be protected as needed and only used for that specific agreed-upon purpose? How can you assure those providing the data, those that are using the data, and the subjects of the data that by participating, they will not be harmed?
CHRISTINA DRUMMOND: Their reputations, their livelihoods, or their revenue generation won't be harmed. And how do you do this at a global scale consistently across organizations? The technical data lake we piloted with access controls doesn't address these questions. Access and cybersecurity controls are part of the puzzle, but so are the data sharing and use agreements.
CHRISTINA DRUMMOND: This is where the appeal surfaces for the International Data Space model. It provides governance, legal, and technology frames to meet the expectations of different stakeholders within a given ecosystem. I specified data governance, security, and use requirements, and putting the technological, operational, legal means in place to manage compliance, International Data Space consortia are helping organizations to streamline this data stage across their data ecosystems.
CHRISTINA DRUMMOND: This helps innovative platforms and services securely access disparate sovereign data managed upstream by those primary data-owning organizations in a way that is as open as possible, but as controlled as necessary to provide for security, trust, and compliance with international data regulations. This also makes it possible to get closer to real-time quality data from the source. Frameworks such as the IDS provide guidance on how to do this across the dimensions of interoperability, trust, data value, and governance.
CHRISTINA DRUMMOND: And to implement such a framework, you first have to have a solid understanding of your data ecosystem. Building upon discussions that started in 2015, with the generous support of Andrew W. Mellon Foundation, our community has now collaboratively documented the data supply chain to stakeholders and the use cases for open access book usage data. Consulting teams from both Clark and Esposito and MoreBrains Consulting interviewed leading stakeholders while our communities of practice members contributed to 42 ideation boards through nearly 500 participant sessions.
CHRISTINA DRUMMOND: We learned an immense amount about the open access book usage data ecosystem, both in terms of the parties involved, their interests and activities with usage data, and a lot of the dynamics between them. For example, we learned how usage data migrates from platforms and services that create the usage data up to the publishers, libraries who aggregate it to then apply it for decision-making or provide reporting to authors and funders.
CHRISTINA DRUMMOND: We noted how at multiple points, original data sources are copied and then manipulated, adjusted, or processed prior to passing it on, and the errors that can be introduced and propagated show up then downstream, impacting both data quality and trust in the usage analytics for data-driven decision-making. As I mentioned previously, to deepen our understanding, we piloted open source software infrastructure to pilot secure aggregation visualization of that usage data for four university presses, a commercial publisher, and OAPEN.
CHRISTINA DRUMMOND: Our effort explored how one might bring together that open data and privileged usage data so that individual organizations could better view their collective usage across platforms and repositories. Using this technology, leveraging work completed in Europe on her newest project, and in Australia, that Curtin Open Knowledge Initiative, we were able to securely align the copies of the privilege data provided by partners and their services alongside those open sources.
CHRISTINA DRUMMOND: We worked through community consultations to identify consensus around data authorities and processing norms. And while you think our journey may have been complete, we've learned through a legal analysis that while a dashboarding service was emerging, emerging data regulation and, in particular, the European Union's Data Governance Act, required us to pause and rethink our approach.
CHRISTINA DRUMMOND: We've all heard of GDPR, especially given the recent news about Google Analytics and calling it into question. But in addition, Europe is advancing a Data Governance Act to shape how multiparty data exchanges involving European partners would be required to function. Namely, to maintain neutrality, the forthcoming regulation require data intermediator service providers to not also provision analytical services or visualizations.
CHRISTINA DRUMMOND: As for our data trust, a single legal entity could not both facilitate the exchange and aggregation of usage data while also providing data dashboards or analytics as a service. And this is a point where those of us in the US start to question what I'm proposing, as often, dashboarding services are seen as the value add behind many revenue models. This is where I pause to appreciate how much we are truly in the point of transformation within the data economy.
CHRISTINA DRUMMOND: There's a shift on the horizon away from how you always approach data governance and data exchange. Thinking proactively, our project team and advisory board ended up forking our efforts, splitting it into two separate teams late last year. Now it's two distinct efforts, a usage dashboarding service for small, medium publishers, headed by [? co-PIs ?] Niels Stern of OAPEN, Cameron Neylon and Lucy Montgomery of COKI, Katherine Skinner of Educopia, that will continue separate, spitting out with its own governing body separate from the data trust, which I'll be working on with our new board of trustees as its executive director.
CHRISTINA DRUMMOND: So as you can see here in our data trust service to impact model, this effort aims to offer streamlined data exchange benchmarking data along critical mass of open access book publishers and platforms with transparency and accountability baked in. To do this, we know that our next project phase must focus on defining as a community what expectations and guardrails are needed for the data as it traverses trust so that we can put that information into standard data sharing contracts for all participants.
CHRISTINA DRUMMOND: We also need to model a scalable trusted solution built on strategic partnerships among key players in this qualcomm ecosystem. So we don't want to recreate efforts that already exist, but rather, reinforce and leverage them. We are looking at the IDS model as a way to efficiently broker the data, sharing relationships and data transfer protocols across the wide array of public and private parties that generate and use book usage data for downstream innovation and reporting.
CHRISTINA DRUMMOND: Our hope is to streamline data aggregation while improving data quality and also while processing and providing novel data of interest for participants, such as those benchmarking metrics. To do this across global players at the system level, we have to consider how to do this in a scalable, sustainable way that is ethical and trusted. Oh, don't forget, we're doing this balancing act while standing in the shifting sands of global data regulation, which is where we need to adapt as data generated from cookies and Google Analytics may become seen as non-compliant in large segments of the world.
CHRISTINA DRUMMOND: And so one of the ways to manage this diversity data regulation compliance from a data ownership and processing perspective is to stop making copies and instead focus on improving access protocols, checks, and balances, and this is where the IDS model suggests a path forward. So luckily, the IDS reference architecture model, which is now in its third version, is already recognized as a German standard.
CHRISTINA DRUMMOND: And the design principles and data rulebook templates that are provided through the IDS framework others have already used to operationally define scalable systems and associated data obligations in other industries. At the core of this concept is this idea that to be successful, data spaces must work across public and private data generators and data consumers to develop and enforce the norms, governance, and agreements that are necessary to steward the data.
CHRISTINA DRUMMOND: Such governance building blocks are operationalized and scaled via the technical, operational, and legal mechanisms that are appropriate for a given sector. And so with that, we have to manage the compliance of those data rules as codified in that data rulebook. So we're now preparing to do this in our next project phase, where we will look to community stakeholders to help define that data rulebook for usage data trust participation while also piloting the IDS frame architecture with OA book publishing stakeholders.
CHRISTINA DRUMMOND: This brings me to the interactive part of my talk where I'd love to tap into the wisdom of this audience to see who you think is best positioned to play key roles in a data space for open access book usage data. While any organization can provide source data, thereby becoming a data provider, or consume and use the aggregate benchmark metrics that are being generated, acting as a data consumer, there is an open question about which existing organizations are best positioned to play those key roles that are required within the data space ecosystem.
CHRISTINA DRUMMOND: So let's start first by thinking about what is described here as vocabularies providers. So in this case, think of the organizations that manage the standards for open access book-related data. They may manage metadata vocabularies. They may be acting as an authority for the data standards themselves. And so my question to you is, who do you think should be playing this role for open access book usage?
CHRISTINA DRUMMOND: You'll see on your screen, we have an interactive board set up for you to add your ideas. It says if you open a browser, the link is join.groupmap.com/34A-5D4-312. And if you go ahead onto that page, we would love to have your thoughts. I know we're already thinking of editor because of their [INAUDIBLE] in COUNTER, but I'm curious what you think.
CHRISTINA DRUMMOND: Who else is in this space of providing and managing a vocabulary, those standard descriptions for book usage data? So please go ahead and pop them in there. Add your ideas or upvote them. You can just add a thumbs up to the ideas you'd like or I guess a thumbs down if you disagree with what you see. But while you're logging in and doing that, I'm going to go ahead and describe the other thing we'd love to get your thoughts on in this talk, which is, who do you think can play the role of identity provider?
CHRISTINA DRUMMOND: And this is really where we get into identity and access management, authentication. Who, within the open access books usage space, is positioned to manage the identity records for those who participate in the trust? Can certify they are who they say they are and manage those access controls to validate the identity of the participants and what they can access?
CHRISTINA DRUMMOND: So again, on that exact same board, we would love any thoughts about who you have in mind when you think of organizations that are positioned to play this role. So with that, I'm going to go ahead and let folks ideate on their own after this talk and transition us into the next phase [INAUDIBLE].. So thank you for helping to launch us as we go into this next planning effort and so the coming year.
CHRISTINA DRUMMOND: I'll note, we are continuing to learn. This is very much a learning process of what it means to apply the IDS model and the lessons from other industries and sectors to this world of global open access usage data. We're actively developing the agenda and ways to engage the community around the development of that data rulebook and looking for partners interested in piloting the IDS model as data providers or data consumers or even key service providers.
CHRISTINA DRUMMOND: So if you're interested in collaborating in any of those areas, please don't hesitate to reach out. You see by email here, christina@educopia.org. Thanks so much. [MUSIC PLAYING]
Segment:0 .
[MUSIC PLAYING]
SPEAKER: Hello, everyone, and welcome to the session about usage metrics, or more specific, about usage data reporting in exchange for open access material and books. My name is [INAUDIBLE],, and I'm the moderator for this session. We have, in this session, two presentations and three presenters who approach the topic from different angles. In the first presentation, titled Consistent Usage Metrics in an Open Access World, we will hear about how culture, the standard of corporate practice, can help to facilitate usage reports for open access material and why this is important.
SPEAKER: The session aims to answer questions such as how usage research metrics might help demonstrate return on open access investment, whose usage is relevant for open access, and how publishers might benefit from providing item-level content usage as part of a suite of metrics. The presenters are Tasha Mellins-Cohen and Lorraine Estelle. Tasha is an independent publishing consultant and a member of the COUNTER executive committee, and Lorraine is the COUNTER project director.
SPEAKER: The second presentation has the title From Distributed Data Hovering to Upstream Data Quality, How International Data Space Frameworks and Standards May Transform Usage Metrics. It will focus on an open access equal usage reporting and discuss how an infrastructure supporting the exchange and accessibility of trusted usage data may look like. It will address questions such as how an existing infrastructure maps to an international data space model for such usage data, what active research is going on to develop policy guardrails, and how international data space approach can support and reinforce norms for the ethical use of usage data across diverse stakeholders.
SPEAKER: The presenter is Christina Drummond. Christina is the data trust program officer of Educopia Institute. This was my short introduction, and with that, we start with the first presentation, Consistent Usage Metrics in an Open Access World.
TASHA MELLINS-COHEN: Good afternoon, everybody. This is Tasha Mellins-Cohen, presenting on behalf of COUNTER and speaking about evolving COUNTER statistics for open access. So let's kick off. People tend to think about COUNTER reports as one of the pieces of information, one of the inputs, that information librarians use to evaluate subscription content. In our increasingly open world, COUNTER really needs to evolve to address new needs.
TASHA MELLINS-COHEN: So in this session, we will outline the ways in which consistent, credible, comparable usage data remains relevant in a-- sorry, remains relevant in evaluating the investment that libraries are making in open access journal and book content. We will discuss what COUNTER has been doing to facilitate open access reporting within the bounds of the existing release 5.
TASHA MELLINS-COHEN: And we will also explore some future options for enhanced and extended item-level open access reporting in the upcoming release 5.1. So why is COUNTER usage data relevant today if it's historically helped people to evaluate subscriptions? As we've already said, libraries have traditionally relied on COUNTER usage reports for measuring subscription content.
TASHA MELLINS-COHEN: That could include total usage or cost per use of the material to which they subscribe or the number of turnaways, that is, how many times their users were unable to access content to which the library does not subscribe. As an increasingly large component of scholarly output is being made open access, libraries are now becoming increasingly interested in the usage of that OA content.
TASHA MELLINS-COHEN: That might be the usage of the content published by authors affiliated with that institution or the cost per use of the OA materials that the library has funded or the wider institution has funded. There are a lot of claims made about open access providing better value for money as well as its ability to expand the reach of scholarly outputs and help authors attract more readers. By making use of COUNTER reports, libraries and publishers and funders can really start to validate the truth of those claims.
TASHA MELLINS-COHEN: Just before we move on, a quick word on cost per use. This is a really interesting concept that should be calculated consistency with a small variation of input. So to calculate cost per use for a subscription journal, you need to know the price or cost of the subscription that was paid by your institution, you need to know the number of times that people associated with your institution have made use of content covered by the subscription-- and we recommend using COUNTER's unique item requests metric for this-- and then you divide the first by the second to get your cost per use for your subscription.
TASHA MELLINS-COHEN: For open access concept, the same concept applies, but the calculation is usually run at the article level, not the subscription level. So for this, you need to know the cost paid to make the article OA. That might be an article processing charge, it might be a share of your publishing retail, if you have one. You need to know the total number of times the article has been used by anyone in the world, and that's still using the unique item requests.
TASHA MELLINS-COHEN: And then you divide the first by the second to get your cost per use for your open access spend. So please, do shout if you've got questions about that. I will be answering questions later. So we know about cost per use. How does COUNTER actually facilitate OA reporting? Back in 2016, when we were starting to develop release 5 of the COUNTER code of practice, we incorporated a mechanism for publishers to easily report on OA content, and it is written like this.
TASHA MELLINS-COHEN: The COUNTER code of practice currently states that where you've got the institution name in the report header, that for open access publishers and repositories where it is not possible to identify usage by individual institutions, the usage should be attributed to the world. And that's section 3.2.1, table 3F. And I know that's a lot to remember. When the rule was added, the focus was on fully open access publishers.
TASHA MELLINS-COHEN: The expectation we had was that those fully OA'd publishers would not try to attribute usage to an institution. So a report to the world was intended to include all global usage, whether it was attributed to an institution or not. To explain that a little, I need to go back to authentication and authorization.
TASHA MELLINS-COHEN: And if you've read my article on the subject, you know I quite enjoy this. Authentication and authorization start with a user, in our case, Sam, visiting a publisher platform. The first thing that platform will do, usually invisibly, is check to see whether Sam can be authenticated. That is, does their IP match an institution's records? Have they already logged in with Shibboleth? Have they come through GetFTR, whatever?
TASHA MELLINS-COHEN: If Sam can be authenticated as belonging to an institution, all of their usage will then be attributed to that institution. If Sam cannot be authenticated, they may still be able to use content if it's open access, if it's free to read. And the platform will still track that usage, but the usage cannot be attributed to a particular organization. Now, we know that fully open access publishers can make use databases of IP details for institutions all over the world to help them attribute usage, but this isn't necessary.
TASHA MELLINS-COHEN: Even when a publisher does use such a database of IP ranges, so much usage happens outside of institutional domains outside of campus, it cannot be attributed to an institution. Some publishers have reported up to 80% of their content use is not within an institutional agreement. So now we think about attribution. Total usage of publisher platforms is composed of non-attributed and attributed usage, though most open access platforms will only have non-attributed usage.
TASHA MELLINS-COHEN: Within that split, whether the content is paywalled or free-to-read or open access is entirely secondary. By reporting total usage, that is, by reporting to the world, open access publishers can easily provide COUNTER reports to libraries and funders and even make them openly available if they choose to do so.
TASHA MELLINS-COHEN: The latest release of the code of practice, that's release 5.0.2, which came out in summer last year, includes some optional elements that allow people to provide more granular usage reporting. One of those is attributed. So this element simply shows whether or not the content provider was able to attribute the usage to an institution. So the valid values are yes or no.
TASHA MELLINS-COHEN: So you could have a world report that says attributed, not attributed. You can also-- or publishers and report providers could also include country name and country code, which makes use of ISO 3166-1 to show global usage broken down by country. And they could also implement region name and region code, which makes use of the complementary ISO 3166-2 to show that global usage broken down by country subdivision.
TASHA MELLINS-COHEN: So for example, that would be New South Wales within Australia. These new elements mean that in addition to reporting total usage, the world reports could be broken down to show attribution types or geographic locations. And we worked with librarians during 2021 to understand what they needed to be able to do with our total usage reports, the world reports, to make them really valuable for OA reporting.
TASHA MELLINS-COHEN: And the four things that I've listed on this slide came through as priorities. The absolute top one was that a librarian might want to retrieve a report that showed their institutional usage plus all other usage so they can contextualize what their institution is doing. The second one was that of their institutional usage, they would want to compare that with all other attributed usage, so all other institutional usage and with all non-attributed usage.
TASHA MELLINS-COHEN: And the reason that they wanted that was to be able to see whether publisher claims of most usage coming from outside of library bounds, outside of institutional affiliation, was accurate. The third place request from librarians was an interest in breaking down usage by country of user, so a geographic breakdown where unknown would indicate users whose location can't be placed. And then fourthly, seeing their institutional usage within the context of their country.
TASHA MELLINS-COHEN: Now, a lot of the librarians we spoke to were very aware that IP ranges are occasionally reallocated and that with the use of VPNs and similar workarounds, geographic breakdowns may not be entirely reliable. And because of that, while country usage was of interest, regional usage really was not, particularly. We did, of course, speak to publishers as well. They weren't unanimously in favor of the new elements, but they could see the value of them.
TASHA MELLINS-COHEN: And because these are optional elements, none of the publishers that we spoke to raised any objections to their inclusion in the code of practice. So believe me, we know that everyone, publishers, COUNTER report providers, librarians, they all want COUNTER to be as stable as possible so that the whole community can feel secure that reports from different providers are consistent and credible and comparable.
TASHA MELLINS-COHEN: So just to reassure everybody, I will briefly describe the process behind a COUNTER release. It starts with a lot of internal development work undertaken by our really dedicated volunteers from the technical group, the communications group, and in this case, the open access board. These people evaluate all the requests we've received, pick up on potential areas of confusion in the code, and assess what's going on more broadly within scholarly communications to create a list of target areas for code revision.
TASHA MELLINS-COHEN: That list is then taken to the executive committee for approval before returning to the working groups for development. Once the working groups are satisfied they've adequately addressed everything, all the proposed amendments go to the executive committee and then to COUNTER's board for approval. And those approved items form the basis for our community consultation process, which is open to everyone interested in COUNTER.
TASHA MELLINS-COHEN: We aim to collect as much feedback as possible over a period of a couple of months with concerns, questions, comments, and all the rest of it returned to the working groups so they can revise their amendments before another round of sign-off and so on. Even once the updated code is released, because we're aware that development can take time, we're careful to ensure that there's at least a 12-month lead time before compliance with the new code becomes mandatory.
TASHA MELLINS-COHEN: Right now, we're at the very first part of the internal development for release 5.1, so it's the very first starting point of the process. So our objectives for release 5.1 are to make the process and reliability of COUNTER audits more simple and more straightforward, to encourage more content providers, more publishers to become compliant, because we feel that that means that more libraries can trust COUNTER reports and makes them more useful.
TASHA MELLINS-COHEN: We are also looking at changing the primary report output. So at the moment, report output is both tabular in CSV and in JSON. We're considering suggesting that JSON should be the primary output. Again, this will help make the auditing process easier. We are carefully considering the terminologies used in the code of practice, including report-naming conventions.
TASHA MELLINS-COHEN: For example, master report and standard view maybe not the most sensible naming conventions for how the reports work. So we're considering just calling them reports and derived views rather than most reporting standard view. We are looking at the use and the value of search metrics in light of changing technologies to see whether they're still reliable and helpful.
TASHA MELLINS-COHEN: Of course, we're trying to resolve any issues that we've identified during the development of earlier versions of the code of practice. And we will be collaborating with the community to encourage more transparency about COUNTER metrics while respecting commercial confidentiality. Now, not all of our objectives for 5.1 relate solely to open access, so for the rest of this talk, I'm going to be focusing in very strongly on those bits which are applicable to OA.
TASHA MELLINS-COHEN: You'll have noticed I didn't specifically mention OA in that objectives list. That's because it's touched on in so many other areas, starting with the reports themselves. So currently, platforms that deliver journal and/or book content only need to provide platform, title, and database reports. They don't need to provide item reports that specify the usage of each article or book chapter.
TASHA MELLINS-COHEN: And particularly, when it comes to evaluating the use of OA materials, item-level reporting is important. So consider the case of an article in hybrid journal, published open access after an APC was paid by the library. Under the terms of the current release of the code of practice, the content provider must offer both platform and title reports that allow the library to assess usage of the journal as a whole within the context of the institution.
TASHA MELLINS-COHEN: However, the content provider does not have to offer the world reporting to assess total usage of the channel globally, though we do recommend it. And the content provider does not need to offer an item report that would allow the library to assess usage of specific articles. So we are now considering whether to introduce the item report as a requirement for those host types that deliver journals and books.
TASHA MELLINS-COHEN: Library feedback has been that they would find this valuable, while providers have implied that those additional mandatory and potentially very large reports do add additional barriers to compliance. Resolving this is a priority, and solutions under discussion include making the item report as showed rather than a must and considering size limits on the number of items requested as part of an item report. And as I said, the other aspect of reports that we're looking at is nomenclature.
TASHA MELLINS-COHEN: A second aspect of our terminology work is focused on access types, or for those of you less intimately familiar with the code of practice, whether or not something is OA. In release 5, we introduced four access types, only two of which are in widespread use, which are controlled. So at the time of usage, the content item was behind a paywall.
TASHA MELLINS-COHEN: So that would include access to content because of trial subscriptions, anything made free temporarily for marketing purposes, and also includes delayed open access, so content made available after an embargo. The other item in common use is OA gold, which is specifically content that is immediately and permanently available as open access from the day of publication. OA delayed we introduced with the intention of implementing it at a later point.
TASHA MELLINS-COHEN: It has never been implemented. And other free to read is only for institutional repositories. We know that our definitions are not fit for purpose, so we are discussing alternatives. And the current state of play, which is absolutely subject to change, is that controlled would be content that is behind a paywall. No change to that. Open would indicate any content that is openly available, whether it was born open under a Creative Commons license or has been made openly available after an embargo, for example, those publishers who make their archives freely available, provided that there is no intention for the content to return behind a paywall in the future.
TASHA MELLINS-COHEN: So this is permanently available from the day of use. And the third item would be not controlled. So this would be material that is free for marketing purposes, for example, where it should be paywalled or it was born paywalled, and it will return behind a paywall in the future, but it is not currently controlled. I hope that makes sense. Please do ask me questions if you have any.
TASHA MELLINS-COHEN: Finally, on the open access front, we have questions of transparency. One aspect of this is the idea of making the world reports openly available. We would love to see usage take its place among the suite of article-level metrics or general-level metrics that publishers make available. And a second aspect relates to the very widespread nature of usage.
TASHA MELLINS-COHEN: At the moment, publishers don't know how much usage their content receives on abstracting and indexing platforms and scholarly social media sites. So we would love to encourage sharing of such information to make the world reports more representative of actual total usage. Both of these aspects of transparency do raise questions of commercial confidentiality, so we're treading very carefully.
TASHA MELLINS-COHEN: And with apologies if I'm starting to sound like a broken record. We will be consulting in detail with all of those who could be affected by any changes. So when we released version 5.02, the code of practice last year, we also announced that a larger release, 5.1, will be coming up for consultation in 2022. I've given you a sneak peek as to what's going to be coming. Please do contact me, contact Lorraine, and get involved in that consultation.
TASHA MELLINS-COHEN: We would love to have you. Bye.
CHRISTINA DRUMMOND: I'm Christina Drummond, and over the next 20 minutes, I'm going to introduce you to the concept of international data spaces and describe how global team is aiming to leverage this framework to support community-governed sharing of high-quality, interoperable open access book usage data. To get started, I'm going to define some terminology. You're likely familiar with the evolution from flat data to relational databases and data warehouses and lakes to the link data bridges that connect them.
CHRISTINA DRUMMOND: As these data storage structures have evolved, so has the structured metadata and query standards required to navigate them. We've witnessed an evolution in siloed data sets to multi-dimensional link data environments. Yet we're on the cusp of another evolution as we shift to data spaces as a way to manage the data agreements, secure data exchange, and audit allowable use of sovereign data at scale, or in other words, across the stakeholders within a given data ecosystem.
CHRISTINA DRUMMOND: Our work in the OA ebook data trust, or the open access ebook usage data trust effort, has been occurring on this leading edge of innovation. So we're gleaning directions from emerging standards and frameworks in Europe so that we can improve the state of open access usage analytics for scholarly communications, research, and innovation policy around the world. This is vital because to tell the story of an author's open access publication impacts, we need to understand usage across the myriad of internet-based platform services and websites, some of which provision usage data in the open, but others which treat this usage data as proprietary or sensitive.
CHRISTINA DRUMMOND: And currently, individual libraries, presses, publishers, and research offices are all working to curate, aggregate, and analyze such multiple streams of data so they can leverage the results for strategic decision-making. Yet such data stewardship is no simple feat. This brings me to the heart of our OA ebook data trust effort. We grew from a shared interest among partners to ease the burdens associated with usage data reporting by exploring shared infrastructure to generate economies of scale.
CHRISTINA DRUMMOND: Last year, our board members clarified this into a mission and vision for our data trust, focusing in on fostering the exchange of reliable usage data in a trusted, equitable, and community-governed way. The challenge we took on is figuring out how to do this in a trusted and transparent way that works at scale. Luckily for us, we're not the only sector industry exploring how to do this, and we were able to learn from others.
CHRISTINA DRUMMOND: Over the next 15 minutes, I'm going to describe the international data space model that has most inspired our path forward, while sharing the journey and lessons learned regarding open access book usage analytics. Then I'm going to ask, actually, for your input and help considering which organizations are positioned to play key roles in this data space for open access conversion. With that, you should note as we begin that organizational networks to exchange, process, and steward data are not new.
CHRISTINA DRUMMOND: There are many wonderful resources out there to guide the development of such efforts from the minds at Sage Bionetworks, the Open Data Institute, the GovLabs Data Stewards Network, Mozilla's Data Futures Lab, and others. What's important to note here is that the governance mechanisms, sustainability models, and technical infrastructure requirements vary based on the type of data institution and the terms for the data cooperatives.
CHRISTINA DRUMMOND: But common across many of these organizational networks is their ability to clarify participation obligations related to how that shared data is to be governed, stewarded, secured, and processed by those who are participating. These data institutions can span everything from open data commons with minimal obligations to complex public-private partnership agreements. And models are emerging to inform how this data exchange can be controlled.
CHRISTINA DRUMMOND: One of the phrases I love is as open as possible, but as controlled as necessary. The EU-based data spaces model appears to be the farthest along, but it's similar in many ways to Canada's developing Zero-Copy Integration standard and MIT's open analytics paradigm. For this talk, I'm going to speak to the IDS model, or International Data Spaces model.
CHRISTINA DRUMMOND: Analogous to how institutions set access and data use policies for staff as required for security, privacy, and regulatory compliance, the IDS approach sets forth tailored data rules as well as the technical and operational mechanisms to ensure compliance with such rules. This may feel like a lot of formality and complexity, and in fact, we're still learning how much of this model is required for scholarly communications data. Over the course of this talk, I'm going to share my personal opinions on why I feel such complexity may be required for book usage, even though we have efforts like IRIS and standards such as COUNTER now are in existence.
CHRISTINA DRUMMOND: And to kick us off, I just want to have us all think about a document and tracked changes for a moment. So over the past few decades, we've seen word processing evolve from flat files distributed for edits and laboriously reaggregated by aggregators to today's dynamic real-time collaborative processing online. If you think of where we're at with usage data, open data is freely shared, but data that must be closed cannot be represented in the same platform in the same way.
CHRISTINA DRUMMOND: We have issues with teams, users not being able to collaborate in GDocs. So if you think about this for a second, we're passing around lots of copies of usage data to someone downstream to innovate with and repackage. The problem with this, though, is beyond the labor expended. The data quality and interoperability degrades as things are passed around without provenance and transformation tracking.
CHRISTINA DRUMMOND: It can leave us with a lot of versions, a few version numbers, and unclear authorities downstream. So if we want to prepare for a future with automated data exchange, contract management, and compliance, the community of stakeholders needs to agree upon shared policies and have the ability to audit and certify compliance with those policies. This is why for all these data intermediary platforms, you have to have diverse community stakeholder governance and engagement.
CHRISTINA DRUMMOND: This is a good time to note that within our effort, we continue to find ways to diversify and engage partners in the development of a shared infrastructure. Since 2020, I have been humbled to work alongside members of this community with experts in open access book publishing and usage data through the support of the Andrew W. Mellon Foundation. I've engaged thought leaders across five continents in this process and continue to have an open door to others who want to join.
CHRISTINA DRUMMOND: So how does a community of scholarly communication stakeholders interested in usage data leverage the IDS model? What you see here is our internal model that informed a beta open source environment for a dashboard pilot that we operated over the past two years. Our technical team, based at Curtin University, pulled together public and private sources of usage data to then secure that data, process it, and share it back the aggregated views of the information in a controlled fashion.
CHRISTINA DRUMMOND: What we've learned in piloting our model is that while the open source software is relatively straightforward from a technical perspective, one must still manage the legal and regulatory data governance issues related to bringing together open and sensitive data. Data aggregators and brokers must consider constantly the shifting landscape of data use terms, service transition, such as when MAG sunsetted and OpenAlex came up, as well as those institutional data sharing agreements with the data owners, in addition to national data regulatory frameworks.
CHRISTINA DRUMMOND: So as this work spans library information science, it's not just those offices. It's legal offices, it's privacy offices, data governance teams that are all involved at each institution. And so one of things we're trying to do is investigate, what would it take to support this data curation aggregation and benchmarking? And here, we piloted it with six institutions and confirmed that to think about how to do this at scale, we have to address sustainability and operational integrity and while streamlining that data sharing agreement process and the compliance process for all involved so people will trust us with their protected data.
CHRISTINA DRUMMOND: This is where the IDS model helps us to think forward. At the core, the IDS model aims to operationally maintain trust across stakeholders, which raises a number of questions. How do you do this? How do you help partners trust that they can share data for a specific purpose and that the data will be protected as needed and only used for that specific agreed-upon purpose? How can you assure those providing the data, those that are using the data, and the subjects of the data that by participating, they will not be harmed?
CHRISTINA DRUMMOND: Their reputations, their livelihoods, or their revenue generation won't be harmed. And how do you do this at a global scale consistently across organizations? The technical data lake we piloted with access controls doesn't address these questions. Access and cybersecurity controls are part of the puzzle, but so are the data sharing and use agreements.
CHRISTINA DRUMMOND: This is where the appeal surfaces for the International Data Space model. It provides governance, legal, and technology frames to meet the expectations of different stakeholders within a given ecosystem. I specified data governance, security, and use requirements, and putting the technological, operational, legal means in place to manage compliance, International Data Space consortia are helping organizations to streamline this data stage across their data ecosystems.
CHRISTINA DRUMMOND: This helps innovative platforms and services securely access disparate sovereign data managed upstream by those primary data-owning organizations in a way that is as open as possible, but as controlled as necessary to provide for security, trust, and compliance with international data regulations. This also makes it possible to get closer to real-time quality data from the source. Frameworks such as the IDS provide guidance on how to do this across the dimensions of interoperability, trust, data value, and governance.
CHRISTINA DRUMMOND: And to implement such a framework, you first have to have a solid understanding of your data ecosystem. Building upon discussions that started in 2015, with the generous support of Andrew W. Mellon Foundation, our community has now collaboratively documented the data supply chain to stakeholders and the use cases for open access book usage data. Consulting teams from both Clark and Esposito and MoreBrains Consulting interviewed leading stakeholders while our communities of practice members contributed to 42 ideation boards through nearly 500 participant sessions.
CHRISTINA DRUMMOND: We learned an immense amount about the open access book usage data ecosystem, both in terms of the parties involved, their interests and activities with usage data, and a lot of the dynamics between them. For example, we learned how usage data migrates from platforms and services that create the usage data up to the publishers, libraries who aggregate it to then apply it for decision-making or provide reporting to authors and funders.
CHRISTINA DRUMMOND: We noted how at multiple points, original data sources are copied and then manipulated, adjusted, or processed prior to passing it on, and the errors that can be introduced and propagated show up then downstream, impacting both data quality and trust in the usage analytics for data-driven decision-making. As I mentioned previously, to deepen our understanding, we piloted open source software infrastructure to pilot secure aggregation visualization of that usage data for four university presses, a commercial publisher, and OAPEN.
CHRISTINA DRUMMOND: Our effort explored how one might bring together that open data and privileged usage data so that individual organizations could better view their collective usage across platforms and repositories. Using this technology, leveraging work completed in Europe on her newest project, and in Australia, that Curtin Open Knowledge Initiative, we were able to securely align the copies of the privilege data provided by partners and their services alongside those open sources.
CHRISTINA DRUMMOND: We worked through community consultations to identify consensus around data authorities and processing norms. And while you think our journey may have been complete, we've learned through a legal analysis that while a dashboarding service was emerging, emerging data regulation and, in particular, the European Union's Data Governance Act, required us to pause and rethink our approach.
CHRISTINA DRUMMOND: We've all heard of GDPR, especially given the recent news about Google Analytics and calling it into question. But in addition, Europe is advancing a Data Governance Act to shape how multiparty data exchanges involving European partners would be required to function. Namely, to maintain neutrality, the forthcoming regulation require data intermediator service providers to not also provision analytical services or visualizations.
CHRISTINA DRUMMOND: As for our data trust, a single legal entity could not both facilitate the exchange and aggregation of usage data while also providing data dashboards or analytics as a service. And this is a point where those of us in the US start to question what I'm proposing, as often, dashboarding services are seen as the value add behind many revenue models. This is where I pause to appreciate how much we are truly in the point of transformation within the data economy.
CHRISTINA DRUMMOND: There's a shift on the horizon away from how you always approach data governance and data exchange. Thinking proactively, our project team and advisory board ended up forking our efforts, splitting it into two separate teams late last year. Now it's two distinct efforts, a usage dashboarding service for small, medium publishers, headed by [? co-PIs ?] Niels Stern of OAPEN, Cameron Neylon and Lucy Montgomery of COKI, Katherine Skinner of Educopia, that will continue separate, spitting out with its own governing body separate from the data trust, which I'll be working on with our new board of trustees as its executive director.
CHRISTINA DRUMMOND: So as you can see here in our data trust service to impact model, this effort aims to offer streamlined data exchange benchmarking data along critical mass of open access book publishers and platforms with transparency and accountability baked in. To do this, we know that our next project phase must focus on defining as a community what expectations and guardrails are needed for the data as it traverses trust so that we can put that information into standard data sharing contracts for all participants.
CHRISTINA DRUMMOND: We also need to model a scalable trusted solution built on strategic partnerships among key players in this qualcomm ecosystem. So we don't want to recreate efforts that already exist, but rather, reinforce and leverage them. We are looking at the IDS model as a way to efficiently broker the data, sharing relationships and data transfer protocols across the wide array of public and private parties that generate and use book usage data for downstream innovation and reporting.
CHRISTINA DRUMMOND: Our hope is to streamline data aggregation while improving data quality and also while processing and providing novel data of interest for participants, such as those benchmarking metrics. To do this across global players at the system level, we have to consider how to do this in a scalable, sustainable way that is ethical and trusted. Oh, don't forget, we're doing this balancing act while standing in the shifting sands of global data regulation, which is where we need to adapt as data generated from cookies and Google Analytics may become seen as non-compliant in large segments of the world.
CHRISTINA DRUMMOND: And so one of the ways to manage this diversity data regulation compliance from a data ownership and processing perspective is to stop making copies and instead focus on improving access protocols, checks, and balances, and this is where the IDS model suggests a path forward. So luckily, the IDS reference architecture model, which is now in its third version, is already recognized as a German standard.
CHRISTINA DRUMMOND: And the design principles and data rulebook templates that are provided through the IDS framework others have already used to operationally define scalable systems and associated data obligations in other industries. At the core of this concept is this idea that to be successful, data spaces must work across public and private data generators and data consumers to develop and enforce the norms, governance, and agreements that are necessary to steward the data.
CHRISTINA DRUMMOND: Such governance building blocks are operationalized and scaled via the technical, operational, and legal mechanisms that are appropriate for a given sector. And so with that, we have to manage the compliance of those data rules as codified in that data rulebook. So we're now preparing to do this in our next project phase, where we will look to community stakeholders to help define that data rulebook for usage data trust participation while also piloting the IDS frame architecture with OA book publishing stakeholders.
CHRISTINA DRUMMOND: This brings me to the interactive part of my talk where I'd love to tap into the wisdom of this audience to see who you think is best positioned to play key roles in a data space for open access book usage data. While any organization can provide source data, thereby becoming a data provider, or consume and use the aggregate benchmark metrics that are being generated, acting as a data consumer, there is an open question about which existing organizations are best positioned to play those key roles that are required within the data space ecosystem.
CHRISTINA DRUMMOND: So let's start first by thinking about what is described here as vocabularies providers. So in this case, think of the organizations that manage the standards for open access book-related data. They may manage metadata vocabularies. They may be acting as an authority for the data standards themselves. And so my question to you is, who do you think should be playing this role for open access book usage?
CHRISTINA DRUMMOND: You'll see on your screen, we have an interactive board set up for you to add your ideas. It says if you open a browser, the link is join.groupmap.com/34A-5D4-312. And if you go ahead onto that page, we would love to have your thoughts. I know we're already thinking of editor because of their [INAUDIBLE] in COUNTER, but I'm curious what you think.
CHRISTINA DRUMMOND: Who else is in this space of providing and managing a vocabulary, those standard descriptions for book usage data? So please go ahead and pop them in there. Add your ideas or upvote them. You can just add a thumbs up to the ideas you'd like or I guess a thumbs down if you disagree with what you see. But while you're logging in and doing that, I'm going to go ahead and describe the other thing we'd love to get your thoughts on in this talk, which is, who do you think can play the role of identity provider?
CHRISTINA DRUMMOND: And this is really where we get into identity and access management, authentication. Who, within the open access books usage space, is positioned to manage the identity records for those who participate in the trust? Can certify they are who they say they are and manage those access controls to validate the identity of the participants and what they can access?
CHRISTINA DRUMMOND: So again, on that exact same board, we would love any thoughts about who you have in mind when you think of organizations that are positioned to play this role. So with that, I'm going to go ahead and let folks ideate on their own after this talk and transition us into the next phase [INAUDIBLE].. So thank you for helping to launch us as we go into this next planning effort and so the coming year.
CHRISTINA DRUMMOND: I'll note, we are continuing to learn. This is very much a learning process of what it means to apply the IDS model and the lessons from other industries and sectors to this world of global open access usage data. We're actively developing the agenda and ways to engage the community around the development of that data rulebook and looking for partners interested in piloting the IDS model as data providers or data consumers or even key service providers.
CHRISTINA DRUMMOND: So if you're interested in collaborating in any of those areas, please don't hesitate to reach out. You see by email here, christina@educopia.org. Thanks so much. [MUSIC PLAYING]
Form Title Download
Video
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode