Cadmore media player playing video ED TECH TRENDS: Un-Siloing & Digitizing Professional Education
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
ED TECH TRENDS: Un-Siloing & Digitizing Professional Education
Description:
ED TECH TRENDS: Un-Siloing & Digitizing Professional Education
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/24824b65-7e30-4c4c-85bf-2b3c447f968f/thumbnails/24824b65-7e30-4c4c-85bf-2b3c447f968f.png
Duration:
T00H59M23S
Embed URL:
https://stream.cadmore.media/player/24824b65-7e30-4c4c-85bf-2b3c447f968f
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/24824b65-7e30-4c4c-85bf-2b3c447f968f/PS22 July recording.mp4?sv=2019-02-02&sr=c&sig=ZRnAX%2FnEpGgBRPEaS%2FO2AtdD0liVaLTPgs9%2BqZHWV4s%3D&st=2024-09-08T22%3A29%3A32Z&se=2024-09-09T00%3A34%3A32Z&sp=r
Upload Date:
2022-07-14T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
STEPHANIE LOVEGROVE
HANSEN: Welcome.
HANSEN: My name is Stephanie Lovegrove Hansen. I am the director of marketing at Silverchair. And thank you so much for joining us today. This is the second in the 2022 Platform Strategies webinar series. We're really looking forward to today's discussion on education technology. But before we get started, I'll just cover a few logistics for us. As I mentioned, this is the second event in this year's webinar series.
HANSEN: And we're hoping that when you registered for this event, you also registered for the other events we're hosting. But if you didn't, it's not too late. So you can follow the same registration link to save the date for those events. You can also visit our website for the recording from our kickoff event on tech trends, which we held in May. This series is a set of free virtual events designed for thought leaders to discuss their platform, data, and technology strategies with colleagues.
HANSEN: So we hope that in addition to hearing from our thoughtful speakers today, that you'll continue in the tradition of the event and also engage in the discussion via the chat and/or the Q&A features of the webinar. If you like the topic enough that you want to continue the conversation further, this year we've introduced some small group roundtables in the month following the event where attendees can join in live in more of a meeting format to continue discussions on the topic.
HANSEN: So we just held our first one for the Tech Trends webinar and had a really engaging discussion. So just email strategies@silverchair.com if you'd like to receive an invitation for that. There will also be a link for that in the follow-up email that you'll get from today's event. This event is being recorded, and a copy of the recording will be made available on our website. And finally, at the end of the event, you will get a survey requesting your reading of this event, which just helps us with future planning.
HANSEN: And we really appreciate your feedback. With that, I will hand it over to today's moderator, Jake Zarnegar, Silverchair's Chief Business Development Officer. Jake.
JAKE ZARNEGAR: OK, thank you, Stephanie. I'm glad you didn't have to go through a list of side effects that this webinar might cause. So I'm really excited about today and this hour of content that we have for you all today. We're going to be talking about the future of education, edtech. We're going to be talking about it, in this case, from the perspective of medical and health content. But a lot of the concepts you hear today will be applicable to many aspects of digital and online education as well.
JAKE ZARNEGAR: And we have just a great panel today. We have joining us Laura Ricci of Clarke & Esposito. We have Dr. Amit Joshi of the American Board of Surgery, Amanda Fielding of McGraw-Hill Professional, and Anna Salt Troise of the American Academy of Orthopedic Surgeons. Now, if you'll notice there, we had someone representing a board who does certification, a commercial publisher, a professional society, and an industry analyst there as well.
JAKE ZARNEGAR: So it's a great kind of 360 survey of the participants in education. And Dr. Joshi, of course, also a participant as a taker of this education as well as a provider. So he can also provide that perspective for us. So to get us started today, Laura Ricci is going to share some trends and market analysis that Clarke & Esposito has done recently. Silverchair actually commissioned research from Clarke & Esposito recently, and it was such an interesting outcome that we thought it would be great to share with all of you here.
JAKE ZARNEGAR: So, Laura, I'm going to turn it over to you to kick us off.
LAURA RICCI: Great. Thank you, Jake. Let me share my screen. All right. So I'm really excited to be part of this fabulous panel, and I'm looking forward to hearing from the other speakers. But in thinking about this topic, I wanted to use my time to set the stage a little bit. As Jake said, I work for Clarke & Esposito, which is a consultancy focused on professional publishing in education.
LAURA RICCI: At our firm, we work with a lot of societies and publishers, typically those who serve active professionals to help them thrive in their roles. So that's where our view of the industry really tends to focus. And I think a lot of the things happening in professional education space really mirror other trends throughout other levels of education too. So today, I want to focus on the three key trends that we're seeing in professional medical education.
LAURA RICCI: And they are-- if I can get the screen to advance. Oh, dear. Already the PowerPoint is doing the thing. Back. OK. So those three key trends I mentioned. So first, the delivery of professional education shifting from in-person to online.
LAURA RICCI: Second, the closer interplay developing between assessment and education. And lastly, how those two trends together allow for more data-driven approaches to education, which unlocks new capabilities that improve engagement and outcomes. And I'm going to take these one at a time. So let's start with the move of education online. Now, I'm sure this is not a surprise to anyone.
LAURA RICCI: We've all lived through the COVID pandemic. But let's put some data behind it. So this data that we're looking at here comes from the Accreditation Council for Continuing Medical Education, or ACCME. And it shows the number of CME activities offered in 2019 and 2020. Now, there are lots of different kinds of medical education, and accredited CME is just one piece.
LAURA RICCI: But one of the reasons why we like ACCME data is because it reflects all the different kinds of education. So the key things to see here is that COVID led to a decrease in in-person education, these in-person courses, conferences, et cetera. And there was also, of course, an increase in virtual courses and online enduring-- meaning on demand-- materials. So in 2020, online enduring materials were the most common CME offering for the first time.
LAURA RICCI: Now, we are waiting to see whether the data shows these trends continue into 2021. But it's really likely anecdotally that some of these trends are going to endure. Now, ACCME data is also great because it shows how education is not just coming from publishers. There are lots of different sources of education. Now, publishers and societies are the first and third leading sources of CME, but physicians are also getting education from work, from their health care system employer or their affiliated school of medicine.
LAURA RICCI: But if you think about that shift towards online learning, you can see how publishers and societies are really the leading source of those activities, from journals CME, internet enduring materials, and live internet activities. So these providers have a unique opportunity to really lead change in education. So that's trend one. The second key trend I'll talk about is not just how physicians are receiving education, it's also how they're being assessed and certified.
LAURA RICCI: So physicians are typically certified by their specialties member board. And the American Board of Medical Specialties, or ABMS, is a federation of 24 medical boards which sets standards about physician assessment and certification. Now, in October 2021, ABMS approved new standards for continuing certification among its member boards. Now, two of the major changes we'll talk about in those standards are that instead of offering a single cumulative point in time assessment every 10 years, now they're encouraging boards to assess longitudinally, meaning at regular intervals, say every six months or so, with recertification based on achieving a minimum performance threshold over that time.
LAURA RICCI: And the standards also require boards to offer formative assessment, meaning instead of just providing a snapshot of did you pass or fail, boards will give feedback about gaps in knowledge for that physician. So that physician can then seek out more information and address that gap. Now, where this is relevant is that publishers and societies have long been supportive of physicians studying for board certification.
LAURA RICCI: And what we're seeing now is there are opportunities to provide better education because of this new focus on regular formative assessment. It creates this opportunity for this virtual cycle where a learner can take an assessment, identify gaps in knowledge and skills, and then focus future learning activities on remediating that specific gap. And this cycle is the end goal of that move to continuous assessment.
LAURA RICCI: But it doesn't have to happen in just a formal board assessment, either. So with the move to online, any module can incorporate mini assessments to gauge understanding and whether or not the physician grasped everything, all the important concepts. So what this really is is data-driven learning. And if you combine these two trends, the move online and the increasing information about knowledge gaps, it unlocks all these opportunities to improve the way that education is being developed and delivered.
LAURA RICCI: So when we ask publishers and societies what they look forward to in medical education, this is what we hear. We hear it's increasingly about getting the right information to the learner at the right time, and this can be through providing better discovery and searchability of education content. It could be personalizing recommendations, so suggesting what to study next based on what they already know about that physician.
LAURA RICCI: There's also more information back about how each learner is performing. Are they passing? How do they compare to peers? And data can help to guide the curriculum itself. Supports capture what concepts are most challenging to test takers, and they provide this data to the community so societies and other publishers can develop a curriculum to target those gaps.
LAURA RICCI: So this is not just for individual learners, this is for the specialty as a whole, which is pretty cool. And there's less focus on a particular product, but instead an objective. So content in different formats which relate to the same topic can be presented so that there's more than one way to teach a concept and different learners can gravitate towards different learning styles. So you put this together, and there's an opportunity for online and data-driven education to be more powerful and impactful.
LAURA RICCI: You can give more feedback to physicians. They don't have to wait 10 years to see whether or not they're still in step with their field. Societies and publishers have way more visibility into whether their education modules are teaching effectively. Different concepts across the field can be identified so that educators can respond right away.
LAURA RICCI: And education providers have a lot more avenues to reach learners and more creative ways to keep them engaged as they learn. So that has been a really quick overview. There's a lot more I could say. But personally, I find this really inspiring, and I'm really looking forward to the discussion with the other panelists. But for now, I will turn it back to Jake.
LAURA RICCI: Thank you.
JAKE ZARNEGAR: Great. Thank you, Laura. That was a great setup there of the kind of trends and changes. I, for one, am very interested in the assessing the impact of the education and having more of an immediate feedback loop of whether folks are learning. The internet gives us a great opportunity to do that rather than, say, send something out to someone and not really know whether they've learned the topics from that.
JAKE ZARNEGAR: On that note, we're now going to look at three new and newly developed or newly updated products that address medical education for specific disciplines. And starting with Dr. Joshi, who is not just the editor-in-chief of SCORE, but also professor and associate dean at Cooper Medical School in New Jersey. And I believe you do a little surgery in your spare time also. So I'm going to turn it over to you.
AMIT JOSHI: You can find me every Thursday at a local operating room. Thanks, Jake. Thanks, Stephanie, for the invitation. It's always a pleasure to talk with our friends from across the Silverchair universe. And so I was asked to share a little bit of our origin story about SCORE, and so I'm going to go through that. SCORE was founded in the early 2000s, the Surgical Council On Resident Education, and formed by seven different surgical organizations.
AMIT JOSHI: And the running joke is, if you stop in your local OR lounge and you ask two surgeons for an opinion, you're going to get three different opinions. And so to get seven different organizations to agree to form SCORE was actually a fairly Herculean effort and has paid huge dividends over the last 15 years. But the SCORE consortium was founded in 2004. And the mission is worth repeating-- to improve the education of residents who are trainees in general surgery and related specialties through the development of a national curriculum.
AMIT JOSHI: And so when you read that mission, you realize that prior to this, there was no national curriculum, that you went through surgical residency five, six, seven years and you were sort of just told to know everything, to learn what you needed to learn, but there wasn't actually any prescribed body of knowledge that you needed to know. So just the formation of the curriculum by itself was fairly groundbreaking.
AMIT JOSHI: And it took a few years. So from 2004 to really 2008, the curriculum outline was defined. And that was basically to say, yes, you need to know this, no, you don't need to this, this is advanced, this is basic. Over the next couple of years, all of this curriculum was deployed on a web portal, which back then was a pretty groundbreaking thing. And then a subscription started. So 2008, the first group of beta test subscriptions were 33.
AMIT JOSHI: And then over the next 10 years, it steadily increased and basically leveled out around 2019. You'll see that number of 274 subscribing programs. There really are only about 280 subscribing programs, at least back in 2019. Currently, there are probably something like 285, 300 programs in general surgery. But then if you look at the bottom left, these are 2022 numbers.
AMIT JOSHI: We've subsequently expanded the curricula into other fields. So it's not just general surgery, it's vascular surgery, pediatric surgery, surgical oncology, surgical critical care. And it's not just programs in the US, but international, Canadian-- I know that Canadians always laugh when we don't include them in international-- but programs really across the country and North America.
AMIT JOSHI: And so currently, we stand at around 11,000 trainees across the world who have core subscriptions. This is the curriculum outline. This is one small segment of it. This is anesthesia. And I always pick anesthesia because you wouldn't necessarily think that general surgeons need to know about anesthesia, but we do. And so by defining this and saying, yeah, you need to know this, it forced trainees from across the spectrum to start to learn that.
AMIT JOSHI: And then within anesthesia, these are the specific topics that you need to know. So in SCORE, the module is really the scaffolding around which everything else is built. And so I was go back, and this is what a module looked like in 2013. It was really just a Word document saying, you need to know this, but with really no embedded content. You'll see highlighted in yellow we told you, yes, you need to know this.
AMIT JOSHI: And if you want to know how to know this, go to this textbook. And the textbooks were at that point online, but had just come on so. And so those links were not particularly well-established. This is what SCORE looks like now. It's, first of all, mobile optimized, as you can see. We have a really robust search engine. It's built for point of care use. So imagine a trainee is going to look at a patient who has acidosis in the ICU.
AMIT JOSHI: They'll be pointed to a module. They'll go into this module. And now not only is the learning objectives clearly outlined, but it's all very much there now for a trainee to look at. And so completely different than what a module looked like in 2013. And furthermore, all of this curriculum has now been organized around a two-year cycle. It's two years in general surgery.
AMIT JOSHI: It's one year in some of the other specialties. And so each of these is a collection of modules. Within each module then are what I just showed you. And so this ensures that everyone gets through the entire waterfront of general surgery in a two-year curriculum. And again, in some of the other specialties, it's one year. So this is what a specific week might look like. So it's called Stomach Part 1 of 2.
AMIT JOSHI: So there's another week of stomach as well. It may be the week after. It may be further down in the year. And then each of these is a module. And again, I mentioned that they were organized by core and advanced. Core meaning you really need to know everything about that. Advanced meaning you just need to know introductory initial management for your board exams.
AMIT JOSHI: Self-assessment is a huge part of the way that trainees approach these modules. We're really always trying to direct them to the modules, but many of them enter the website or the portal through self-assessment. They just take hundreds, hundreds, and thousands of self-assessment questions to try and learn the content. So we've spent a lot of time trying to make sure our self-assessment is good.
AMIT JOSHI: There are some comparative analytics so you can compare your program to other programs to get a sense of how you're performing and how your trainees are performing. To Laura's point, we've really tried to expand multimedia offerings on score. We have a really nice collaboration with Emory University and their illustration division there, and they produce just these beautiful images for us.
AMIT JOSHI: You'll see they're sort of consistent. The colors are consistent. We've worked it out now that every single label in here is part of the search database. So if you're studying for it, this is a thyroid. If you're studying for your thyroidectomy for tomorrow and you're trying to figure out where is that middle thyroid vein, you can plug that in and it'll point to this image. I always enjoy showing this slide.
AMIT JOSHI: This is really the proof of concept, and it came almost 15 years after the formation of SCORE. But one of the points of SCORE was to provide equivalent resources across the spectrum-- small programs, large programs, university, non-university, rural, urban. And this study in 2019 proved that, that whether you were a university program, an independent program, and we have a geographic disparity slide here as well, the usage was the same.
AMIT JOSHI: And so that's a really important thing. So you may have the world's experts on surgical disease, or you may just have general surgeons who are practicing, taking care of patients every day. But the fact that you subscribed to SCORE meant that you had equivalent access to resources that were really peer-reviewed and world-class. About two years ago, through Silverchair's help, we were able to develop this thing called role-based viewing.
AMIT JOSHI: So I had mentioned that we have a couple of different curricula now hosted on the portal. And so now when you go in, you'll see here my specialty is tagged to general surgery. But you could toggle it. Let's say you're an oral surgeon. I could go in and toggle it, and that will provide you with a completely different view of the content you see.
AMIT JOSHI: So the oral surgeons will see the oral and maxillofacial surgery content. The vascular surgeons will see theirs. It's worth just digressing for a second about oral surgery, the only specialty in this universe, at least of ours, that is not actually within the American Board of Surgery. So that's been a really, really neat collaboration for us. The oral surgery saw our portal and said, wow, I mean, you've put in 14 years of work here, probably $8 billion of investment, why are we going to recreate the wheel?
AMIT JOSHI: We'd rather just post our content on your portal. And that's turned out to be a really fruitful collaboration. And they just rolled out their first subscriptions in July, and so we're really looking forward to that content taking off. This is a response to our pandemic, when all of our in-person didactics disappeared by virtue of public health decrees in hospitals. And so we rolled out really what is an international set of webinars, and these are actually hosted by world experts.
AMIT JOSHI: And so this is just another way to appeal to different learning styles. So there are videos, there are procedural videos. These are narrated PowerPoints. There are images. There's obviously text and modules. And so SCORE continues to try to explore how to deliver content in different kinds of media. We have a pretty robust lineup of changes coming this year.
AMIT JOSHI: We're always trying to update. We're probably two months away from rolling out our most recent UI/UX update, and we're really excited about that. Our authoring tool is what many of you in the publishing or journal world will call your editorial manager, and we do a lot of work behind the scenes on that. We're trying to make it more modern and user-friendly. I had mentioned that even though we've always considered modules to be the central scaffolding of all our content, up until a month from now, our questions were never actually linked to modules.
AMIT JOSHI: So that's going to be a really important link, to make sure that every single self-assessment is linked to a module so that any time we change a module, we know exactly which questions are linked to that and vice versa. If a question gets challenged, we can go back to the module and say, well, was the module accurate, was it not accurate? So that crosslinking will be really helpful. We're really trying to think hard about how to use analytics, not just across the site, but actually within programs through a program director dashboard, to really drive our content change.
AMIT JOSHI: We've historically relied on focus groups and live webinars to kind of see how trainees navigate through the site. But we're interested in harnessing the power of analytics to try to say, OK, this is what people are using, this is not what people are using. This is what we want people to use, but let's use the user analytics to drive people to that. And finally, hopefully for July 23, we're finally going to pull the trigger and deploy a mobile app.
AMIT JOSHI: We're really excited about how that will drive more usage to the site, and hopefully, actually allow SCORE to drive content back to users through notifications and through really customized views for users. So I think that's the end of my slides. I'm really looking forward to the discussion, and thank you very much.
JAKE ZARNEGAR: Great. Thank you, Dr. Joshi. That was fascinating. I have lots of questions for you, which I'm going to hold till later. I think one of the biggest points you made was how a kind of dispersed education system with a lot of different variety of, and probably quality levels, has been kind of nationalized and organized into more of a central system.
JAKE ZARNEGAR: And so I have lots of questions about that and its applicability to other disciplines as well. But I'm going to hold on that, and I'd like Amanda Fielding to join us. Now, McGraw-Hill Professional publishes, in all disciplines, a ton of very high quality expert content. So I'd love to hear how you all are approaching next generation education.
AMANDA FIELDING: Thanks, Jake, and thanks to all the panelists and for inviting me to speak. I'm excited to geek out with everyone about things that I'm immersed in every single day and talking about these trends and how we can solution for them for our users. So I'm going to start by showing how it started. We launched AccessMedicine as our sort of mothership about 20 years ago now. And it really felt very revolutionary at the time.
AMANDA FIELDING: We were bringing our print textbooks to the digital space, and it was a really great response. And we built a very successful business around this. We took content and sold it twice. We had our print, and we had our amazing digital business that grew as a B2B, meeting the needs of librarians, really, primarily. They had this gorgeous digital bookshelf that they could offer their students.
AMANDA FIELDING: And we're very proud of it. And this is a screenshot from the Wayback Machine. I had to go on the Wayback Machine to find what AccessMedicine looked like 20 years ago. And I love that there's PDA content on there already. Does everyone remember PDAs? So how it's going now, we've modernized a lot of our textbooks. We've added new content types to really address the need of learners becoming more and more aware of what works for them.
AMANDA FIELDING: Some can't sit down and hunker down with a two-volume brick of text and make notes. They need multimodal learning, podcasts, videos. Definitely playing into what Laura mentioned before and having those assessments there to know where to go back in and remediate. So bringing all of that content to life on a platform that was originally ideated as a library has become-- I'm not going to say a challenge in a bad way, but that's what we're moving towards, right?
AMANDA FIELDING: We're trying to get this content to be perceived as more than just textbook content and actual dynamic content that can help the learner really hone in on what it is they're trying to learn. We've worked to streamline the presentation, modernize the look and feel of it, and indeed, modernize the way the content is being interacted with. One thing that we learned is that content provided in a textbook works in that format.
AMANDA FIELDING: But when you bring it online, that's not necessarily how an end user wants to interact with it. How do we rename or re-present the content in a way that is meant for digital dynamic interaction? So though things are going wonderfully right now with our existing products, we need to stay ahead of the expectations our users have and the trends that are starting to show us a bit of poor brand awareness.
AMANDA FIELDING: They know our book brand. They might not Access. There's a shift away from textbooks as the core learning, and to Laura's point again, moving towards assessment as the way to learn, or indeed, moving to point of care as an opportunity to learn. And so AccessMedicine is driven primarily towards medical students, but we do have products that go into more of the specialized residents.
AMANDA FIELDING: And it's really the same thing, point of care is taking over where learners are learning. It is no longer necessarily back at home with a textbook and a notebook and highlighter or in the library. It's happening on the go. And so that really reinforces our need to represent our content in a way that's accessible when and where and how the end user or a learner wants to ingest and interact with the content.
AMANDA FIELDING: And with that, we're starting to see an expanding indirect market landscape where we're no longer necessarily just up against our fellow publishers, but we're competing with digital startups, like Osmosis or AMBOSS, other organizations that are very defined in what they do, and they're starting to take market share in that learning journey. And so we are no longer just looking at what other publishers are doing.
AMANDA FIELDING: We have to look beyond. We have to look at Google. We have to look at Apple and where their tech trends are going. So how are we getting there? We're starting to shift our thinking into more user- and customer-driven ideation and solutioning, which seems like, of course, that's what we should be doing. But back in the day, we were publishers.
AMANDA FIELDING: We said, OK, we're going to make this book, and we're going to deliver it. So we're taking that same mentality and applying it now to digital format. So we're leading with, what is it that our customers need? How do our users choose to interact? And in the case of B2B, they're not always the same. And that is kind of a gap in the chasm we have to navigate. But really making sure that we are creating products that are in line with what a purchaser feels compelled to buy and what an end user feels compelled to use.
AMANDA FIELDING: And in that same vein, using data as a primary point of information, we, to date, had used data in the sense of how our resources were performing, what book is making the most traffic on a site or something. But that doesn't really tell the story of how our product is being interacted with. So going more deeply into those analytics and understanding the behaviors of our end users, who are end users are, and trying to get a story to tell.
AMANDA FIELDING: And does that data support the success of the product at the future of the product? Thirdly, we're looking at content a little bit differently. We're really looking to acquire, design, develop, and display content differently than we had ever thought about before. Again, we're a legacy publisher. We knew how to make really great content, put it between two covers, and sell it off.
AMANDA FIELDING: Now we're talking about content outside of those book covers. What is the best way to display this? And one thing we're really looking forward to doing on Access is connecting that content in a bit more of a meaningful way, leveraging related content widgets. Trying to tag our content in such a way where if you're in here, we can-- I like to call it rabbit holing, but we'll bring you down the rabbit hole, kind of that Wikipedia effect, where you just keep clicking and you want to start spiraling into the depths of Access to really help reinforce your mastery.
AMANDA FIELDING: And that comes not just from technological advances, but also in the way that we structure our content, the way we acquire it, develop it, label it, tag it. So there's a lot that goes on behind the scenes before you even get to that front-end interaction that we're really looking to invest in. And finally, though not least, yeah, the investment in technology, I mean, again, content was king and is king, really, in our business.
AMANDA FIELDING: We know that the contents are the goods, but we have to be investing in the methods to deliver it. And one thing we're looking to do is bring that content mobile. We've had amazing response to our responsive design, but we need to personalize that experience. So one thing we're looking to do is leverage and really hype up one of the features on AccessMedicine called MyAccess, where an end user can personalize their experience and bring their preferred content, interact with others perhaps in their class, share content, and create a community around our content.
AMANDA FIELDING: And that all happens with technology. And so leveraging some of those widgets, looking for partners to deliver mobiley, that is our biggest push in the coming years. And we're excited to see that next iteration of McGraw-Hill in the digital space. Thank you. I told you I'd geek out.
JAKE ZARNEGAR: Great. Now, I assume that's the good type of spiraling, right?
AMANDA FIELDING: Yes. Not like the bad kind, no.
JAKE ZARNEGAR: When I think of spiraling down, that doesn't usually end up well for that person.
AMANDA FIELDING: It's a good spiral. It's a rabbit hole.
JAKE ZARNEGAR: All right, we'll call it the happy spiral.
AMANDA FIELDING: Yeah.
JAKE ZARNEGAR: OK. Well, thank you. Anna, you are batting cleanup today to talk about the American Academy of Orthopedic Surgeons' new ROCK product. Whereas the ABS has SCORE, you have Rock. You both have a very strong acronym game. But I'd love to hear about your product, and please take it away.
ANNA SALT TROISE: Awesome. Thank you so much. Similarly to others, I was asked to discuss our origin story. And I think a lot of the themes that I'm going to talk about today are really similar to those of the other talks, particularly Dr. Joshi's. You'll see imitation is the highest form of flattery. So in 2017, our board of directors approved a new education strategy. And in it, they took a different approach to engaging stakeholders.
ANNA SALT TROISE: Education is not about product formats. It's not about access. It's about providing personalized needs-based content to customers and stakeholders when and how they want or need it. It's about building digital relationships based on frequency, emotion, and convenience, the idea being to optimize experiences with customers to meet their needs. The overall strategy was really clear-- we needed to win in specific segments.
ANNA SALT TROISE: And the best part was that the overall organization was absolutely committed to helping our team be able to do this. So at the same exact time, just to make it a little more complicated, the Academy developed a new five-year overall strategic plan and a new set of core values. So anything that we do within the educational team, but also within the rest of the organization, always has to map to the overall strategic plan and to our values.
ANNA SALT TROISE: For those of you who are in associations, you'll probably know the phrase "we do this because it adds member value." At the Academy, we really consider the phrase "member value" a four-letter word. It generally means that no one can figure out how to justify an action, a program, or a service, so they couch it in a really vague sentence. To us, value means reach, it means net income, or it means strategic alignment.
ANNA SALT TROISE: So what does this mean specifically when we think about product development? In listening to our customers, we kept hearing our residents say that their main goal was to perform well on the orthopedic and training exam, ultimately so that they can pass the Part I Exam and become a certified orthopedic surgeon. We heard them say over and over again that they have no time and that they're really stressed out.
ANNA SALT TROISE: We heard program directors say of their residents that they spend themselves a huge amount of time developing a five-year curriculum. But every single academic center, as Dr. Joshi said earlier, has a different curriculum and handles their education differently. There's a limited ability to test a resident and compare it to other centers across the country. There's no clear ability to identify an area of strength or weakness.
ANNA SALT TROISE: And there's no way to support a struggling learner. We also heard that there are a couple of other competitive products in the market to help residents with their educational journey, but nothing that really mapped to the orthopedic and training exam and nothing that mapped to the Part I Board Exam. So ultimately, as you've heard over and over again, there was no structured, standardized way to teach residents.
ANNA SALT TROISE: So this is a really big problem. And the Academy decided that we could actually help solve this. We've got a ton going for us. We have a really, really passionate, dedicated group of volunteer surgeons. We develop and own and administer the orthopedic and training exam. We have a really great relationship with the ABOS, and we have a ton of content.
ANNA SALT TROISE: So we played the what-if game. So what if we could link the In-Training Exam to anticipate how a resident will perform on the Part I exam? What if we could build out a content set to help programs standardize what they teach? What if that content was mapped to the In-Training Exam? What if we could target that content so the resident learns exactly what they need to know so that they save their time?
ANNA SALT TROISE: What if we could tell the program director how the resident performed who was struggling, what areas were a strength or a weakness, and how and what they can improve on? And what if we could do all of this by repurposing our own assets? So we built a business plan, we pitched it to the board, and we got a $5 million investment. We were really, really excited.
ANNA SALT TROISE: But we were incredibly overwhelmed because how on Earth were we going to be able to do this? So from the tech side, we looked at the real classic build, partner by utilize the current platforms to see what would be the best thing to do something on this scale. Our technology needs were really clear. We wanted a company that had a strong medical and education knowledge and leadership, experience in a similar market, had standardized tool sets, workflow integrations, could work with an aggressive timeline, be flexible, be aligned with our values, and share a similar mindset around overall product development.
ANNA SALT TROISE: So we conducted an RFP. And after evaluation, we chose Silverchair. So with that, we've invited authors. We've developed our workflow and our processes. We've developed our go-to-market strategy, our pricing strategy. We've onboarded authors, blah, blah, blah, blah, and we did the build. So the ROCK, the Resident Orthopedic Core Knowledge, is a massive product.
ANNA SALT TROISE: It's got 552 chapters, 11 specialties, 22 subspecialty editors, 90 section editors, over 500 authors, over 1,000 full-text journal articles and book chapters, over 500 videos, and over 3,000 examination questions. Each chapter itself has a mandatory element and an optional element. Each chapter roughly takes a resident about an hour to complete.
ANNA SALT TROISE: And if they want to dig deeper, they can, and they can spend countless hours doing that. We're currently actually live, which is super exciting. We have two specialties that are live on the platform, and we're continuing to roll it out throughout the course of the year. We're providing complimentary access for 220 US programs for the July '22 through '23 academic year. And then we do plan to charge for it afterwards.
ANNA SALT TROISE: So ultimately, going back to our original strategy, we believe that the ROCK will help us own this market through investing in an ecosystem that helps the Academy win and protect market share. Our products span across the board. ROCK is a key piece of this. Our video platform, Orthopedic Video Theater, or OVT, is another piece.
ANNA SALT TROISE: Our In-Training Exam, our testing product, ResStudy, are all powered ultimately by our analytics engines, which helps struggling learners evaluate their residence progress through all of these items in all five years of their residency. What's really cool is we're starting to think about what's next and how do we use emerging technology to be able to really invest even further in this.
ANNA SALT TROISE: So how can we layer in skills training using AR, VR, et cetera, to enhance their training and assessment even further? So I think with that, I'm going to stop sharing my screen, and I'm going to pass it back to you, Jake.
JAKE ZARNEGAR: OK. Well, thank you all for those products origin and also keeping us up-to-date on the progress on those products. I do have a series of questions, some of which I had pre-prepared. But then hearing you all talk, I just had to add some new ones, which I'm going to jump into right now. So I'm going to open this up for all of the panelists to answer these questions.
JAKE ZARNEGAR: Also, if anyone would like to ask a question, we have a Q&A feature in this webinar. So I saw one question was already asked and answered. But if you'd like to ask a question or put one there for me to ask live, please do. We've got about 20 minutes to ask questions. So I'm going to kick this off with probably the question that most folks are asking, which is, do all disciplines have these kind of national curricula or these top-down curricula, like surgery potentially with ROCK?
JAKE ZARNEGAR: Or did you have to develop these? Or are there folks building these curriculum that you could map an education product to? Maybe I'll start with you, Dr. Joshi.
AMIT JOSHI: OK, yeah. I think almost every specialty has a defined curriculum now, but they run the spectrum of how well developed they are. So I showed you that 2013 module, for example. I think there are many curricula out there that are not even up to that. They basically say "anesthesia, moderate sedation, you need to know that." But that doesn't provide you with any more resources to actually dig in and do that.
AMIT JOSHI: And then it goes to things like ROCK, which has thousands and thousands of resources, videos, and self-assessment questions. So I think there are many stages of evolution.
LAURA RICCI: Yeah, I can add on to this too.
JAKE ZARNEGAR: Yeah, Laura.
LAURA RICCI: And I'm actually glad you asked this question because one thing I didn't have time to touch on in my talk but that I think is important to know is, when I mentioned the ABMS and the federation of medical boards that set the standards for what recertification and so on looks like, that is the standard that all boards are supposed to follow. But each board is able to implement its own version of that within those standards. So what you see is actually, from specialty to specialty, what that recertification exam looks like is very different.
LAURA RICCI: And boards focus on different kinds of education. Some recertify are based on knowledge of certain journal articles. Others have more of a walking around knowledge kind of thing. So it really does vary based on discipline what learners are expected to know as they're certified and as they're recertified too. So that, I think, is an important nuance when you think about medical education because there really is no one size fits all, even at the level of those overarching standards.
JAKE ZARNEGAR: So you really got to look at it discipline by discipline and see what's there, what the level of their curriculum, how well defined it is, what their board certification process looks like and their testing looks like. Right. Great.
AMIT JOSHI: And I'll just add, there are so many nuances based on the different organizations and what they do. So the organizational structure in orthopedics looks different than the organizational structure in surgery and pediatrics and blah, blah, blah. So that becomes really important because each organization, to Anna's point, is going to do that analysis and say, is this really in our wheelhouse, or is it not? Are we going to put this giant investment if it doesn't align with our strategic plan?
JAKE ZARNEGAR: Great. Well, another phrase that jumped out at me that I heard from both Amanda and Anna was assessment first, organizing the learning around the assessment. Anna, you had noted that you were thinking about the exam and mapping to the exam. Amanda, you used assessment first and how folks want to begin a topic. Could you all talk about that approach and how your products are kind of mapped to the assessment or supporting the assessment, or even putting the assessment up front to identify gaps and strengths?
AMANDA FIELDING: Yeah, I can jump in. And, I mean, McGraw-Hill is not a test prep company. We have test prep products, and we have test prep content. I think we know that the importance of these milestones in medical school-- the licensing exam, the USMLE-- is key. And so we want to make sure that while we are not a test prep tool on Access, we do offer opportunities to assess your knowledge along the way and then remediate on the same site with our deeper dive content.
AMANDA FIELDING: So it's twofold, not just on the content side, but also on the behavioral side. That is how our learners are learning. To the point-- I'm sorry I can't remember whose presentation it was, I think it was yours, Anna-- that they don't have time. They don't have time to sit down and hunker down and read a book. So they need to bang through this Q&A, figure out where they're not doing so great, and that's where they're going to invest their time in.
AMANDA FIELDING: And so we need to be able to facilitate that and provide the content that they need to fill those gaps, reassess and make sure they're good, and then move on to the next product. And that's where innovation is coming in at McGraw-Hill. How do we meet that need, whether within Access or other products? But we try to not consider our Access at least a test prep tool, but we do have that core foundational knowledge to help reinforce the material-- the knowledge you'll need to know.
AMANDA FIELDING: Sorry, we have the core foundational material to reinforce the knowledge you will need for these milestone exams, whether they're boards or otherwise.
JAKE ZARNEGAR: Great.
ANNA SALT TROISE: So a couple of things from our perspective. I'd say, first of all, orthopedic surgeons are really, really good test takers. And that is an important piece because they quiz and they can test-- I would imagine general surgery is the same-- but it doesn't necessarily mean they're learning. And so if you look at the Part I Board Exam, there is a general blueprint of what they're going to need to cover.
ANNA SALT TROISE: We've taken our In-Training Exam and sort of mapped that overall blueprint so it's similar from that standpoint. The ROCK then maps to the In-Training Exam. So ultimately, as you progress, you'll be able to understand how you're going to perform on these various tests. Within the ROCK, too, we have built out a pretest and a post-test.
ANNA SALT TROISE: So you should be able to then see at the beginning, presumably, you're not going to perform very well. You go through the curriculum, you're able to take the test. You're able to improve. And then ultimately, you're In-Training score should be higher. At the end of all of this, we should be able to do a study on that to statistically show that this all maps out and that your time should be spent here.
JAKE ZARNEGAR: Great. Great. Laura, I think you introduced this, but I'd love to hear everyone's thoughts on this. We're talking about medical school here, we're talking about residency. But what we're seeing is that actual professional practice now is also turning into a time of continuous certification, continuous assessment, the introduction of competencies, as well as I guess the new thing that I learned about during this process was Entrustable Professional Activities.
JAKE ZARNEGAR: Dr. Joshi, I think you told me about those. But this way of constantly assessing someone during their practice, which, in the past, had been kind of you get board certified and you could go do anything. But now it does seem like there's a kind of more continuous top-down requirement and evaluation of folks. So I don't know, Dr. Joshi, if you could say how that is working in your discipline.
AMIT JOSHI: Yeah, so EPAs, Entrustable Professional Activities, are becoming very, very important across the ABMS universe. So again, different boards are in different levels, but surgery is actually probably at the forefront now of deploying EPAs for 2023. EPAs are, to what you said, Jake, they point to ongoing micro assessments. As opposed to the other extreme, every 10 years you take the giant exam, this is, OK, I'm assessing you now, ooh, and I'm assessing you now, and I'm assessing you now.
AMIT JOSHI: Many, many low-stakes assessments, which, in aggregate, provide a constellation of durable data versus a singular every 10 year high-stakes assessment. And so we think that-- and there's data to show this very well-- that for good learning theory, the micro assessments are going to be much more powerful for both validity, but also more importantly, driving change in practice.
AMIT JOSHI: And that's what's going to be really important. We've talked a lot here about trainees. We really haven't talked-- Amanda talked probably the most about people in practice. But neither the ROCK curriculum nor the SCORE curriculum is oriented actually towards people in practice yet. But that's going to become a really important thing as EPAs expand beyond just trainees. We actually want to drive practice and drive change in practice for people who are 30 years out of training, not just six months out of training.
JAKE ZARNEGAR: Yes, that does seem like the direction of more oversight and kind of micro assessment in the workplace. Exactly. Great. Anyone else comment on that type of continuous certification?
AMIT JOSHI: I was going to make one more point, which is that we've talked-- Anna and I had very similar points, which was it's at least starting off as a rigid system. We have this curriculum, we have the assessments, Part I of the exams that people need to study for. What we found actually even in the last couple of months is that we can use SCORE to drive important initiatives that are not even on examinations yet. And so I'll give a great example of that.
AMIT JOSHI: Just this morning, I was reviewing three new modules that we have on the SCORE curriculum dealing with DEI issues in surgical training. And so I reviewed an allyship and bystander module that's going to go into SCORE. Are we testing on that topic at the board? We are not. But is it a really great way to deploy that content that's timely, that's needed, that's important to surgical trainees?
AMIT JOSHI: Absolutely. So I think we're going to see more and more of that as time goes on that it's not necessarily all devoted to meeting that final assessment, it's actually a way to change maybe not even core surgical things within practice.
ANNA SALT TROISE: 100%. We needed to start somewhere, and that's sort of the core surgical practice piece. But a lot of the items that our members are really interested in, too, is more on the practice management side, so the business of orthopedic surgery-- contracting, leasing, hiring, firing, things like that-- because so many of them are still in private practice. So it's another good tool to be able to get education information out to them.
JAKE ZARNEGAR: It does seem like a big opportunity. You're in training for a certain amount of time, and hopefully you're in practice much longer than you were in training. So if you're going to need a constant flow of education and assessment during that period, then especially organizations who are professional associations in those disciplines would have a great opportunity to, again, save them the time as well as organize the things that they need to know during their practice, not just in training.
JAKE ZARNEGAR: I have a question for Laura. Maybe an update if you found any on the trend of, for particular, for CME activities. In the near past, physicians have a lot of leeway in selecting their own CME activities, the format, but also the topics. Certainly, there's a minimum amount of required training that they would need to submit to their licensure boards.
JAKE ZARNEGAR: But from there, there was a lot of individual control of what the components of that training was. Are you seeing more either employer- or system-based control over what people need, what topics that people need to cover in their CME training? Or is it still a physician-decided choice there?
LAURA RICCI: Yeah, that's a really good question. And this actually sort of ties into something I was thinking when we were talking about the last question and regular assessment and really driving education off of data. So I think where we are right now is it's still pretty early days in terms of how health systems are getting involved in education. We are seeing that more physicians are employed by health systems and not private practice.
LAURA RICCI: And so that's really driving more education-driven choice of what gets taught or what physicians are learning. Physicians, we still see that they do have a lot of discretion. They can make their own choices about what CME activities that they are pursuing. But at the health systems level, there's more opportunity to get data about actual health outcomes. So not just the assessment and at the individual physician level and what they need to know to be recertified, but also there's an opportunity which is still nascent but I think is interesting to think about, where health outcomes can actually be improved.
LAURA RICCI: So are there areas where there's a particularly risky procedure or something that a health system wants to improve its outcomes? It can tailor education for its employers and encourage them to learn more of the modern techniques and improve the ways that they learn on that basis. Now, this is, as you can imagine, a tough thing to do in practice. Health systems data is pretty all over the place.
LAURA RICCI: But there are smart people working on this problem. So I think if we could project forward a little bit, that kind of focus on health outcomes could certainly be an important part of where employers become more involved in driving education for physicians.
JAKE ZARNEGAR: Dr. Joshi.
AMIT JOSHI: I was going to just add on to Laura's point, which is two ways. One is, there are increasing state and hospital mandates about different types of topics. And so two examples that come off the top of my head, for New Jersey, for example, I need two hours of child abuse training every year, and I need two hours of appropriate opiate prescription practices. If I'm a trauma surgeon, I need so many hours of trauma. If I am doing something else, I'll need so many of that.
AMIT JOSHI: So those are increasing kind of niches that you have to meet. And the ABMS has just recently in the last year approved a new set of standards that are mandating tracking of quality and outcomes within your practice. And so as time goes on, we, as content providers, are going to need to provide that, for those of us who provide content for diplomates are going to need to provide that kind of tracking mechanism, but also education about how to do quality.
AMIT JOSHI: And so that's going to be something coming as well.
LAURA RICCI: I--
AMANDA FIELDING: I want to piggyback a little-- oh, sorry, go ahead.
LAURA RICCI: No, go ahead, Amanda.
AMANDA FIELDING: I just find it interesting that we're sort of in the same-- even though we're content providers, there's more being asked of us as these content providers. And it goes into analytics and data, and we're not set up yet to be that engine. And so that's one area in technology where we're pushing a lot. We need to have those metrics and be able to tell the story to our customers just as much as we need that information ourselves.
AMANDA FIELDING: And so providing those analytics, that data, those stories as part of our "service" as a content provider, if you will, is a new space for a legacy organization who is used to just a pretty black and white set-up. You create content, you deliver it, the end. And now with this digital age, there's more being asked of that. And I find it really interesting. And I'm sure a lot of people are up against this, so it'll be interesting to see how it's handled.
JAKE ZARNEGAR: And you mean as far as reporting on activities, strengths, gaps?
AMANDA FIELDING: Yeah, I mean, on an individualized level, going back to assessment, what is assessment without analytics? It's nothing. And even for our larger institutions, it goes beyond what resource is being trafficked the most. It goes into, well, how many people are returning? What is the story? Who is using this site? Is it really my budget that should be paying this?
AMANDA FIELDING: Should it be another budget? There's a lot of questions. And now that our librarians are getting more tech-savvy and very into analytics and data, we have to be ready and able to supply them with that, for their edification and for our own, for product development, for sales, whatever it is. And it's a gap right now, and we're trying to close it. Because it's a service really, and it's not one that we, as content providers, have ever really explored and are now feeling the pressure to actually step it up.
JAKE ZARNEGAR: Right. Right I mean, I definitely saw in Anna's presentation that you do consider that 360 data from their interactions with your products as being a key data point.
ANNA SALT TROISE: We think it's going to be the main differentiator for us, and it's the only way we'll be able to compete in the marketplace. Because, I mean, think about everything we're talking about, they're all competition regardless in the education space. The piece, too, on top of that is training staff and customers on what to do with it. So you can collect everything you want-- and, Jake, you and I talked a lot about this over the years-- but making sure that people know what to do with it and how to be able to tell the story and get the insights out of it.
ANNA SALT TROISE: Because otherwise, it's pointless.
AMANDA FIELDING: It's just data dump. It's just numbers. Yeah, absolutely.
JAKE ZARNEGAR: Well, on that note, I will have to call our time here today. So, Amanda, you got us on spiraling and data dump there, so.
AMANDA FIELDING: It's a new vocabulary for everyone.
JAKE ZARNEGAR: Appreciate it. Well, first of all, thank you to the panelists. This, again, has been a fantastic content, fantastic information. I learned something, and I talk with you all regularly. But getting you all together to talk through these topics, the idea of producing more data out of your education products into these assessment systems is really fascinating. Now, for the folks who have joined us, when you leave the webinar, you are going to get a one-question survey about how you enjoyed the event.
JAKE ZARNEGAR: I'd appreciate you if you take two clicks to complete that. And for everyone to note, our next event in this series will take place on September 7, and we'll cover data trends. So thank you all for the lead-in to that. That was very well done. And the title is "The Alchemy of Turning Your Disparate Data Into Gold." OK. So we hope to see you there.
JAKE ZARNEGAR: Thank you again to the panelists. This was unbelievably useful content. And thank you to the attendees for asking questions as well. So this wraps us for today. Thank you very much. STEPHANIE LOVEGROVE
HANSEN: Thanks, everyone.
00:
Segment:0 .
STEPHANIE LOVEGROVE
HANSEN: Welcome.
HANSEN: My name is Stephanie Lovegrove Hansen. I am the director of marketing at Silverchair. And thank you so much for joining us today. This is the second in the 2022 Platform Strategies webinar series. We're really looking forward to today's discussion on education technology. But before we get started, I'll just cover a few logistics for us. As I mentioned, this is the second event in this year's webinar series.
HANSEN: And we're hoping that when you registered for this event, you also registered for the other events we're hosting. But if you didn't, it's not too late. So you can follow the same registration link to save the date for those events. You can also visit our website for the recording from our kickoff event on tech trends, which we held in May. This series is a set of free virtual events designed for thought leaders to discuss their platform, data, and technology strategies with colleagues.
HANSEN: So we hope that in addition to hearing from our thoughtful speakers today, that you'll continue in the tradition of the event and also engage in the discussion via the chat and/or the Q&A features of the webinar. If you like the topic enough that you want to continue the conversation further, this year we've introduced some small group roundtables in the month following the event where attendees can join in live in more of a meeting format to continue discussions on the topic.
HANSEN: So we just held our first one for the Tech Trends webinar and had a really engaging discussion. So just email strategies@silverchair.com if you'd like to receive an invitation for that. There will also be a link for that in the follow-up email that you'll get from today's event. This event is being recorded, and a copy of the recording will be made available on our website. And finally, at the end of the event, you will get a survey requesting your reading of this event, which just helps us with future planning.
HANSEN: And we really appreciate your feedback. With that, I will hand it over to today's moderator, Jake Zarnegar, Silverchair's Chief Business Development Officer. Jake.
JAKE ZARNEGAR: OK, thank you, Stephanie. I'm glad you didn't have to go through a list of side effects that this webinar might cause. So I'm really excited about today and this hour of content that we have for you all today. We're going to be talking about the future of education, edtech. We're going to be talking about it, in this case, from the perspective of medical and health content. But a lot of the concepts you hear today will be applicable to many aspects of digital and online education as well.
JAKE ZARNEGAR: And we have just a great panel today. We have joining us Laura Ricci of Clarke & Esposito. We have Dr. Amit Joshi of the American Board of Surgery, Amanda Fielding of McGraw-Hill Professional, and Anna Salt Troise of the American Academy of Orthopedic Surgeons. Now, if you'll notice there, we had someone representing a board who does certification, a commercial publisher, a professional society, and an industry analyst there as well.
JAKE ZARNEGAR: So it's a great kind of 360 survey of the participants in education. And Dr. Joshi, of course, also a participant as a taker of this education as well as a provider. So he can also provide that perspective for us. So to get us started today, Laura Ricci is going to share some trends and market analysis that Clarke & Esposito has done recently. Silverchair actually commissioned research from Clarke & Esposito recently, and it was such an interesting outcome that we thought it would be great to share with all of you here.
JAKE ZARNEGAR: So, Laura, I'm going to turn it over to you to kick us off.
LAURA RICCI: Great. Thank you, Jake. Let me share my screen. All right. So I'm really excited to be part of this fabulous panel, and I'm looking forward to hearing from the other speakers. But in thinking about this topic, I wanted to use my time to set the stage a little bit. As Jake said, I work for Clarke & Esposito, which is a consultancy focused on professional publishing in education.
LAURA RICCI: At our firm, we work with a lot of societies and publishers, typically those who serve active professionals to help them thrive in their roles. So that's where our view of the industry really tends to focus. And I think a lot of the things happening in professional education space really mirror other trends throughout other levels of education too. So today, I want to focus on the three key trends that we're seeing in professional medical education.
LAURA RICCI: And they are-- if I can get the screen to advance. Oh, dear. Already the PowerPoint is doing the thing. Back. OK. So those three key trends I mentioned. So first, the delivery of professional education shifting from in-person to online.
LAURA RICCI: Second, the closer interplay developing between assessment and education. And lastly, how those two trends together allow for more data-driven approaches to education, which unlocks new capabilities that improve engagement and outcomes. And I'm going to take these one at a time. So let's start with the move of education online. Now, I'm sure this is not a surprise to anyone.
LAURA RICCI: We've all lived through the COVID pandemic. But let's put some data behind it. So this data that we're looking at here comes from the Accreditation Council for Continuing Medical Education, or ACCME. And it shows the number of CME activities offered in 2019 and 2020. Now, there are lots of different kinds of medical education, and accredited CME is just one piece.
LAURA RICCI: But one of the reasons why we like ACCME data is because it reflects all the different kinds of education. So the key things to see here is that COVID led to a decrease in in-person education, these in-person courses, conferences, et cetera. And there was also, of course, an increase in virtual courses and online enduring-- meaning on demand-- materials. So in 2020, online enduring materials were the most common CME offering for the first time.
LAURA RICCI: Now, we are waiting to see whether the data shows these trends continue into 2021. But it's really likely anecdotally that some of these trends are going to endure. Now, ACCME data is also great because it shows how education is not just coming from publishers. There are lots of different sources of education. Now, publishers and societies are the first and third leading sources of CME, but physicians are also getting education from work, from their health care system employer or their affiliated school of medicine.
LAURA RICCI: But if you think about that shift towards online learning, you can see how publishers and societies are really the leading source of those activities, from journals CME, internet enduring materials, and live internet activities. So these providers have a unique opportunity to really lead change in education. So that's trend one. The second key trend I'll talk about is not just how physicians are receiving education, it's also how they're being assessed and certified.
LAURA RICCI: So physicians are typically certified by their specialties member board. And the American Board of Medical Specialties, or ABMS, is a federation of 24 medical boards which sets standards about physician assessment and certification. Now, in October 2021, ABMS approved new standards for continuing certification among its member boards. Now, two of the major changes we'll talk about in those standards are that instead of offering a single cumulative point in time assessment every 10 years, now they're encouraging boards to assess longitudinally, meaning at regular intervals, say every six months or so, with recertification based on achieving a minimum performance threshold over that time.
LAURA RICCI: And the standards also require boards to offer formative assessment, meaning instead of just providing a snapshot of did you pass or fail, boards will give feedback about gaps in knowledge for that physician. So that physician can then seek out more information and address that gap. Now, where this is relevant is that publishers and societies have long been supportive of physicians studying for board certification.
LAURA RICCI: And what we're seeing now is there are opportunities to provide better education because of this new focus on regular formative assessment. It creates this opportunity for this virtual cycle where a learner can take an assessment, identify gaps in knowledge and skills, and then focus future learning activities on remediating that specific gap. And this cycle is the end goal of that move to continuous assessment.
LAURA RICCI: But it doesn't have to happen in just a formal board assessment, either. So with the move to online, any module can incorporate mini assessments to gauge understanding and whether or not the physician grasped everything, all the important concepts. So what this really is is data-driven learning. And if you combine these two trends, the move online and the increasing information about knowledge gaps, it unlocks all these opportunities to improve the way that education is being developed and delivered.
LAURA RICCI: So when we ask publishers and societies what they look forward to in medical education, this is what we hear. We hear it's increasingly about getting the right information to the learner at the right time, and this can be through providing better discovery and searchability of education content. It could be personalizing recommendations, so suggesting what to study next based on what they already know about that physician.
LAURA RICCI: There's also more information back about how each learner is performing. Are they passing? How do they compare to peers? And data can help to guide the curriculum itself. Supports capture what concepts are most challenging to test takers, and they provide this data to the community so societies and other publishers can develop a curriculum to target those gaps.
LAURA RICCI: So this is not just for individual learners, this is for the specialty as a whole, which is pretty cool. And there's less focus on a particular product, but instead an objective. So content in different formats which relate to the same topic can be presented so that there's more than one way to teach a concept and different learners can gravitate towards different learning styles. So you put this together, and there's an opportunity for online and data-driven education to be more powerful and impactful.
LAURA RICCI: You can give more feedback to physicians. They don't have to wait 10 years to see whether or not they're still in step with their field. Societies and publishers have way more visibility into whether their education modules are teaching effectively. Different concepts across the field can be identified so that educators can respond right away.
LAURA RICCI: And education providers have a lot more avenues to reach learners and more creative ways to keep them engaged as they learn. So that has been a really quick overview. There's a lot more I could say. But personally, I find this really inspiring, and I'm really looking forward to the discussion with the other panelists. But for now, I will turn it back to Jake.
LAURA RICCI: Thank you.
JAKE ZARNEGAR: Great. Thank you, Laura. That was a great setup there of the kind of trends and changes. I, for one, am very interested in the assessing the impact of the education and having more of an immediate feedback loop of whether folks are learning. The internet gives us a great opportunity to do that rather than, say, send something out to someone and not really know whether they've learned the topics from that.
JAKE ZARNEGAR: On that note, we're now going to look at three new and newly developed or newly updated products that address medical education for specific disciplines. And starting with Dr. Joshi, who is not just the editor-in-chief of SCORE, but also professor and associate dean at Cooper Medical School in New Jersey. And I believe you do a little surgery in your spare time also. So I'm going to turn it over to you.
AMIT JOSHI: You can find me every Thursday at a local operating room. Thanks, Jake. Thanks, Stephanie, for the invitation. It's always a pleasure to talk with our friends from across the Silverchair universe. And so I was asked to share a little bit of our origin story about SCORE, and so I'm going to go through that. SCORE was founded in the early 2000s, the Surgical Council On Resident Education, and formed by seven different surgical organizations.
AMIT JOSHI: And the running joke is, if you stop in your local OR lounge and you ask two surgeons for an opinion, you're going to get three different opinions. And so to get seven different organizations to agree to form SCORE was actually a fairly Herculean effort and has paid huge dividends over the last 15 years. But the SCORE consortium was founded in 2004. And the mission is worth repeating-- to improve the education of residents who are trainees in general surgery and related specialties through the development of a national curriculum.
AMIT JOSHI: And so when you read that mission, you realize that prior to this, there was no national curriculum, that you went through surgical residency five, six, seven years and you were sort of just told to know everything, to learn what you needed to learn, but there wasn't actually any prescribed body of knowledge that you needed to know. So just the formation of the curriculum by itself was fairly groundbreaking.
AMIT JOSHI: And it took a few years. So from 2004 to really 2008, the curriculum outline was defined. And that was basically to say, yes, you need to know this, no, you don't need to this, this is advanced, this is basic. Over the next couple of years, all of this curriculum was deployed on a web portal, which back then was a pretty groundbreaking thing. And then a subscription started. So 2008, the first group of beta test subscriptions were 33.
AMIT JOSHI: And then over the next 10 years, it steadily increased and basically leveled out around 2019. You'll see that number of 274 subscribing programs. There really are only about 280 subscribing programs, at least back in 2019. Currently, there are probably something like 285, 300 programs in general surgery. But then if you look at the bottom left, these are 2022 numbers.
AMIT JOSHI: We've subsequently expanded the curricula into other fields. So it's not just general surgery, it's vascular surgery, pediatric surgery, surgical oncology, surgical critical care. And it's not just programs in the US, but international, Canadian-- I know that Canadians always laugh when we don't include them in international-- but programs really across the country and North America.
AMIT JOSHI: And so currently, we stand at around 11,000 trainees across the world who have core subscriptions. This is the curriculum outline. This is one small segment of it. This is anesthesia. And I always pick anesthesia because you wouldn't necessarily think that general surgeons need to know about anesthesia, but we do. And so by defining this and saying, yeah, you need to know this, it forced trainees from across the spectrum to start to learn that.
AMIT JOSHI: And then within anesthesia, these are the specific topics that you need to know. So in SCORE, the module is really the scaffolding around which everything else is built. And so I was go back, and this is what a module looked like in 2013. It was really just a Word document saying, you need to know this, but with really no embedded content. You'll see highlighted in yellow we told you, yes, you need to know this.
AMIT JOSHI: And if you want to know how to know this, go to this textbook. And the textbooks were at that point online, but had just come on so. And so those links were not particularly well-established. This is what SCORE looks like now. It's, first of all, mobile optimized, as you can see. We have a really robust search engine. It's built for point of care use. So imagine a trainee is going to look at a patient who has acidosis in the ICU.
AMIT JOSHI: They'll be pointed to a module. They'll go into this module. And now not only is the learning objectives clearly outlined, but it's all very much there now for a trainee to look at. And so completely different than what a module looked like in 2013. And furthermore, all of this curriculum has now been organized around a two-year cycle. It's two years in general surgery.
AMIT JOSHI: It's one year in some of the other specialties. And so each of these is a collection of modules. Within each module then are what I just showed you. And so this ensures that everyone gets through the entire waterfront of general surgery in a two-year curriculum. And again, in some of the other specialties, it's one year. So this is what a specific week might look like. So it's called Stomach Part 1 of 2.
AMIT JOSHI: So there's another week of stomach as well. It may be the week after. It may be further down in the year. And then each of these is a module. And again, I mentioned that they were organized by core and advanced. Core meaning you really need to know everything about that. Advanced meaning you just need to know introductory initial management for your board exams.
AMIT JOSHI: Self-assessment is a huge part of the way that trainees approach these modules. We're really always trying to direct them to the modules, but many of them enter the website or the portal through self-assessment. They just take hundreds, hundreds, and thousands of self-assessment questions to try and learn the content. So we've spent a lot of time trying to make sure our self-assessment is good.
AMIT JOSHI: There are some comparative analytics so you can compare your program to other programs to get a sense of how you're performing and how your trainees are performing. To Laura's point, we've really tried to expand multimedia offerings on score. We have a really nice collaboration with Emory University and their illustration division there, and they produce just these beautiful images for us.
AMIT JOSHI: You'll see they're sort of consistent. The colors are consistent. We've worked it out now that every single label in here is part of the search database. So if you're studying for it, this is a thyroid. If you're studying for your thyroidectomy for tomorrow and you're trying to figure out where is that middle thyroid vein, you can plug that in and it'll point to this image. I always enjoy showing this slide.
AMIT JOSHI: This is really the proof of concept, and it came almost 15 years after the formation of SCORE. But one of the points of SCORE was to provide equivalent resources across the spectrum-- small programs, large programs, university, non-university, rural, urban. And this study in 2019 proved that, that whether you were a university program, an independent program, and we have a geographic disparity slide here as well, the usage was the same.
AMIT JOSHI: And so that's a really important thing. So you may have the world's experts on surgical disease, or you may just have general surgeons who are practicing, taking care of patients every day. But the fact that you subscribed to SCORE meant that you had equivalent access to resources that were really peer-reviewed and world-class. About two years ago, through Silverchair's help, we were able to develop this thing called role-based viewing.
AMIT JOSHI: So I had mentioned that we have a couple of different curricula now hosted on the portal. And so now when you go in, you'll see here my specialty is tagged to general surgery. But you could toggle it. Let's say you're an oral surgeon. I could go in and toggle it, and that will provide you with a completely different view of the content you see.
AMIT JOSHI: So the oral surgeons will see the oral and maxillofacial surgery content. The vascular surgeons will see theirs. It's worth just digressing for a second about oral surgery, the only specialty in this universe, at least of ours, that is not actually within the American Board of Surgery. So that's been a really, really neat collaboration for us. The oral surgery saw our portal and said, wow, I mean, you've put in 14 years of work here, probably $8 billion of investment, why are we going to recreate the wheel?
AMIT JOSHI: We'd rather just post our content on your portal. And that's turned out to be a really fruitful collaboration. And they just rolled out their first subscriptions in July, and so we're really looking forward to that content taking off. This is a response to our pandemic, when all of our in-person didactics disappeared by virtue of public health decrees in hospitals. And so we rolled out really what is an international set of webinars, and these are actually hosted by world experts.
AMIT JOSHI: And so this is just another way to appeal to different learning styles. So there are videos, there are procedural videos. These are narrated PowerPoints. There are images. There's obviously text and modules. And so SCORE continues to try to explore how to deliver content in different kinds of media. We have a pretty robust lineup of changes coming this year.
AMIT JOSHI: We're always trying to update. We're probably two months away from rolling out our most recent UI/UX update, and we're really excited about that. Our authoring tool is what many of you in the publishing or journal world will call your editorial manager, and we do a lot of work behind the scenes on that. We're trying to make it more modern and user-friendly. I had mentioned that even though we've always considered modules to be the central scaffolding of all our content, up until a month from now, our questions were never actually linked to modules.
AMIT JOSHI: So that's going to be a really important link, to make sure that every single self-assessment is linked to a module so that any time we change a module, we know exactly which questions are linked to that and vice versa. If a question gets challenged, we can go back to the module and say, well, was the module accurate, was it not accurate? So that crosslinking will be really helpful. We're really trying to think hard about how to use analytics, not just across the site, but actually within programs through a program director dashboard, to really drive our content change.
AMIT JOSHI: We've historically relied on focus groups and live webinars to kind of see how trainees navigate through the site. But we're interested in harnessing the power of analytics to try to say, OK, this is what people are using, this is not what people are using. This is what we want people to use, but let's use the user analytics to drive people to that. And finally, hopefully for July 23, we're finally going to pull the trigger and deploy a mobile app.
AMIT JOSHI: We're really excited about how that will drive more usage to the site, and hopefully, actually allow SCORE to drive content back to users through notifications and through really customized views for users. So I think that's the end of my slides. I'm really looking forward to the discussion, and thank you very much.
JAKE ZARNEGAR: Great. Thank you, Dr. Joshi. That was fascinating. I have lots of questions for you, which I'm going to hold till later. I think one of the biggest points you made was how a kind of dispersed education system with a lot of different variety of, and probably quality levels, has been kind of nationalized and organized into more of a central system.
JAKE ZARNEGAR: And so I have lots of questions about that and its applicability to other disciplines as well. But I'm going to hold on that, and I'd like Amanda Fielding to join us. Now, McGraw-Hill Professional publishes, in all disciplines, a ton of very high quality expert content. So I'd love to hear how you all are approaching next generation education.
AMANDA FIELDING: Thanks, Jake, and thanks to all the panelists and for inviting me to speak. I'm excited to geek out with everyone about things that I'm immersed in every single day and talking about these trends and how we can solution for them for our users. So I'm going to start by showing how it started. We launched AccessMedicine as our sort of mothership about 20 years ago now. And it really felt very revolutionary at the time.
AMANDA FIELDING: We were bringing our print textbooks to the digital space, and it was a really great response. And we built a very successful business around this. We took content and sold it twice. We had our print, and we had our amazing digital business that grew as a B2B, meeting the needs of librarians, really, primarily. They had this gorgeous digital bookshelf that they could offer their students.
AMANDA FIELDING: And we're very proud of it. And this is a screenshot from the Wayback Machine. I had to go on the Wayback Machine to find what AccessMedicine looked like 20 years ago. And I love that there's PDA content on there already. Does everyone remember PDAs? So how it's going now, we've modernized a lot of our textbooks. We've added new content types to really address the need of learners becoming more and more aware of what works for them.
AMANDA FIELDING: Some can't sit down and hunker down with a two-volume brick of text and make notes. They need multimodal learning, podcasts, videos. Definitely playing into what Laura mentioned before and having those assessments there to know where to go back in and remediate. So bringing all of that content to life on a platform that was originally ideated as a library has become-- I'm not going to say a challenge in a bad way, but that's what we're moving towards, right?
AMANDA FIELDING: We're trying to get this content to be perceived as more than just textbook content and actual dynamic content that can help the learner really hone in on what it is they're trying to learn. We've worked to streamline the presentation, modernize the look and feel of it, and indeed, modernize the way the content is being interacted with. One thing that we learned is that content provided in a textbook works in that format.
AMANDA FIELDING: But when you bring it online, that's not necessarily how an end user wants to interact with it. How do we rename or re-present the content in a way that is meant for digital dynamic interaction? So though things are going wonderfully right now with our existing products, we need to stay ahead of the expectations our users have and the trends that are starting to show us a bit of poor brand awareness.
AMANDA FIELDING: They know our book brand. They might not Access. There's a shift away from textbooks as the core learning, and to Laura's point again, moving towards assessment as the way to learn, or indeed, moving to point of care as an opportunity to learn. And so AccessMedicine is driven primarily towards medical students, but we do have products that go into more of the specialized residents.
AMANDA FIELDING: And it's really the same thing, point of care is taking over where learners are learning. It is no longer necessarily back at home with a textbook and a notebook and highlighter or in the library. It's happening on the go. And so that really reinforces our need to represent our content in a way that's accessible when and where and how the end user or a learner wants to ingest and interact with the content.
AMANDA FIELDING: And with that, we're starting to see an expanding indirect market landscape where we're no longer necessarily just up against our fellow publishers, but we're competing with digital startups, like Osmosis or AMBOSS, other organizations that are very defined in what they do, and they're starting to take market share in that learning journey. And so we are no longer just looking at what other publishers are doing.
AMANDA FIELDING: We have to look beyond. We have to look at Google. We have to look at Apple and where their tech trends are going. So how are we getting there? We're starting to shift our thinking into more user- and customer-driven ideation and solutioning, which seems like, of course, that's what we should be doing. But back in the day, we were publishers.
AMANDA FIELDING: We said, OK, we're going to make this book, and we're going to deliver it. So we're taking that same mentality and applying it now to digital format. So we're leading with, what is it that our customers need? How do our users choose to interact? And in the case of B2B, they're not always the same. And that is kind of a gap in the chasm we have to navigate. But really making sure that we are creating products that are in line with what a purchaser feels compelled to buy and what an end user feels compelled to use.
AMANDA FIELDING: And in that same vein, using data as a primary point of information, we, to date, had used data in the sense of how our resources were performing, what book is making the most traffic on a site or something. But that doesn't really tell the story of how our product is being interacted with. So going more deeply into those analytics and understanding the behaviors of our end users, who are end users are, and trying to get a story to tell.
AMANDA FIELDING: And does that data support the success of the product at the future of the product? Thirdly, we're looking at content a little bit differently. We're really looking to acquire, design, develop, and display content differently than we had ever thought about before. Again, we're a legacy publisher. We knew how to make really great content, put it between two covers, and sell it off.
AMANDA FIELDING: Now we're talking about content outside of those book covers. What is the best way to display this? And one thing we're really looking forward to doing on Access is connecting that content in a bit more of a meaningful way, leveraging related content widgets. Trying to tag our content in such a way where if you're in here, we can-- I like to call it rabbit holing, but we'll bring you down the rabbit hole, kind of that Wikipedia effect, where you just keep clicking and you want to start spiraling into the depths of Access to really help reinforce your mastery.
AMANDA FIELDING: And that comes not just from technological advances, but also in the way that we structure our content, the way we acquire it, develop it, label it, tag it. So there's a lot that goes on behind the scenes before you even get to that front-end interaction that we're really looking to invest in. And finally, though not least, yeah, the investment in technology, I mean, again, content was king and is king, really, in our business.
AMANDA FIELDING: We know that the contents are the goods, but we have to be investing in the methods to deliver it. And one thing we're looking to do is bring that content mobile. We've had amazing response to our responsive design, but we need to personalize that experience. So one thing we're looking to do is leverage and really hype up one of the features on AccessMedicine called MyAccess, where an end user can personalize their experience and bring their preferred content, interact with others perhaps in their class, share content, and create a community around our content.
AMANDA FIELDING: And that all happens with technology. And so leveraging some of those widgets, looking for partners to deliver mobiley, that is our biggest push in the coming years. And we're excited to see that next iteration of McGraw-Hill in the digital space. Thank you. I told you I'd geek out.
JAKE ZARNEGAR: Great. Now, I assume that's the good type of spiraling, right?
AMANDA FIELDING: Yes. Not like the bad kind, no.
JAKE ZARNEGAR: When I think of spiraling down, that doesn't usually end up well for that person.
AMANDA FIELDING: It's a good spiral. It's a rabbit hole.
JAKE ZARNEGAR: All right, we'll call it the happy spiral.
AMANDA FIELDING: Yeah.
JAKE ZARNEGAR: OK. Well, thank you. Anna, you are batting cleanup today to talk about the American Academy of Orthopedic Surgeons' new ROCK product. Whereas the ABS has SCORE, you have Rock. You both have a very strong acronym game. But I'd love to hear about your product, and please take it away.
ANNA SALT TROISE: Awesome. Thank you so much. Similarly to others, I was asked to discuss our origin story. And I think a lot of the themes that I'm going to talk about today are really similar to those of the other talks, particularly Dr. Joshi's. You'll see imitation is the highest form of flattery. So in 2017, our board of directors approved a new education strategy. And in it, they took a different approach to engaging stakeholders.
ANNA SALT TROISE: Education is not about product formats. It's not about access. It's about providing personalized needs-based content to customers and stakeholders when and how they want or need it. It's about building digital relationships based on frequency, emotion, and convenience, the idea being to optimize experiences with customers to meet their needs. The overall strategy was really clear-- we needed to win in specific segments.
ANNA SALT TROISE: And the best part was that the overall organization was absolutely committed to helping our team be able to do this. So at the same exact time, just to make it a little more complicated, the Academy developed a new five-year overall strategic plan and a new set of core values. So anything that we do within the educational team, but also within the rest of the organization, always has to map to the overall strategic plan and to our values.
ANNA SALT TROISE: For those of you who are in associations, you'll probably know the phrase "we do this because it adds member value." At the Academy, we really consider the phrase "member value" a four-letter word. It generally means that no one can figure out how to justify an action, a program, or a service, so they couch it in a really vague sentence. To us, value means reach, it means net income, or it means strategic alignment.
ANNA SALT TROISE: So what does this mean specifically when we think about product development? In listening to our customers, we kept hearing our residents say that their main goal was to perform well on the orthopedic and training exam, ultimately so that they can pass the Part I Exam and become a certified orthopedic surgeon. We heard them say over and over again that they have no time and that they're really stressed out.
ANNA SALT TROISE: We heard program directors say of their residents that they spend themselves a huge amount of time developing a five-year curriculum. But every single academic center, as Dr. Joshi said earlier, has a different curriculum and handles their education differently. There's a limited ability to test a resident and compare it to other centers across the country. There's no clear ability to identify an area of strength or weakness.
ANNA SALT TROISE: And there's no way to support a struggling learner. We also heard that there are a couple of other competitive products in the market to help residents with their educational journey, but nothing that really mapped to the orthopedic and training exam and nothing that mapped to the Part I Board Exam. So ultimately, as you've heard over and over again, there was no structured, standardized way to teach residents.
ANNA SALT TROISE: So this is a really big problem. And the Academy decided that we could actually help solve this. We've got a ton going for us. We have a really, really passionate, dedicated group of volunteer surgeons. We develop and own and administer the orthopedic and training exam. We have a really great relationship with the ABOS, and we have a ton of content.
ANNA SALT TROISE: So we played the what-if game. So what if we could link the In-Training Exam to anticipate how a resident will perform on the Part I exam? What if we could build out a content set to help programs standardize what they teach? What if that content was mapped to the In-Training Exam? What if we could target that content so the resident learns exactly what they need to know so that they save their time?
ANNA SALT TROISE: What if we could tell the program director how the resident performed who was struggling, what areas were a strength or a weakness, and how and what they can improve on? And what if we could do all of this by repurposing our own assets? So we built a business plan, we pitched it to the board, and we got a $5 million investment. We were really, really excited.
ANNA SALT TROISE: But we were incredibly overwhelmed because how on Earth were we going to be able to do this? So from the tech side, we looked at the real classic build, partner by utilize the current platforms to see what would be the best thing to do something on this scale. Our technology needs were really clear. We wanted a company that had a strong medical and education knowledge and leadership, experience in a similar market, had standardized tool sets, workflow integrations, could work with an aggressive timeline, be flexible, be aligned with our values, and share a similar mindset around overall product development.
ANNA SALT TROISE: So we conducted an RFP. And after evaluation, we chose Silverchair. So with that, we've invited authors. We've developed our workflow and our processes. We've developed our go-to-market strategy, our pricing strategy. We've onboarded authors, blah, blah, blah, blah, and we did the build. So the ROCK, the Resident Orthopedic Core Knowledge, is a massive product.
ANNA SALT TROISE: It's got 552 chapters, 11 specialties, 22 subspecialty editors, 90 section editors, over 500 authors, over 1,000 full-text journal articles and book chapters, over 500 videos, and over 3,000 examination questions. Each chapter itself has a mandatory element and an optional element. Each chapter roughly takes a resident about an hour to complete.
ANNA SALT TROISE: And if they want to dig deeper, they can, and they can spend countless hours doing that. We're currently actually live, which is super exciting. We have two specialties that are live on the platform, and we're continuing to roll it out throughout the course of the year. We're providing complimentary access for 220 US programs for the July '22 through '23 academic year. And then we do plan to charge for it afterwards.
ANNA SALT TROISE: So ultimately, going back to our original strategy, we believe that the ROCK will help us own this market through investing in an ecosystem that helps the Academy win and protect market share. Our products span across the board. ROCK is a key piece of this. Our video platform, Orthopedic Video Theater, or OVT, is another piece.
ANNA SALT TROISE: Our In-Training Exam, our testing product, ResStudy, are all powered ultimately by our analytics engines, which helps struggling learners evaluate their residence progress through all of these items in all five years of their residency. What's really cool is we're starting to think about what's next and how do we use emerging technology to be able to really invest even further in this.
ANNA SALT TROISE: So how can we layer in skills training using AR, VR, et cetera, to enhance their training and assessment even further? So I think with that, I'm going to stop sharing my screen, and I'm going to pass it back to you, Jake.
JAKE ZARNEGAR: OK. Well, thank you all for those products origin and also keeping us up-to-date on the progress on those products. I do have a series of questions, some of which I had pre-prepared. But then hearing you all talk, I just had to add some new ones, which I'm going to jump into right now. So I'm going to open this up for all of the panelists to answer these questions.
JAKE ZARNEGAR: Also, if anyone would like to ask a question, we have a Q&A feature in this webinar. So I saw one question was already asked and answered. But if you'd like to ask a question or put one there for me to ask live, please do. We've got about 20 minutes to ask questions. So I'm going to kick this off with probably the question that most folks are asking, which is, do all disciplines have these kind of national curricula or these top-down curricula, like surgery potentially with ROCK?
JAKE ZARNEGAR: Or did you have to develop these? Or are there folks building these curriculum that you could map an education product to? Maybe I'll start with you, Dr. Joshi.
AMIT JOSHI: OK, yeah. I think almost every specialty has a defined curriculum now, but they run the spectrum of how well developed they are. So I showed you that 2013 module, for example. I think there are many curricula out there that are not even up to that. They basically say "anesthesia, moderate sedation, you need to know that." But that doesn't provide you with any more resources to actually dig in and do that.
AMIT JOSHI: And then it goes to things like ROCK, which has thousands and thousands of resources, videos, and self-assessment questions. So I think there are many stages of evolution.
LAURA RICCI: Yeah, I can add on to this too.
JAKE ZARNEGAR: Yeah, Laura.
LAURA RICCI: And I'm actually glad you asked this question because one thing I didn't have time to touch on in my talk but that I think is important to know is, when I mentioned the ABMS and the federation of medical boards that set the standards for what recertification and so on looks like, that is the standard that all boards are supposed to follow. But each board is able to implement its own version of that within those standards. So what you see is actually, from specialty to specialty, what that recertification exam looks like is very different.
LAURA RICCI: And boards focus on different kinds of education. Some recertify are based on knowledge of certain journal articles. Others have more of a walking around knowledge kind of thing. So it really does vary based on discipline what learners are expected to know as they're certified and as they're recertified too. So that, I think, is an important nuance when you think about medical education because there really is no one size fits all, even at the level of those overarching standards.
JAKE ZARNEGAR: So you really got to look at it discipline by discipline and see what's there, what the level of their curriculum, how well defined it is, what their board certification process looks like and their testing looks like. Right. Great.
AMIT JOSHI: And I'll just add, there are so many nuances based on the different organizations and what they do. So the organizational structure in orthopedics looks different than the organizational structure in surgery and pediatrics and blah, blah, blah. So that becomes really important because each organization, to Anna's point, is going to do that analysis and say, is this really in our wheelhouse, or is it not? Are we going to put this giant investment if it doesn't align with our strategic plan?
JAKE ZARNEGAR: Great. Well, another phrase that jumped out at me that I heard from both Amanda and Anna was assessment first, organizing the learning around the assessment. Anna, you had noted that you were thinking about the exam and mapping to the exam. Amanda, you used assessment first and how folks want to begin a topic. Could you all talk about that approach and how your products are kind of mapped to the assessment or supporting the assessment, or even putting the assessment up front to identify gaps and strengths?
AMANDA FIELDING: Yeah, I can jump in. And, I mean, McGraw-Hill is not a test prep company. We have test prep products, and we have test prep content. I think we know that the importance of these milestones in medical school-- the licensing exam, the USMLE-- is key. And so we want to make sure that while we are not a test prep tool on Access, we do offer opportunities to assess your knowledge along the way and then remediate on the same site with our deeper dive content.
AMANDA FIELDING: So it's twofold, not just on the content side, but also on the behavioral side. That is how our learners are learning. To the point-- I'm sorry I can't remember whose presentation it was, I think it was yours, Anna-- that they don't have time. They don't have time to sit down and hunker down and read a book. So they need to bang through this Q&A, figure out where they're not doing so great, and that's where they're going to invest their time in.
AMANDA FIELDING: And so we need to be able to facilitate that and provide the content that they need to fill those gaps, reassess and make sure they're good, and then move on to the next product. And that's where innovation is coming in at McGraw-Hill. How do we meet that need, whether within Access or other products? But we try to not consider our Access at least a test prep tool, but we do have that core foundational knowledge to help reinforce the material-- the knowledge you'll need to know.
AMANDA FIELDING: Sorry, we have the core foundational material to reinforce the knowledge you will need for these milestone exams, whether they're boards or otherwise.
JAKE ZARNEGAR: Great.
ANNA SALT TROISE: So a couple of things from our perspective. I'd say, first of all, orthopedic surgeons are really, really good test takers. And that is an important piece because they quiz and they can test-- I would imagine general surgery is the same-- but it doesn't necessarily mean they're learning. And so if you look at the Part I Board Exam, there is a general blueprint of what they're going to need to cover.
ANNA SALT TROISE: We've taken our In-Training Exam and sort of mapped that overall blueprint so it's similar from that standpoint. The ROCK then maps to the In-Training Exam. So ultimately, as you progress, you'll be able to understand how you're going to perform on these various tests. Within the ROCK, too, we have built out a pretest and a post-test.
ANNA SALT TROISE: So you should be able to then see at the beginning, presumably, you're not going to perform very well. You go through the curriculum, you're able to take the test. You're able to improve. And then ultimately, you're In-Training score should be higher. At the end of all of this, we should be able to do a study on that to statistically show that this all maps out and that your time should be spent here.
JAKE ZARNEGAR: Great. Great. Laura, I think you introduced this, but I'd love to hear everyone's thoughts on this. We're talking about medical school here, we're talking about residency. But what we're seeing is that actual professional practice now is also turning into a time of continuous certification, continuous assessment, the introduction of competencies, as well as I guess the new thing that I learned about during this process was Entrustable Professional Activities.
JAKE ZARNEGAR: Dr. Joshi, I think you told me about those. But this way of constantly assessing someone during their practice, which, in the past, had been kind of you get board certified and you could go do anything. But now it does seem like there's a kind of more continuous top-down requirement and evaluation of folks. So I don't know, Dr. Joshi, if you could say how that is working in your discipline.
AMIT JOSHI: Yeah, so EPAs, Entrustable Professional Activities, are becoming very, very important across the ABMS universe. So again, different boards are in different levels, but surgery is actually probably at the forefront now of deploying EPAs for 2023. EPAs are, to what you said, Jake, they point to ongoing micro assessments. As opposed to the other extreme, every 10 years you take the giant exam, this is, OK, I'm assessing you now, ooh, and I'm assessing you now, and I'm assessing you now.
AMIT JOSHI: Many, many low-stakes assessments, which, in aggregate, provide a constellation of durable data versus a singular every 10 year high-stakes assessment. And so we think that-- and there's data to show this very well-- that for good learning theory, the micro assessments are going to be much more powerful for both validity, but also more importantly, driving change in practice.
AMIT JOSHI: And that's what's going to be really important. We've talked a lot here about trainees. We really haven't talked-- Amanda talked probably the most about people in practice. But neither the ROCK curriculum nor the SCORE curriculum is oriented actually towards people in practice yet. But that's going to become a really important thing as EPAs expand beyond just trainees. We actually want to drive practice and drive change in practice for people who are 30 years out of training, not just six months out of training.
JAKE ZARNEGAR: Yes, that does seem like the direction of more oversight and kind of micro assessment in the workplace. Exactly. Great. Anyone else comment on that type of continuous certification?
AMIT JOSHI: I was going to make one more point, which is that we've talked-- Anna and I had very similar points, which was it's at least starting off as a rigid system. We have this curriculum, we have the assessments, Part I of the exams that people need to study for. What we found actually even in the last couple of months is that we can use SCORE to drive important initiatives that are not even on examinations yet. And so I'll give a great example of that.
AMIT JOSHI: Just this morning, I was reviewing three new modules that we have on the SCORE curriculum dealing with DEI issues in surgical training. And so I reviewed an allyship and bystander module that's going to go into SCORE. Are we testing on that topic at the board? We are not. But is it a really great way to deploy that content that's timely, that's needed, that's important to surgical trainees?
AMIT JOSHI: Absolutely. So I think we're going to see more and more of that as time goes on that it's not necessarily all devoted to meeting that final assessment, it's actually a way to change maybe not even core surgical things within practice.
ANNA SALT TROISE: 100%. We needed to start somewhere, and that's sort of the core surgical practice piece. But a lot of the items that our members are really interested in, too, is more on the practice management side, so the business of orthopedic surgery-- contracting, leasing, hiring, firing, things like that-- because so many of them are still in private practice. So it's another good tool to be able to get education information out to them.
JAKE ZARNEGAR: It does seem like a big opportunity. You're in training for a certain amount of time, and hopefully you're in practice much longer than you were in training. So if you're going to need a constant flow of education and assessment during that period, then especially organizations who are professional associations in those disciplines would have a great opportunity to, again, save them the time as well as organize the things that they need to know during their practice, not just in training.
JAKE ZARNEGAR: I have a question for Laura. Maybe an update if you found any on the trend of, for particular, for CME activities. In the near past, physicians have a lot of leeway in selecting their own CME activities, the format, but also the topics. Certainly, there's a minimum amount of required training that they would need to submit to their licensure boards.
JAKE ZARNEGAR: But from there, there was a lot of individual control of what the components of that training was. Are you seeing more either employer- or system-based control over what people need, what topics that people need to cover in their CME training? Or is it still a physician-decided choice there?
LAURA RICCI: Yeah, that's a really good question. And this actually sort of ties into something I was thinking when we were talking about the last question and regular assessment and really driving education off of data. So I think where we are right now is it's still pretty early days in terms of how health systems are getting involved in education. We are seeing that more physicians are employed by health systems and not private practice.
LAURA RICCI: And so that's really driving more education-driven choice of what gets taught or what physicians are learning. Physicians, we still see that they do have a lot of discretion. They can make their own choices about what CME activities that they are pursuing. But at the health systems level, there's more opportunity to get data about actual health outcomes. So not just the assessment and at the individual physician level and what they need to know to be recertified, but also there's an opportunity which is still nascent but I think is interesting to think about, where health outcomes can actually be improved.
LAURA RICCI: So are there areas where there's a particularly risky procedure or something that a health system wants to improve its outcomes? It can tailor education for its employers and encourage them to learn more of the modern techniques and improve the ways that they learn on that basis. Now, this is, as you can imagine, a tough thing to do in practice. Health systems data is pretty all over the place.
LAURA RICCI: But there are smart people working on this problem. So I think if we could project forward a little bit, that kind of focus on health outcomes could certainly be an important part of where employers become more involved in driving education for physicians.
JAKE ZARNEGAR: Dr. Joshi.
AMIT JOSHI: I was going to just add on to Laura's point, which is two ways. One is, there are increasing state and hospital mandates about different types of topics. And so two examples that come off the top of my head, for New Jersey, for example, I need two hours of child abuse training every year, and I need two hours of appropriate opiate prescription practices. If I'm a trauma surgeon, I need so many hours of trauma. If I am doing something else, I'll need so many of that.
AMIT JOSHI: So those are increasing kind of niches that you have to meet. And the ABMS has just recently in the last year approved a new set of standards that are mandating tracking of quality and outcomes within your practice. And so as time goes on, we, as content providers, are going to need to provide that, for those of us who provide content for diplomates are going to need to provide that kind of tracking mechanism, but also education about how to do quality.
AMIT JOSHI: And so that's going to be something coming as well.
LAURA RICCI: I--
AMANDA FIELDING: I want to piggyback a little-- oh, sorry, go ahead.
LAURA RICCI: No, go ahead, Amanda.
AMANDA FIELDING: I just find it interesting that we're sort of in the same-- even though we're content providers, there's more being asked of us as these content providers. And it goes into analytics and data, and we're not set up yet to be that engine. And so that's one area in technology where we're pushing a lot. We need to have those metrics and be able to tell the story to our customers just as much as we need that information ourselves.
AMANDA FIELDING: And so providing those analytics, that data, those stories as part of our "service" as a content provider, if you will, is a new space for a legacy organization who is used to just a pretty black and white set-up. You create content, you deliver it, the end. And now with this digital age, there's more being asked of that. And I find it really interesting. And I'm sure a lot of people are up against this, so it'll be interesting to see how it's handled.
JAKE ZARNEGAR: And you mean as far as reporting on activities, strengths, gaps?
AMANDA FIELDING: Yeah, I mean, on an individualized level, going back to assessment, what is assessment without analytics? It's nothing. And even for our larger institutions, it goes beyond what resource is being trafficked the most. It goes into, well, how many people are returning? What is the story? Who is using this site? Is it really my budget that should be paying this?
AMANDA FIELDING: Should it be another budget? There's a lot of questions. And now that our librarians are getting more tech-savvy and very into analytics and data, we have to be ready and able to supply them with that, for their edification and for our own, for product development, for sales, whatever it is. And it's a gap right now, and we're trying to close it. Because it's a service really, and it's not one that we, as content providers, have ever really explored and are now feeling the pressure to actually step it up.
JAKE ZARNEGAR: Right. Right I mean, I definitely saw in Anna's presentation that you do consider that 360 data from their interactions with your products as being a key data point.
ANNA SALT TROISE: We think it's going to be the main differentiator for us, and it's the only way we'll be able to compete in the marketplace. Because, I mean, think about everything we're talking about, they're all competition regardless in the education space. The piece, too, on top of that is training staff and customers on what to do with it. So you can collect everything you want-- and, Jake, you and I talked a lot about this over the years-- but making sure that people know what to do with it and how to be able to tell the story and get the insights out of it.
ANNA SALT TROISE: Because otherwise, it's pointless.
AMANDA FIELDING: It's just data dump. It's just numbers. Yeah, absolutely.
JAKE ZARNEGAR: Well, on that note, I will have to call our time here today. So, Amanda, you got us on spiraling and data dump there, so.
AMANDA FIELDING: It's a new vocabulary for everyone.
JAKE ZARNEGAR: Appreciate it. Well, first of all, thank you to the panelists. This, again, has been a fantastic content, fantastic information. I learned something, and I talk with you all regularly. But getting you all together to talk through these topics, the idea of producing more data out of your education products into these assessment systems is really fascinating. Now, for the folks who have joined us, when you leave the webinar, you are going to get a one-question survey about how you enjoyed the event.
JAKE ZARNEGAR: I'd appreciate you if you take two clicks to complete that. And for everyone to note, our next event in this series will take place on September 7, and we'll cover data trends. So thank you all for the lead-in to that. That was very well done. And the title is "The Alchemy of Turning Your Disparate Data Into Gold." OK. So we hope to see you there.
JAKE ZARNEGAR: Thank you again to the panelists. This was unbelievably useful content. And thank you to the attendees for asking questions as well. So this wraps us for today. Thank you very much. STEPHANIE LOVEGROVE
HANSEN: Thanks, everyone.
00:
Form Title Download
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode