Cadmore media player playing video Business Not As Usual: Learnings from Digital Transformation
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
Business Not As Usual: Learnings from Digital Transformation
Description:
Business Not As Usual: Learnings from Digital Transformation
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/b3fcaa51-a9bf-4579-a072-3d82713cfd1f/videoscrubberimages/Scrubber_10.jpg
Duration:
T00H57M38S
Embed URL:
https://stream.cadmore.media/player/b3fcaa51-a9bf-4579-a072-3d82713cfd1f
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/b3fcaa51-a9bf-4579-a072-3d82713cfd1f/session_2d__business_not_as_usual__learnings_from_digital_tr.mp4?sv=2019-02-02&sr=c&sig=mZoXKz96X9Kodddwv72fAIYNKUZeS39VYGieog3OYSo%3D&st=2024-11-20T01%3A26%3A34Z&se=2024-11-20T03%3A31%3A34Z&sp=r
Upload Date:
2024-02-02T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
Thank you for joining us today. I'm Lori Carlin, chief Commercial Officer at Delta think. And I've been asked to moderate this session today for my colleagues. Today, our panel will be talking about digital transformation and their thoughts on disrupting business as usual. Our panel includes experts from both within and outside our industry.
First, we'll hear remotely from Reese Jackson, chief technology innovation officer from rocketmill, a digital marketing agency specializing in digital analytics and building proprietary technology to solve some of the toughest marketing challenges faced by publishers today, including cookie lists, attribution, mixed media optimization. He's known to clients he works with as a Data Wizard and data therapist, and he will talk us through how you can find the determination to embrace data and analytics in the way you've always intended and a measurement framework strategy as well.
Next, we'll hear from Patty Kurtz, director of Digital Strategy for health affairs, which is the nation's leading publication where health policy advances in her role. She oh, sorry. In her role, she oversees all things digital, including the website analytics and reporting. I feel like the mic is going in and out. Is that just me?
Pick it up because I feel hunched over. OK Role she oversees all things digital, including the website analytics and reporting, the tech stack, social media, email, anything, digital and product development, among other things. She describes herself as a digital transformation ist, if that's a word and change manager. And we'll share examples of how to put what Reese talks about into practice.
And then lastly, we'll have an exciting discussion between oh, sorry. Yeah, between Reese, patty and Paul g. Paul is the vice president of digital product management and development at the American Medical Association. Paul is a solutions engineer and product development executive with 22 years experience. Transforming medical publications into digital platforms that drive audience and revenue growth.
His leadership experience crosses diverse business functions, including editorial publishing, platform operations, product management and development. And with that, I'm going to hand it over to Patti to get us started. Thanks, everyone. We're actually, since Reese can't attend, he's in the UK, but we thought it'd be really helpful for everyone to hear from him.
As Lori said, we're trying to bring this session together to bring people in from outside the scholarly publishing industry. Controversial, you know, to talk about digital transformation and some approaches that we've taken within our organization. So we'll hear about a 10 minute video from him, but just pretend that he's. And just to kind of reaffirm the agenda for today of what we're going to cover is finding determination to embrace data and analytics and the way you've always intended.
We're overcoming your fears of data and analytics, looking at your measurement framework as the core for digital transformation. Measurement is a very important element of a successful digital transformation. And once Reese finishes his section, I'm going to actually bring some of what he talked about to life to give you some real examples of this is how we've done it, and we're trying to not just make it theoretical, but also actionable.
And then the conversation that we're going to talk about is going to be kind of wider than scholarly publishing. Let's see if this works. Reese Yeah. It is working. No sound is coming out. OK ready?
Nope Hi. Thank you all so much for attending this talk. And we're going to try it again. Ready? Hi. Thank you all so much for attending this talk. I'm really sorry I can't be there with you in person today, but I hope you're all enjoying the conference.
My name is Reece Jackson and I'm the Chief Technology innovation officer for the digital marketing agency rocketmill, based in the UK. We've worked really closely with Patty and the wider Health Affairs team over the past few years, and today I'd like to share with you some of the steps that we took to mature Health Affairs use of data and alleviate some of the fears and concerns around using analytics.
The steps are going to be talking through include understanding your current date of maturity, building measurement frameworks, documenting and understanding your technology stack reporting and how to ensure the adoption of analytics into people's day to day work. And the best starting point I can suggest is to make use of a maturity model. You may have seen things like this before, but they're extremely useful for quickly identifying areas for improvement.
Yet here is not just to see where you sit on this scale from 1 to 5, but to identify what needs to be done next in order to improve. It might be really tempting to see where you are on this, on this diagram and then immediately aim for the top tier. However, most businesses won't be able to jump from an immature state to a very mature state because good analytics isn't just about having the right tools and data, but also about having the right skills, behaviors and processes in your organization, all of which take a lot of time and effort to nurture.
A maturity model like this can be useful to help self-diagnose issues and understand the next step of your journey. You can also take maturity models just one step further by looking deeper than just the 5 points shown on the previous slide. So in the case of health affairs, when we looked at performance, we reviewed a range of different data related topics, including things like consent management, data collection, analysis, optimization and technology.
Now, your next step in almost all cases is going to make sure that you have an effective measurement framework. The measurement framework is really simple by design. The goal is just to align your strategic objectives as a business with measurable performance metrics. I'm going through this process. It's a very deliberate hierarchy for the metrics and highlights the ones that matter, and that's more importantly, the ones that don't matter.
The measurement framework gives your stakeholders clarity, consistency and transparency on which KPIs to monitor. It will also be used as a foundation for auditing your data collection processes to make sure they're configured to deliver the KPIs that are defined in your framework. For example, if you build your measurement framework and find that you've got a KPIs in there, that is very important to understanding business performance, that isn't currently collected or easy to access through any of your existing technology.
That could be a gap that you'd want to want to fill. Important decisions like which KPIs to prioritize. Quite often the consensus on a range of different stakeholders. And for this reason you might want to consider hosting a workshop to help develop your measurement framework. There's a variety of different ways to run these workshops, but we've found we get some great results by discussing very high level questions like why does your website even exist and what does a successful year look like for your business?
And just to give you an example of what the result might look like, here's the measurement framework that we created for Health Affairs. And what it does is it explicitly link the organizational vision of creating better health policy for people to organizational goals, such as improving the authority of the published content, which in turn is then linked to measurable KPIs by altmetric scores and impact factor.
Now that you've got a measurement framework in place, the next step would be to make sure that you review all of your existing digital technologies at your disposal. This doesn't need to be super technical, just a document that shows which tools you used, what they used for and how they connect, if at all. And who is responsible for their maintenance.
Use this information alongside your measurement framework to spot gaps in your tech stack. So, for example, if you had KPIs relating to improving the customer experience, you want to make sure that you've got tools that are capable of measuring, measuring and optimizing the customer experience. And if necessary, improvements and prioritize them according to the impact that they're going to have on the KPIs that are in your measurement framework.
Next, you want to bring your framework to life by creating these standardized reports that are shared and consumed across the business and creating consistency in how performance metrics are presented and consumed is really key because it makes it easier to create a common understanding of data and performance across your teams. It's almost impossible to create consistency or improve data literacy when your reports are being manually pulled together periodically in something like Google Sheets or excel, and they may vary in format from time to time.
Once everybody is familiar with how to use the dashboard, they should then be used as a foundation or for discussing performance at every opportunity. So you can bring them to review meetings, for example. The idea is to create a more proactive conversation around why something has happened and what you can do about it, as opposed to simply reviewing the past or bringing or using meetings to bring everybody up to speed with the latest numbers.
Now, I appreciate I might have made that last slide. Very simple. Dashboarding can and is a very complex process, but unfortunately it's outside the scope of this presentation. But if you're not really sure where to start, I would definitely recommend checking out tools like Google's looker studio, Microsoft Power bi, or tableau, and each of these offer capabilities to import data and quickly visualize it in a compelling and interactive format.
So, for example, with health affairs, we were able to use power bi to import data from all of the tools I showed a moment ago in the technology stack, as well as from sources like crossref and social media. And then by combining all of these together, we were able to display citation metrics alongside website engagement, right alongside business performance. Now, it might be kind of tempting to stop here and say job done as soon as the dashboards are delivered.
This talk is really about overcoming fears with working with data, and I've got three last tips to help with that. And these are especially important when working with a team that may not have a lot of experience with digital analytics tools and may be unfamiliar about how to incorporate them into their work. The first tip is to provide written supporting documentation for any systems that you expect your team to be using regularly because you're trying to help these people get started.
They don't necessarily need to be highly technical documents and should include really basic information, even things just like how to log in and who to contact if they have any further problems. So in this screenshot here, what we're looking at, you can see we just added a very simple section to the Health Affairs dashboard explaining what each of the different pages showed and how they could best be used.
On my next tip is about building habits and routines. As leaders, it's really important to set your expectations about when and how your new dashboard should be used and who should be using them. In my experience, if you don't make it explicitly clear, many people will assume that it's just not their responsibility because they're not data people. In practical terms, this usually means is things like explicitly telling your team that your dashboards will be discussed in each team meeting and that everybody should be familiar with the numbers ahead of time and be prepared with any notable observations from their expertise to discuss in the meeting.
My final tip is to embrace the fact that your analytic capabilities and priorities will always be shifting. Data maturity is about improving the use of data across the whole organization, not just hiring a few data experts or onboarding the latest tools. Ask people what they're finding, confusing about the reports and dashboards and adapt accordingly. They're able to foster an environment in which people feel very comfortable to talk about their knowledge gaps or concerns or problems that they're having with working with the data.
You'll find it much easier to know where to improve. Let's quickly recap. I recommend that you start by auditing your existing data capabilities by using a data maturity model. You can use this to spot areas for most immediate improvement. Next, we want to follow up your measurement framework and then use that as your source of truth for the metrics that matter and create this real hierarchy of the most important through to the least important.
You'll then want to review your technology stack through the lens of your measurement framework to see if there's any significant gaps that need to be filled in order to deliver upon your goals. And then next, you want to set up your interactive dashboards again, aligning them to your measurement framework to decide which information to display and importantly, which information not to display. And then finally, you want to set yourself up for success by creating the right expectations about when and how data should be used.
The more you can incorporate KPIs into your day to day work and create an environment in which people feel comfortable talking about data, the faster your data, maturity will grow. Thank you all so much for coming to this talk and hopefully I'll join you very shortly for a panel discussion. Thank you. But we're going to try to Zoom him in. But we're having some technical difficulties with that as well.
So any hard questions we can't answer? Oh, OK. And just as background recent work together for probably 8 or 9 years I've worked in the Uc and then in the UK at different marketing agencies and consultancies. I joined the scholarly publishing world in February 2020 working for Health Affairs at the start of a global pandemic, never having worked in health policy whatsoever before.
So my role was to come in as a change manager to kind of essentially take the journal from where it was. And to modern day and hopefully the future. So the first order of business was setting up our measurement framework in the first year with Reese's health. There's a lot of technical requirements to bring our tech stack together and I will talk about how we fund it kind of at the end or how we fund some of our innovation work.
So I know that's going to be a focus, but I'm going to talk you through some of the ways that we actually brought this to life. We're a small and nimble team. I head up our digital team. We have eight people on it. Total Health Affairs is 45 to 50 people. Just to give you an indication of the size of our team, we have a really big editorial team and they're great.
But for us with our dashboards, we had to centralize many data points. And I will say that we're not 100% there. We're probably 90% there. We have a few elements to get over the line in terms of our global data layer, which is what we use to capture the customer information and marry it with our fulfillment information. But this is a lot of data to put in one place.
And I will tell you, if you try to hire an analyst at $65,000 a year, they're not going to be able to bring all this together. That's why we took on an agency because they had a whole team that knows how to use all of these different platforms to bring them in at one at one point. And we could pay them kind of on a project basis to help us do it and then help maintain it.
There's a lot of really cool things that we've done with the dashboards. And again, it's just kind of our starting point for analysis. The goal and strategy of this is to make better informed business decisions. Where we talked about at the start, the model with the five different pillars and there's one for data and then there's one that we have for our customer experience.
We've moved forward on our customer experience. It's been three years. We have not moved forward on our data and analytics because we want to, but it's still really hard to embed. So we're trying to kind of consistently revamp it and incorporate principles of change management. It's not to say that we've succeeded at everything, so you'll kind of hear that theme throughout, but in order to have that culture change happen, we give access to the reports to everyone within the organization.
A lot of internal training. We do a lot of here's how you can use it here. Why? here's why it is important for your job with this. We have a cross department reporting team which people are actually required to participate in the reporting team. So have editors that are on the reporting team. And I'll share a few stories about why that's really interesting in a minute.
But anyone can access this and it's easy to access. I think that's really important. Also, I don't know. I'm going to use some examples. You can nod, smile, whatever you've set up. Google Analytics as an example for your brand. You've given everyone access. Your team hasn't signed in six months and you're like, so if you want to look at performance of our organic, you can go to Google analytics, you wait a whole other week and they say, I'm too embarrassed to tell you I didn't know how to access Google Analytics.
Yes, we've all experienced it. So we actually ours is integrated with Microsoft because we use Microsoft within the organization. It's single sign on that is really important because you're decreasing that barrier to entry for people to kind of opt in on this. Also, it's incredibly visual. I blurred this out just because it's our it's our analytics that would be happy to show it to you and how we actually use the data and information.
Some of the coolest things that we've integrated the API from altmetrics so you can sort by the top best performing altmetric score, you can leave aggregated it by the issue and you can search by topic. So if it included the word coven, I could aggregate all of our altmetric data for COVID as an example and lots of really cool things where we've made it easier. So people aren't going to multiple platforms.
We want them to stay within the dashboard. Restart at the start about having that know when you said here's all the things that you can find from the dashboards. I did a data therapy session with every head of department this year and said, is your team using the dashboards? Yes no. Why? when was the last time you signed in to the dashboard? Yes no.
Why? what is missing from it? We have to treat this the same way that you would a customer experience on the website and make it easy for people to opt in. So this one is different. And it's important to show you this because this is a separate dashboard that we created for funders because we wanted to show them the how, what they invested in with us to publish on and how that actually led to something.
So this one's completely different because where you saw kind of a cover page with a lot of words, here's the different tabs and why you should use it. Anybody can read this dashboard, we put a question at the top. And underneath it answers it. So how visible are we in Google analytics? Search engine results on the top left. That is a graph we all want to see. We started publishing on health equity.
We had our inaugural issue on racism and health in February 2020. That's some fantastic correlation. And now I can take this information and go tell the funders, not only did you get this many page views, we've increased our visibility in search and that's organic search news, search and scholar search, among other places.
However, bottom right what was the average click through rate from Google search engine results? If you take those two data points together, that's telling me something quite insightful. Our click through rate were going up. So we're visible in search engine result pages, but people are not clicking on us. Why? what's wrong with our metadata?
What is our search engine result look like? Why aren't they clicking on us? Who's close to us? Spoiler alert am always ranks higher for these key terms, but I can then kind of know that this is a place for analysis to kind of look and see why people aren't clicking on it. So we're trying to embed this across the organization with the cross-departmental reporting team.
We do incentivize them with an annual lunch with our CEO, which is a big deal to some people. We also write it into their performance report. If you are required to report, we want you to report on it. And they are also responsible for sections of our monthly report on this one. I just want to tell you this other story, which I think is great. This is like a metric of success for me.
So we've launched this reporting team. We've launched a few other processes in terms of thinking about search and consumer search. And an editor came to me and said, patty, we should use Aca in this article title no, you should use Affordable Care Act. I think we should use Aca. It's like, let's figure out the answer to this. So we opened Google search trends and we pulled up both.
Spoiler alert. Aca ranks higher. I was like, you just proved me wrong. So now we put in all of our article titles and not Affordable Care act, but the reason why it was a metric of success is because this editor came to me and asked that Director of digital, I'm not on the editorial team. So they're now using these tools to be able to work with authors to say, if you really want your article to have more views and stand out, here are words that you should use instead.
We also do quarterly tech checks where that 50 word article title no, Thank you. We help coach them and the authors say if you want someone to read your content, we have the authority to be able to do so. Take our advice. And so we show the authors the same way of how to approach their article writing. So it's a pretty cool.
And also with incentivizing reporting. We require everyone on my team to get Google Analytics certified as a minimum. It's not necessarily because Google Analytics is all that we use. It's an easy certification and you're teaching someone to create that behavior to sign into a platform on a regular basis.
not, unfortunately. Don't influence the entire organization would love all of Health Affairs to be Google Analytics certified. But here we are. And OK, so now I'm going to switch gears a little bit from the measurement and framework and talk about how we try to treat what we're doing at Health Affairs as a commercial business.
I know it's a little bit of a different topic, but this is the one that Paul and I are going to talk about a little bit more later. So in a previous session, I heard someone talking about competitors and who your competitors are within publishing. And I'm sure you know this, but it's worth saying that your competitors for content consumption are not in this room.
I'm not going to do a competitive analysis, me versus you. I'm going to do a competitive analysis, me versus the bigger players. And that's because regardless of who your readers are, they are your consumers. And so you want to make sure that you are creating experiences that are easy for them to follow. If it takes three steps, four steps, five steps to purchase something to sign in, you are creating a barrier that they are not used to used to Google.
They're just using Netflix, they're just using any of these. And it will continue because those brands are leading the way with UX. So the plug here is that please don't compare yourself to each other when you're trying to look for inspiration, look outside the industry and figure out how to make it easier to give users access to your content. Also with product development.
So if you have followed Health Affairs over the last kind of 3 to 5 years, you will notice we've changed considerably. As I mentioned earlier, we did have investment for kind of innovation, digital transformation, and we've launched a boatload of products, very tired and don't want to launch any more products. But we follow an 80 over 20 rule, which when I came into the organization, it was a culture shock for me.
It was a culture shock for everyone else. We're still learning, but part of it was I believe that we got to get stuff out in live so that we can learn in vivo. Like we have to figure out what do people like this? Do they not as long as it doesn't hurt your brand, you got to try to launch it. Now in an organization where my peers have been there for 30 plus years, this was foreign to them and was very scary.
So our podcast, for example, health policy, that is a great play on words, isn't it? Policy odyssey, podcast, Odyssey. Who knows? Up until the day before launch, I had people internally saying, we can't launch with this. We cannot. This is this is not Siri and it's serious enough for our brand.
And I kept saying, it's a podcast. We are up against other really cool podcasts. We want people to listen to ours. Let's just try it. Within the first 24 hours, we had hundreds of tweets, one that said, I've been waiting 10 years for Health Affairs to come out with a podcast, so that was great. And also when we launched our COVID 19 resource center, which, yes, everyone should have had one that year, it took us a little bit more time and we got one up and then we optimized it.
And when we optimized it, we did this big launch for it. We had subcategory pages, right? So we're not just going to publish everything on COVID. We have to figure out what we as a brand are going to publish on COVID. Here's our eight categories. You can see it on our website. The day before, after we created this digital environment, people were like, oh, actually we need to change those categories.
We had a pivotal moment. Do we relaunch this Resource Center that the world needs or do we pay attention to this internal discussion? Let's just go ahead and launch it and see what happens. And if we need to change it in the future, we will. And then we were able to use performance metrics to see what people were actually reading. What were they searching to get to our site, what was attracting them to our site?
And did we need to keep those categories or change them? We kept them and we added a few more over time. Similarly with some of the other products that we've launched, we are now at this place of 80 over 20. So as long as it's really good, not perfect, we're comfortable with launching it. And all of our products also are built on the foundation of our journal. Every single one of these products is not creating new content.
It's curating content in a different way. And a different format. I actually believe that that's one of our many commitments to inclusivity and accessibility is you have to be trained to read a peer reviewed journal article, and I'm not going to read all of Health Affairs because it's great content, but it probably falls asleep. I would listen to our podcast or attend one of our events or read our newsletter where we're trying to bring in other voices to curate our content.
And then again, a whole other session. But we're monetizing it and we're figuring out ways to diversify our revenue from doing that. We sell newsletter ad space, we sell events that are sponsored, and we work with ad often to figure out how to put it behind a paywall. So our events are behind a paywall and we can either give you access if you pay for access to that individual event or membership event, or if a sponsor wants to come and give me $25,000 to sponsor event, I'll tell the whole world they did that and make it open for everyone.
So it's a really exciting business model. Also with Health Affairs forefront, it used to be Health Affairs blog and it was for 15 years we migrated it and then we're actually able we, we couldn't do anything with it because it was a blog. It was more than a blog. And so we couldn't put altmetric on it because it was a blog. Would you compare it to another blog?
This blog was way better than. Now so now it's a digital publication. We can actually deposit it with crossref and you're able to go back and do $7,500 articles and just deposit them. And then all of a sudden our author community is like, hey, this is showing up in my author profile. Again, that one didn't really lead to money, but now we're selling forefront series.
Now funders are interested in forefront series instead of just general series. Additionally, with the product development operating like a commercial business is really investing please in conversion rate optimization and user experience. If you look in the Wayback Machine and look at our website from a couple of years ago, the left side was how you subscribe.
I'm just going to let you sit with that and see if you would purchase that. Does that feel great? No it's so confusing. And how do you. How do you even do it from. From mobile. And then there's tabs at the top. And I'm not a fan of tabs on a page, but that's personal preprints.
And so we actually worked with a subscription agency that used to advise Netflix on how to optimize their subscription journey. And we came up with this. We actually identify users when they come to our page and say, are you an individual or institution? And then we send them on a different journey. What that does is actually improve conversion rate and we even improved it by over 20% We used to also on the left that page was hosted on our fulfillment site, so we had no ownership over branding, over user experience.
We kind of sent them to a brick wall. Now we host the majority of our information on our own site. That's great for seo, it's great for brand authority, it's great for brand ownership and it's great for conversion. So this is just from commerce revenue alone. We know that not all of our revenue for subscriptions comes from e-commerce, but hey, 20% uplift in the long term, that's enough money. And also how we work a commercial business is that we invest in marketing.
Now we have marketing team members and we also put money out there to make marketing things happen like apply for awards. You have to pay to get awards and to be considered for awards. I think that's a weird business model, but I'm for it because it's not my personal money. And because of that, we are able to put these things in our advertising kit until advertisers who are thinking of giving us money to promote their brand, we're able to say, hey, we're actually receiving these accolades ones I'm most proud of actually all of them.
But we got the search award for 2021 for our coverage of pandemic policy. That was a time we couldn't use the word COVID because it was scary and people were trying to do bad things with the word COVID. And we also want to telly award like we are a single journal and we want to telly award. How do we do that? Because we produced a video.
Don't know how we did that still, but we did. And it was to accompany our racism and health issue that was published in February 2022. So we had a 1 to 1 interview with historian Harriet Washington, who wrote Medical Apartheid. It's a great video. You can find it on YouTube. It's freely available to everybody. But the reason why we did that is because if we were just going to launch this theme issue, we're kind of still going to alienate the community that we were trying to reach.
So we have a video and it sets the groundwork for everything that's in our theme issue. And then we won an award, which is like a marketing award. We also got the web award for best publishing website. Sorry if you did not win it, please apply next year. I don't know, but we were one of the only publishers that applied, so there were a few, not too many. But we get to put that award in our advertising kit. And then the most exciting one we find out on June 8.
But we are finalists for the drum awards, which is the largest global marketing awards, and it's for a digital transformation work at Health Affairs. So that's a testament to the entire team that's worked at Health Affairs at large, but also within the digital team. Also with how we operate is testing, testing and testing. Again, we have an audience 360 feedback program.
I kind of want to work on the branding for that, but basically we have rolled this out across the entire organization. So again, just this is one to think in your head, you know, when you're talking about something and someone says to you, we did that three years ago, it won't work, or we did that 20 years ago and it won't work. And I'm like, that's even worse. But or our readers want this, our researchers want this, our authors want this.
And when I came into this industry, there's a lot of that happening and it was very overwhelming. And I felt like, wow, you guys know everything and I know nothing. Why am I here? Um, and then we started to do things and launch things and try things, and they would fall flat on their face because it was based on things that people were saying internally.
Our authors want this one, which was great, is, well, our readers come and read forefront, which is formerly Health Affairs blog, and then they go on and read a journal article and they buy a subscription was like, that is an awesome user. Does that really happen? I have no idea. And I was like, OK, so let's figure out how to actually measure that and see if it's true.
Now we know what content sells right? But it took us a while to build that machine. So this 360 feedback group, we use it to overcome any of those internal kind of pressures or disagreements. So we get to the point of if we can't agree, let's test it and how do we test it? Also, we kind suck up these hypotheses from around the organization and we have rolled this out so that everyone knows we're testing these things.
If you have an opinion about something, we're testing it. Currently, the debate that we're having is our director of events loves to put the entire website, the web page, sorry, copy into the email to get you to come to the event. Why would you do that if all the information is on the web page? Well, because all our readers need all of that information and email. It's a fantastic we're going to test it over three months and we're going to use short copy versus long copy and see which one converts people and gets them to register for the event.
She's still not happy that we're testing it because she wants to be right, but I'm excited to see what the results are going to be. We also do stakeholder interviews, focus groups. That sounds like a lot of money. No, you have your audience, you create a list of people, send them an email asking if they would have 20 minutes of your time. And we host the stakeholder interviews and focus groups ourselves.
Is it perfect? No is it cheap? Yeah does it tell us a lot of information also? Yes and then we have just a general inbox where we encourage people to come to us with questions. We'll see. OK that the quote, teach a man to fish, but let's be inclusive. Let's teach everyone to fish with every new thing that we introduce.
We document it said that before documentation is so helpful because people learn in different ways. Have you ever been to an internal training session or like one from HR and then three months later they're like, why are you having trouble with your expenses? We did a training three months ago and I didn't have expenses three months ago and I need to figure out how to do my expenses. Now Same thing with everything else that we're trying to do.
So we have a content SEO playbook where people know about what keywords to use, which H1, H2, H3, then all of your guys's world of metadata, which is incredibly fascinating, bring them all together. And then we do regular training and regular check ins. So I don't need all of these. I have different members of the organization lead them. And it's been really helpful. And so, yeah, as you can see on the right, we have regular training sessions.
That's Erin Noel, our social media manager, talking about how you can build your social presence. That applies to everyone in our organization. I want my executive publisher to have a social media presence. I want our forefront editor to have social media presence. I can't mandate it because it's their personal life, but I can encourage them and I can also give them a training so that if they want to opt in, they know how to do it. Don't worry, there are actually free trainings for how to optimize your social media presence if you presence if you want it on Health Affairs.
And we also hire folks and vendors outside of publishing. These are some of the organizations that we work with that are traditionally not from publishing. I know blueconic is getting more involved. Rocketmill Reese, you heard from him earlier. That's a performance marketing agency. House of kaizen is a subscription. I don't know what their brand is.
They helped us optimize their subscription experience, so they helped us do a little tear down and figure out what was wrong along the way. And get really overwhelmed. Didn't know what to do. And then we found our way out. So here we are. Long dash was formerly Atlantic 5857, which is the consulting arm of the Atlantic.
They wrote a strategic plan for us. Blueconic is a CDP and echologics. If anybody works at atypon, please cover your ears. If anyone is an add upon client, please open your ears. We wanted to figure out how to do front end development ourselves using page builder. So we did. And we invested some time and we trained an entire development team that works out of India and as cheap as chips.
And they developed our entire website for us using page builder so that we didn't have to get in that queue of return tickets. OK I'll just put that out there. And and then the last two slides is with all of this, with everything that we've done and we will happily talk about this any other time, we had to build a scalable Roi strategy. So when I built the digital department and then we extended it to the rest of the strategic plan, it had to pay for itself.
We were in a fortunate position where we got a grant from Robert Wood Johnson foundation, kind of like an innovation grant. Everyone here wants an innovation grant. If you don't have it or a fund and not have to give your money to the publisher. But we use that to invest in product development and digital transformation so that we could show the value of what we were doing.
There are so many ways where we've increased our revenue because of our digital transformation, including diversifying our revenue portfolio. So our advertising program three years ago was at a certain amount. We are 300% above where we were three years ago and we're trending to be at 500% above where we were next year. What are we doing? We are selling space to our audience.
We have a very niche audience that people want all of you to also. And we also set an annual and campaign performance targets for our team. So I manage my team like a marketing agency. They have goals to hit and if they don't hit their goals, don't worry, they usually like they're not in trouble, but figure out how to help them reach their goals. And all of my goals include all of their goals.
So we're all holding each other accountable. And yes, it's even things like making sure we get a certain number of views on our content, what we do right, the content is OK, and most importantly, we balance our mission and our revenue and our decision making. So sometimes my team will say, I want to work on this project and I say, OK, is it going to make us more money? Should we do it?
It's nice to have. We're a nonprofit. Everyone wants to do the nice to have mission driven stuff. So it's a mindset shift. And lastly, this is the secondary submission that we put in to present. It was not selected, but this is very, very important. Our health equity work. So we were actually we put together this blueprint to how to incorporate health equity in everything that we do, including digital transformation.
And so we invested a lot last year alongside our disability and health issue to be able to tell the world that we are mostly accessible. I would not say fully accessible because we've had some technology barriers, but we are mostly accessible and making sure that we elevate voices of underrepresented individuals and populations. And by really lifting up all of the work that we're doing as an organization.
So my partner in justice is our director of Health equity, Dr. Raven watts, and we've done a lot to figure out how to embed these processes across the organization so it doesn't sit-in a silo and so that everyone is considering this as part of the process. I encourage you to read this article again, happy to talk about it. And before we get to questions covered a lot of good stuff.
And it sounds like we've done so many incredible things and we have failed a lot coming into this industry is a digital native or however you want to label me, it's fine was a culture shock. And did not succeed. A lot of the times I didn't know what I was doing half the time. I would say it took 12 to 18 months until I even understood most of the acronyms that you all know that are part of your DNA.
I have different acronyms. We should go together and come up with our own glossary, but we've failed as well as succeeded and we learn from our failures and we kind of figure out how to pick ourselves up and move forward. And that comes from both the digital transformation. And then change management across the board. I've talked a lot and I've talked really fast. You can find me on LinkedIn and also anywhere else after this.
Laura is going to start with some questions and then we'll open it up to everybody else. Thank you, Abby and Reese from abroad. Unfortunately, we're not going to be able to patch reach Reese in for the Q&A, but Paul and Patty are here and ready to answer questions. And I'm going to start off. But if you folks have questions, let us know.
We've got a few that we're going to go through to start. So first question here, where have you had success with scaling your digital work within the business? Paul you want to start that? But on the scaling side, we got a lot of scale out of creating a network out of java, plus the archives journals about 10 years ago and a lot of the scale all started with that.
It's not just a branding exercise. It impacted how we created content and editorial, how we facilitated peer review, how we tagged our content, how we pull it together. It allowed us to have standard templates, standard modes of operation that allow us to add a new journals when we have strategic need, create new content marketing channels that are by topic rather than by journal. It enhances seo, it enhances everything.
But I think the most important thing is it gave everyone a Jama network, which there's a lot of people at Jama network, a single thing you can hold in your head. That's the concept of what you're building and what you're doing over many years. And that's really, really hard to get into. The entire operating apparatus, like everyone on the same page, a concept that you can kind of Judge things against.
Does it go with or against this single thing and times where it's a struggle or it's failing, it's usually because we get off the same page and trying to get back on is part of how you get scale and more sustainable operations and higher traffic audience understands it if you understand yourself. Right so there are clearly challenges in scholarly publishing that consumer, the consumer world, doesn't face.
Probably vice versa, too. But we have some unique approaches in scholarly publishing for sure. And patty, you've talked about some of them already, but there's many more. So so what would you say are those main challenges of that sort and how does it complicate your work? And what should scholarly publishing be doing to kind of reduce those hurdles?
Well, from. From think. Is this working? Yeah OK. From my personal experience, I think mentioned it, it's learning the language and all the intricacies. Like some things feel very, very, very complicated that could be simple or explained in a way that's like less alienating.
And so how do you make that more inclusive for people that don't work in scholarly publishing or might not be a vendor who's traditionally from scholarly publishing and figuring out how to be able to look at it as a solution even if it sits outside the remit. I think also a challenge I find is that we often think about. Um, siloed audiences like our authors or our funders or our researchers when if we want to do what we're doing right, we need to think about kind of content consumers at large who might find this information interesting, even if they're not like in our instance, a health policy wonk.
The world thought that health policy was really interesting for two or three years. And we saw that and we missed our opportunity by not launching this strategic plan two years prior. But it showed us that people were interested in this topic even if they hadn't been before. I think it's a really interesting question because. I am usually set up as being the progressive in a discussion, but in this time I'm like 22 years embedded in this industry.
And like when you first start working in journals publishing, you're like, well, why can't we just do it? Like, well, the New York Times did it this way. And blah blah, blah. And you rattle off all these consumer things that people can do really easily through scale, but they have really large audiences to shrink down into tiny units. And we start with a small audience, even like physicians, that's 1 million targets, right?
Like that's, that's really, really small. So how do you get scale and revenue from that? And, and content isn't as easy to just content mill when it comes to the types of education educational materials we create, which a journal article is educational, it's based on valid information, and any mistake in that information can damage your brand for longer than some small businesses ever even stand.
And that integrity, you know, it goes a long way. So it's expensive to create the content and you're bringing it to a small unit of users and oftentimes the users aren't the payer. So you have IP ranges that open up content access and you're not allowed to ask for someone's information to open the contact access. And when you start like I'll hire people that are new to the industry to work on the authentication wall and they just want to like take it all away and make it simple.
Just charge for access. But then we wouldn't make nearly the revenue if we did it in the consumer way off of the tiny audience we have. There's no long tail baked in, so it creates these dilemmas, which I think staffing wise, it's really good to have a mix of people with consumer knowledge and the scholarly publishing knowledge to work together and continually bump heads because you find answers in between.
Yeah and want to actually talk to that point too, because it's again, it's just different wasn't from this industry, but a lot of people here stay in their job for decades. That that is eye opening to me. Like I'm so glad you love your job so much. But also, digital folks don't mean if you look at the tech industry, you're changing jobs every 2 to three years.
That's normal. And part of it is because you're trying to learn continually where you go to a consultancy, where you work with a lot of different clients, but then there are people that have been at the same brand doing the same ish work, maybe move around a little bit for 10 or 20 years and think there's something that's missing of if, if, for example, this isn't going to happen, I'm not propositioning you.
But if Paul came to health affairs, he would bring a wealth of knowledge that I don't have and that other people in the organization. But if you don't ever leave, you're kind of just only giving that information to Jama where it's like if you can spread out that talent and that experience, it would probably benefit the entire industry. Just my hypothesis.
I will say it's interesting to me too. I've always been in this industry, although have moved around, um, and coming from a sales and a marketing background, what I have always tried to instill in folks that I'm working with is people are people and what works in terms of marketing techniques and sales techniques and digital, you know, looking at or your data and what's happening on your websites is the same.
It's not so different in scholarly publishing as it is in other areas, and we can learn from those outside organizations for sure. Any questions from the audience before I move on? OK um, so when and how do you decide to partner with external experts and how do you upskill your team in the long term, particularly with limited resources?
Who wants to go first? I'm happy to take this one first. And then mine is kind of like we've it's almost like a startup within the organization, right? So we didn't have a team. Now we're like eight people. Um, and then had to recruit a team and March 2020 and do you remember, no one was looking for a job.
Everyone was just like, not moving. We have no idea what's happening in the world. I was like, yeah, but I really need a social media manager and someone to overhaul our analytics. So in that instance, ended up bringing an agency to set things up so that we could have and then I could bring in more junior to mid-level people and teach them how to do the work. So I tried to bring on the vendors or consultants to do like the setup and creating the infrastructure.
Sometimes I will have the internal team do it, but it's really difficult to find somebody that knows that inside and out sometimes. Um, and then, yeah, we've got a mix now of. Traditional no, don't want. Outside of scholarly publishing digital people. And then we do have a mix of people that come from a deep background of scholarly publishing. And it's awesome because now we're kind of both able to learn from each other.
I would say if we have a big project that we need, you know, outside eyes on and skills, then we'll hire for that. We try to do anything that has to do with making critical accountability decisions directly in house. Maybe with augmented help like think customer research we use outside firms for. But it goes through a filter of questioning at that earns us a reputation for being hard to work with.
But we're really, really tenacious about making sure we understand where we're going and that our employees are what we're building value in, rather than a series of interconnected vendors, which can happen in this industry easy. And it's hard to make those decisions. But we really try to focus on making sure consultants are in gap areas where it's not based to our intellectual property or our concerns as a publisher.
Um, and, but so we're getting the best of both worlds, a really deep bench of people who our IP plus external experts that we can bring in when we need to fill gaps. Great so in addition to kind of the consulting, bringing in that additional skill set or more bandwidth, there's the technology. And you know, we all organizations that will only build and those who are really open to buying.
Usually there's a good there needs to be a mix somewhere along the line. So how would you say that you make those decisions in terms of building or buying, and then how do you onboard and bring that up to speed? What's the process like? What could others learn about making those decisions and bringing in that assistance or not? You go first because you probably build way more than we do?
No, it depends. We we built? If so. So Publisher is a publication. We sell a publication, its content plus a customer list plus a brand. That's your IP. Those things you need on your turf replicated not out in a vendor's hands, not in a like your website is not a CMS.
Every website is a CMS of every kind. The thing that you need on your soil is something you own and control that holds your IP safe in case of downtime or anything like that. And it allows you to scale and replicate your content with ease across multiple repositories. You can sell it multiple times. Getting into that landscape takes a lot of work, but the workflow systems, the repositories for our content that keep it safe, that's what we build the stuff.
That's the layer that users see. That's just that needs to change all the time. It's fungible. We're always going to be behind. We're always going to be improving and transforming. Now we're going to stop. So those we rent journals, we build very, very little. Some other products like education, a little bit more complicated because the transcription services and things like that, that's the IP and that we build.
And so we do a little bit more building on the education side. But ironically, the bigger brand gets less internal build and we rely on vendors. But the, the integrations between those vendors, those are us. And so we just sort of have this rationale of making sure we're in ownership and control of the things that are critical to our mission critical aspects.
Yeah, same. Think we take a similar approach? We try to not have to bring in new tech if we don't have to. But if it makes her job easier, we do it. And then when we do, there's always a named owner. If you ever bring in a new vendor or tech and five people are interested in it, that's great. But we need a clear owner and decision maker and other people are kind of like informed or consented to.
He's a racy often to be able to say like, who's going to be the decision maker on this? And if a decision isn't made, where do we bring it to get that decision made? But that person owns the vendor relationship from billing through regular status meetings to implementation to bringing any kind of challenges and also owning the roadmap. So if we have eight bits of tech across my team, different people have our owners of different tech, but they have to know what's happening in the other kind of tech channel to make sure that we're all working cohesively together.
Great just looking around to make sure there are no audience questions. All right. So in terms of product development and optimizing user experience, what are your lessons learned? I still have no idea what a compendium is. No idea. I don't. I don't get it.
I'm like, how do you sell it? What is it? How? like, how is this different? And it's a very industry word for product development, I'd say. Get a little bit like get some feedback and just try it. So again, that 80 over 20 rule, but ask some of your users what they might think about it. You would probably be surprised how vocal your community is.
Your community is probably really vocal just thinking about it of like, here's what you should do, here's what you shouldn't do, and then taking that feedback on board, but also creating a loop for that feedback. So again, the learnings over the first three years is it kind of came in. A lot of passion and let's do this, let's do this, let's do this.
And you just have to trust me. No one's going to trust me. Who the heck was I? And we did a few things and we got proof of concept, which was great. But then as time went by, we figured out a whole plan for how the team likes to work. So first we get buy in from the leadership team. We're going to do this thing, here's the watertight plan.
And then we get later, we get buy in from the senior managers and then from the full team. And you have to create an opportunity for a feedback loop. And most importantly, you have to create an emergency communications plan that the team know about. So an example that I'll use is that we put an email capture on our blog at the time was a blog. You can't put an email capture that's free content.
It's against our mission. It's like, OK, but like after they view three articles in a 30 day period, we're going to let's just try it. We're going to let's just try it. No, you can't do it. For all these reasons, we tested six well and ended up being six different variants in the end, but we tested different variants and we actually told the whole company, if someone messages you about this email, capture, please send them to me and I will manage it.
Over a 12 month period, we collected over 60,000 emails. Most of them were accurate. Some were like, I hate pop UPS at gmail.com. That's fine. We didn't have two factor authentication because it was a test. How many complaints do you think we got out of the 60,000? Internal or external five. We got 5 internal, we got 2000 external, we got 5.
Like I don't want to give you my email address and we're like, hey, we're a nonprofit. We're trying to figure out get to our users better and all this other stuff. And we're like, Oh yeah, you can have my email address. It's like, OK, Thank you. So stuff like that where you got to kind of put that full plan together for people to be less scared of that change and you think, great all.
Um, I would. I would say that. You know. So when you. No, no organization is going to do things the same way. And should. You're going to hear lots of. Speeches that say what you should do and you should do and you should do.
You need to listen to all of it and try to do as much as what makes sense for who you are. And having a staff that's creative enough to test and apply and do everything that this discussion is talking about also takes like a certain tenacity of finding the right things that you can do and don't like. In my team I have like a block against the word should. If someone says should.
They've kind of called themselves out. They've entered some dreamland. Like we're in a system of constraints for all sorts of reasons. Some are regulatory, some are nonprofit, some are just cost boiling it down to what we can do that will allow us to make like the 80% progress that we need to make so that something changes, something gets enhanced, and we move forward.
That's really the thing that I've learned. Like we're not we're not in a land of infinite resources. So and none of us are. So how do we work within that? And I think that the biggest wins do come from the smallest things. Like it's we're specialty societies because we found a lot of affinity to content that's from specialists and we live in that world.
Even even Jama breaks down into a series of specialties that we serve with different article types and formats. And um, that same thing can be said for all of you. You're all specialists. You all know your thing better than anyone else, and usually the opportunity for your brand to differentiate itself and succeed is inherent in something around your house. It's not in other people's there's not a secret sauce out there that you won't make yourself.
So, you know, the biggest features that we've launched that have registered thousands or changed like our scale, they were tiny features. They were really small things you never would have said about doing. But you saw them, you did them, you tried it. And it's the one thing that didn't fail. In fact, it surprised you how much you didn't fail. And there's just a lot of little things like that that, I don't know, maybe once a year you do something that surprises yourself, but it's fun.
Great Yeah. Should and we've always done it that way. Yeah, those are the two. All right. We're at time. Thank you all. Encourage you to connect with Paul and Patty and Reese on LinkedIn and continue the conversation. Thank you.
Segment:0 .
Thank you for joining us today. I'm Lori Carlin, chief Commercial Officer at Delta think. And I've been asked to moderate this session today for my colleagues. Today, our panel will be talking about digital transformation and their thoughts on disrupting business as usual. Our panel includes experts from both within and outside our industry.
First, we'll hear remotely from Reese Jackson, chief technology innovation officer from rocketmill, a digital marketing agency specializing in digital analytics and building proprietary technology to solve some of the toughest marketing challenges faced by publishers today, including cookie lists, attribution, mixed media optimization. He's known to clients he works with as a Data Wizard and data therapist, and he will talk us through how you can find the determination to embrace data and analytics in the way you've always intended and a measurement framework strategy as well.
Next, we'll hear from Patty Kurtz, director of Digital Strategy for health affairs, which is the nation's leading publication where health policy advances in her role. She oh, sorry. In her role, she oversees all things digital, including the website analytics and reporting. I feel like the mic is going in and out. Is that just me?
Pick it up because I feel hunched over. OK Role she oversees all things digital, including the website analytics and reporting, the tech stack, social media, email, anything, digital and product development, among other things. She describes herself as a digital transformation ist, if that's a word and change manager. And we'll share examples of how to put what Reese talks about into practice.
And then lastly, we'll have an exciting discussion between oh, sorry. Yeah, between Reese, patty and Paul g. Paul is the vice president of digital product management and development at the American Medical Association. Paul is a solutions engineer and product development executive with 22 years experience. Transforming medical publications into digital platforms that drive audience and revenue growth.
His leadership experience crosses diverse business functions, including editorial publishing, platform operations, product management and development. And with that, I'm going to hand it over to Patti to get us started. Thanks, everyone. We're actually, since Reese can't attend, he's in the UK, but we thought it'd be really helpful for everyone to hear from him.
As Lori said, we're trying to bring this session together to bring people in from outside the scholarly publishing industry. Controversial, you know, to talk about digital transformation and some approaches that we've taken within our organization. So we'll hear about a 10 minute video from him, but just pretend that he's. And just to kind of reaffirm the agenda for today of what we're going to cover is finding determination to embrace data and analytics and the way you've always intended.
We're overcoming your fears of data and analytics, looking at your measurement framework as the core for digital transformation. Measurement is a very important element of a successful digital transformation. And once Reese finishes his section, I'm going to actually bring some of what he talked about to life to give you some real examples of this is how we've done it, and we're trying to not just make it theoretical, but also actionable.
And then the conversation that we're going to talk about is going to be kind of wider than scholarly publishing. Let's see if this works. Reese Yeah. It is working. No sound is coming out. OK ready?
Nope Hi. Thank you all so much for attending this talk. And we're going to try it again. Ready? Hi. Thank you all so much for attending this talk. I'm really sorry I can't be there with you in person today, but I hope you're all enjoying the conference.
My name is Reece Jackson and I'm the Chief Technology innovation officer for the digital marketing agency rocketmill, based in the UK. We've worked really closely with Patty and the wider Health Affairs team over the past few years, and today I'd like to share with you some of the steps that we took to mature Health Affairs use of data and alleviate some of the fears and concerns around using analytics.
The steps are going to be talking through include understanding your current date of maturity, building measurement frameworks, documenting and understanding your technology stack reporting and how to ensure the adoption of analytics into people's day to day work. And the best starting point I can suggest is to make use of a maturity model. You may have seen things like this before, but they're extremely useful for quickly identifying areas for improvement.
Yet here is not just to see where you sit on this scale from 1 to 5, but to identify what needs to be done next in order to improve. It might be really tempting to see where you are on this, on this diagram and then immediately aim for the top tier. However, most businesses won't be able to jump from an immature state to a very mature state because good analytics isn't just about having the right tools and data, but also about having the right skills, behaviors and processes in your organization, all of which take a lot of time and effort to nurture.
A maturity model like this can be useful to help self-diagnose issues and understand the next step of your journey. You can also take maturity models just one step further by looking deeper than just the 5 points shown on the previous slide. So in the case of health affairs, when we looked at performance, we reviewed a range of different data related topics, including things like consent management, data collection, analysis, optimization and technology.
Now, your next step in almost all cases is going to make sure that you have an effective measurement framework. The measurement framework is really simple by design. The goal is just to align your strategic objectives as a business with measurable performance metrics. I'm going through this process. It's a very deliberate hierarchy for the metrics and highlights the ones that matter, and that's more importantly, the ones that don't matter.
The measurement framework gives your stakeholders clarity, consistency and transparency on which KPIs to monitor. It will also be used as a foundation for auditing your data collection processes to make sure they're configured to deliver the KPIs that are defined in your framework. For example, if you build your measurement framework and find that you've got a KPIs in there, that is very important to understanding business performance, that isn't currently collected or easy to access through any of your existing technology.
That could be a gap that you'd want to want to fill. Important decisions like which KPIs to prioritize. Quite often the consensus on a range of different stakeholders. And for this reason you might want to consider hosting a workshop to help develop your measurement framework. There's a variety of different ways to run these workshops, but we've found we get some great results by discussing very high level questions like why does your website even exist and what does a successful year look like for your business?
And just to give you an example of what the result might look like, here's the measurement framework that we created for Health Affairs. And what it does is it explicitly link the organizational vision of creating better health policy for people to organizational goals, such as improving the authority of the published content, which in turn is then linked to measurable KPIs by altmetric scores and impact factor.
Now that you've got a measurement framework in place, the next step would be to make sure that you review all of your existing digital technologies at your disposal. This doesn't need to be super technical, just a document that shows which tools you used, what they used for and how they connect, if at all. And who is responsible for their maintenance.
Use this information alongside your measurement framework to spot gaps in your tech stack. So, for example, if you had KPIs relating to improving the customer experience, you want to make sure that you've got tools that are capable of measuring, measuring and optimizing the customer experience. And if necessary, improvements and prioritize them according to the impact that they're going to have on the KPIs that are in your measurement framework.
Next, you want to bring your framework to life by creating these standardized reports that are shared and consumed across the business and creating consistency in how performance metrics are presented and consumed is really key because it makes it easier to create a common understanding of data and performance across your teams. It's almost impossible to create consistency or improve data literacy when your reports are being manually pulled together periodically in something like Google Sheets or excel, and they may vary in format from time to time.
Once everybody is familiar with how to use the dashboard, they should then be used as a foundation or for discussing performance at every opportunity. So you can bring them to review meetings, for example. The idea is to create a more proactive conversation around why something has happened and what you can do about it, as opposed to simply reviewing the past or bringing or using meetings to bring everybody up to speed with the latest numbers.
Now, I appreciate I might have made that last slide. Very simple. Dashboarding can and is a very complex process, but unfortunately it's outside the scope of this presentation. But if you're not really sure where to start, I would definitely recommend checking out tools like Google's looker studio, Microsoft Power bi, or tableau, and each of these offer capabilities to import data and quickly visualize it in a compelling and interactive format.
So, for example, with health affairs, we were able to use power bi to import data from all of the tools I showed a moment ago in the technology stack, as well as from sources like crossref and social media. And then by combining all of these together, we were able to display citation metrics alongside website engagement, right alongside business performance. Now, it might be kind of tempting to stop here and say job done as soon as the dashboards are delivered.
This talk is really about overcoming fears with working with data, and I've got three last tips to help with that. And these are especially important when working with a team that may not have a lot of experience with digital analytics tools and may be unfamiliar about how to incorporate them into their work. The first tip is to provide written supporting documentation for any systems that you expect your team to be using regularly because you're trying to help these people get started.
They don't necessarily need to be highly technical documents and should include really basic information, even things just like how to log in and who to contact if they have any further problems. So in this screenshot here, what we're looking at, you can see we just added a very simple section to the Health Affairs dashboard explaining what each of the different pages showed and how they could best be used.
On my next tip is about building habits and routines. As leaders, it's really important to set your expectations about when and how your new dashboard should be used and who should be using them. In my experience, if you don't make it explicitly clear, many people will assume that it's just not their responsibility because they're not data people. In practical terms, this usually means is things like explicitly telling your team that your dashboards will be discussed in each team meeting and that everybody should be familiar with the numbers ahead of time and be prepared with any notable observations from their expertise to discuss in the meeting.
My final tip is to embrace the fact that your analytic capabilities and priorities will always be shifting. Data maturity is about improving the use of data across the whole organization, not just hiring a few data experts or onboarding the latest tools. Ask people what they're finding, confusing about the reports and dashboards and adapt accordingly. They're able to foster an environment in which people feel very comfortable to talk about their knowledge gaps or concerns or problems that they're having with working with the data.
You'll find it much easier to know where to improve. Let's quickly recap. I recommend that you start by auditing your existing data capabilities by using a data maturity model. You can use this to spot areas for most immediate improvement. Next, we want to follow up your measurement framework and then use that as your source of truth for the metrics that matter and create this real hierarchy of the most important through to the least important.
You'll then want to review your technology stack through the lens of your measurement framework to see if there's any significant gaps that need to be filled in order to deliver upon your goals. And then next, you want to set up your interactive dashboards again, aligning them to your measurement framework to decide which information to display and importantly, which information not to display. And then finally, you want to set yourself up for success by creating the right expectations about when and how data should be used.
The more you can incorporate KPIs into your day to day work and create an environment in which people feel comfortable talking about data, the faster your data, maturity will grow. Thank you all so much for coming to this talk and hopefully I'll join you very shortly for a panel discussion. Thank you. But we're going to try to Zoom him in. But we're having some technical difficulties with that as well.
So any hard questions we can't answer? Oh, OK. And just as background recent work together for probably 8 or 9 years I've worked in the Uc and then in the UK at different marketing agencies and consultancies. I joined the scholarly publishing world in February 2020 working for Health Affairs at the start of a global pandemic, never having worked in health policy whatsoever before.
So my role was to come in as a change manager to kind of essentially take the journal from where it was. And to modern day and hopefully the future. So the first order of business was setting up our measurement framework in the first year with Reese's health. There's a lot of technical requirements to bring our tech stack together and I will talk about how we fund it kind of at the end or how we fund some of our innovation work.
So I know that's going to be a focus, but I'm going to talk you through some of the ways that we actually brought this to life. We're a small and nimble team. I head up our digital team. We have eight people on it. Total Health Affairs is 45 to 50 people. Just to give you an indication of the size of our team, we have a really big editorial team and they're great.
But for us with our dashboards, we had to centralize many data points. And I will say that we're not 100% there. We're probably 90% there. We have a few elements to get over the line in terms of our global data layer, which is what we use to capture the customer information and marry it with our fulfillment information. But this is a lot of data to put in one place.
And I will tell you, if you try to hire an analyst at $65,000 a year, they're not going to be able to bring all this together. That's why we took on an agency because they had a whole team that knows how to use all of these different platforms to bring them in at one at one point. And we could pay them kind of on a project basis to help us do it and then help maintain it.
There's a lot of really cool things that we've done with the dashboards. And again, it's just kind of our starting point for analysis. The goal and strategy of this is to make better informed business decisions. Where we talked about at the start, the model with the five different pillars and there's one for data and then there's one that we have for our customer experience.
We've moved forward on our customer experience. It's been three years. We have not moved forward on our data and analytics because we want to, but it's still really hard to embed. So we're trying to kind of consistently revamp it and incorporate principles of change management. It's not to say that we've succeeded at everything, so you'll kind of hear that theme throughout, but in order to have that culture change happen, we give access to the reports to everyone within the organization.
A lot of internal training. We do a lot of here's how you can use it here. Why? here's why it is important for your job with this. We have a cross department reporting team which people are actually required to participate in the reporting team. So have editors that are on the reporting team. And I'll share a few stories about why that's really interesting in a minute.
But anyone can access this and it's easy to access. I think that's really important. Also, I don't know. I'm going to use some examples. You can nod, smile, whatever you've set up. Google Analytics as an example for your brand. You've given everyone access. Your team hasn't signed in six months and you're like, so if you want to look at performance of our organic, you can go to Google analytics, you wait a whole other week and they say, I'm too embarrassed to tell you I didn't know how to access Google Analytics.
Yes, we've all experienced it. So we actually ours is integrated with Microsoft because we use Microsoft within the organization. It's single sign on that is really important because you're decreasing that barrier to entry for people to kind of opt in on this. Also, it's incredibly visual. I blurred this out just because it's our it's our analytics that would be happy to show it to you and how we actually use the data and information.
Some of the coolest things that we've integrated the API from altmetrics so you can sort by the top best performing altmetric score, you can leave aggregated it by the issue and you can search by topic. So if it included the word coven, I could aggregate all of our altmetric data for COVID as an example and lots of really cool things where we've made it easier. So people aren't going to multiple platforms.
We want them to stay within the dashboard. Restart at the start about having that know when you said here's all the things that you can find from the dashboards. I did a data therapy session with every head of department this year and said, is your team using the dashboards? Yes no. Why? when was the last time you signed in to the dashboard? Yes no.
Why? what is missing from it? We have to treat this the same way that you would a customer experience on the website and make it easy for people to opt in. So this one is different. And it's important to show you this because this is a separate dashboard that we created for funders because we wanted to show them the how, what they invested in with us to publish on and how that actually led to something.
So this one's completely different because where you saw kind of a cover page with a lot of words, here's the different tabs and why you should use it. Anybody can read this dashboard, we put a question at the top. And underneath it answers it. So how visible are we in Google analytics? Search engine results on the top left. That is a graph we all want to see. We started publishing on health equity.
We had our inaugural issue on racism and health in February 2020. That's some fantastic correlation. And now I can take this information and go tell the funders, not only did you get this many page views, we've increased our visibility in search and that's organic search news, search and scholar search, among other places.
However, bottom right what was the average click through rate from Google search engine results? If you take those two data points together, that's telling me something quite insightful. Our click through rate were going up. So we're visible in search engine result pages, but people are not clicking on us. Why? what's wrong with our metadata?
What is our search engine result look like? Why aren't they clicking on us? Who's close to us? Spoiler alert am always ranks higher for these key terms, but I can then kind of know that this is a place for analysis to kind of look and see why people aren't clicking on it. So we're trying to embed this across the organization with the cross-departmental reporting team.
We do incentivize them with an annual lunch with our CEO, which is a big deal to some people. We also write it into their performance report. If you are required to report, we want you to report on it. And they are also responsible for sections of our monthly report on this one. I just want to tell you this other story, which I think is great. This is like a metric of success for me.
So we've launched this reporting team. We've launched a few other processes in terms of thinking about search and consumer search. And an editor came to me and said, patty, we should use Aca in this article title no, you should use Affordable Care Act. I think we should use Aca. It's like, let's figure out the answer to this. So we opened Google search trends and we pulled up both.
Spoiler alert. Aca ranks higher. I was like, you just proved me wrong. So now we put in all of our article titles and not Affordable Care act, but the reason why it was a metric of success is because this editor came to me and asked that Director of digital, I'm not on the editorial team. So they're now using these tools to be able to work with authors to say, if you really want your article to have more views and stand out, here are words that you should use instead.
We also do quarterly tech checks where that 50 word article title no, Thank you. We help coach them and the authors say if you want someone to read your content, we have the authority to be able to do so. Take our advice. And so we show the authors the same way of how to approach their article writing. So it's a pretty cool.
And also with incentivizing reporting. We require everyone on my team to get Google Analytics certified as a minimum. It's not necessarily because Google Analytics is all that we use. It's an easy certification and you're teaching someone to create that behavior to sign into a platform on a regular basis.
not, unfortunately. Don't influence the entire organization would love all of Health Affairs to be Google Analytics certified. But here we are. And OK, so now I'm going to switch gears a little bit from the measurement and framework and talk about how we try to treat what we're doing at Health Affairs as a commercial business.
I know it's a little bit of a different topic, but this is the one that Paul and I are going to talk about a little bit more later. So in a previous session, I heard someone talking about competitors and who your competitors are within publishing. And I'm sure you know this, but it's worth saying that your competitors for content consumption are not in this room.
I'm not going to do a competitive analysis, me versus you. I'm going to do a competitive analysis, me versus the bigger players. And that's because regardless of who your readers are, they are your consumers. And so you want to make sure that you are creating experiences that are easy for them to follow. If it takes three steps, four steps, five steps to purchase something to sign in, you are creating a barrier that they are not used to used to Google.
They're just using Netflix, they're just using any of these. And it will continue because those brands are leading the way with UX. So the plug here is that please don't compare yourself to each other when you're trying to look for inspiration, look outside the industry and figure out how to make it easier to give users access to your content. Also with product development.
So if you have followed Health Affairs over the last kind of 3 to 5 years, you will notice we've changed considerably. As I mentioned earlier, we did have investment for kind of innovation, digital transformation, and we've launched a boatload of products, very tired and don't want to launch any more products. But we follow an 80 over 20 rule, which when I came into the organization, it was a culture shock for me.
It was a culture shock for everyone else. We're still learning, but part of it was I believe that we got to get stuff out in live so that we can learn in vivo. Like we have to figure out what do people like this? Do they not as long as it doesn't hurt your brand, you got to try to launch it. Now in an organization where my peers have been there for 30 plus years, this was foreign to them and was very scary.
So our podcast, for example, health policy, that is a great play on words, isn't it? Policy odyssey, podcast, Odyssey. Who knows? Up until the day before launch, I had people internally saying, we can't launch with this. We cannot. This is this is not Siri and it's serious enough for our brand.
And I kept saying, it's a podcast. We are up against other really cool podcasts. We want people to listen to ours. Let's just try it. Within the first 24 hours, we had hundreds of tweets, one that said, I've been waiting 10 years for Health Affairs to come out with a podcast, so that was great. And also when we launched our COVID 19 resource center, which, yes, everyone should have had one that year, it took us a little bit more time and we got one up and then we optimized it.
And when we optimized it, we did this big launch for it. We had subcategory pages, right? So we're not just going to publish everything on COVID. We have to figure out what we as a brand are going to publish on COVID. Here's our eight categories. You can see it on our website. The day before, after we created this digital environment, people were like, oh, actually we need to change those categories.
We had a pivotal moment. Do we relaunch this Resource Center that the world needs or do we pay attention to this internal discussion? Let's just go ahead and launch it and see what happens. And if we need to change it in the future, we will. And then we were able to use performance metrics to see what people were actually reading. What were they searching to get to our site, what was attracting them to our site?
And did we need to keep those categories or change them? We kept them and we added a few more over time. Similarly with some of the other products that we've launched, we are now at this place of 80 over 20. So as long as it's really good, not perfect, we're comfortable with launching it. And all of our products also are built on the foundation of our journal. Every single one of these products is not creating new content.
It's curating content in a different way. And a different format. I actually believe that that's one of our many commitments to inclusivity and accessibility is you have to be trained to read a peer reviewed journal article, and I'm not going to read all of Health Affairs because it's great content, but it probably falls asleep. I would listen to our podcast or attend one of our events or read our newsletter where we're trying to bring in other voices to curate our content.
And then again, a whole other session. But we're monetizing it and we're figuring out ways to diversify our revenue from doing that. We sell newsletter ad space, we sell events that are sponsored, and we work with ad often to figure out how to put it behind a paywall. So our events are behind a paywall and we can either give you access if you pay for access to that individual event or membership event, or if a sponsor wants to come and give me $25,000 to sponsor event, I'll tell the whole world they did that and make it open for everyone.
So it's a really exciting business model. Also with Health Affairs forefront, it used to be Health Affairs blog and it was for 15 years we migrated it and then we're actually able we, we couldn't do anything with it because it was a blog. It was more than a blog. And so we couldn't put altmetric on it because it was a blog. Would you compare it to another blog?
This blog was way better than. Now so now it's a digital publication. We can actually deposit it with crossref and you're able to go back and do $7,500 articles and just deposit them. And then all of a sudden our author community is like, hey, this is showing up in my author profile. Again, that one didn't really lead to money, but now we're selling forefront series.
Now funders are interested in forefront series instead of just general series. Additionally, with the product development operating like a commercial business is really investing please in conversion rate optimization and user experience. If you look in the Wayback Machine and look at our website from a couple of years ago, the left side was how you subscribe.
I'm just going to let you sit with that and see if you would purchase that. Does that feel great? No it's so confusing. And how do you. How do you even do it from. From mobile. And then there's tabs at the top. And I'm not a fan of tabs on a page, but that's personal preprints.
And so we actually worked with a subscription agency that used to advise Netflix on how to optimize their subscription journey. And we came up with this. We actually identify users when they come to our page and say, are you an individual or institution? And then we send them on a different journey. What that does is actually improve conversion rate and we even improved it by over 20% We used to also on the left that page was hosted on our fulfillment site, so we had no ownership over branding, over user experience.
We kind of sent them to a brick wall. Now we host the majority of our information on our own site. That's great for seo, it's great for brand authority, it's great for brand ownership and it's great for conversion. So this is just from commerce revenue alone. We know that not all of our revenue for subscriptions comes from e-commerce, but hey, 20% uplift in the long term, that's enough money. And also how we work a commercial business is that we invest in marketing.
Now we have marketing team members and we also put money out there to make marketing things happen like apply for awards. You have to pay to get awards and to be considered for awards. I think that's a weird business model, but I'm for it because it's not my personal money. And because of that, we are able to put these things in our advertising kit until advertisers who are thinking of giving us money to promote their brand, we're able to say, hey, we're actually receiving these accolades ones I'm most proud of actually all of them.
But we got the search award for 2021 for our coverage of pandemic policy. That was a time we couldn't use the word COVID because it was scary and people were trying to do bad things with the word COVID. And we also want to telly award like we are a single journal and we want to telly award. How do we do that? Because we produced a video.
Don't know how we did that still, but we did. And it was to accompany our racism and health issue that was published in February 2022. So we had a 1 to 1 interview with historian Harriet Washington, who wrote Medical Apartheid. It's a great video. You can find it on YouTube. It's freely available to everybody. But the reason why we did that is because if we were just going to launch this theme issue, we're kind of still going to alienate the community that we were trying to reach.
So we have a video and it sets the groundwork for everything that's in our theme issue. And then we won an award, which is like a marketing award. We also got the web award for best publishing website. Sorry if you did not win it, please apply next year. I don't know, but we were one of the only publishers that applied, so there were a few, not too many. But we get to put that award in our advertising kit. And then the most exciting one we find out on June 8.
But we are finalists for the drum awards, which is the largest global marketing awards, and it's for a digital transformation work at Health Affairs. So that's a testament to the entire team that's worked at Health Affairs at large, but also within the digital team. Also with how we operate is testing, testing and testing. Again, we have an audience 360 feedback program.
I kind of want to work on the branding for that, but basically we have rolled this out across the entire organization. So again, just this is one to think in your head, you know, when you're talking about something and someone says to you, we did that three years ago, it won't work, or we did that 20 years ago and it won't work. And I'm like, that's even worse. But or our readers want this, our researchers want this, our authors want this.
And when I came into this industry, there's a lot of that happening and it was very overwhelming. And I felt like, wow, you guys know everything and I know nothing. Why am I here? Um, and then we started to do things and launch things and try things, and they would fall flat on their face because it was based on things that people were saying internally.
Our authors want this one, which was great, is, well, our readers come and read forefront, which is formerly Health Affairs blog, and then they go on and read a journal article and they buy a subscription was like, that is an awesome user. Does that really happen? I have no idea. And I was like, OK, so let's figure out how to actually measure that and see if it's true.
Now we know what content sells right? But it took us a while to build that machine. So this 360 feedback group, we use it to overcome any of those internal kind of pressures or disagreements. So we get to the point of if we can't agree, let's test it and how do we test it? Also, we kind suck up these hypotheses from around the organization and we have rolled this out so that everyone knows we're testing these things.
If you have an opinion about something, we're testing it. Currently, the debate that we're having is our director of events loves to put the entire website, the web page, sorry, copy into the email to get you to come to the event. Why would you do that if all the information is on the web page? Well, because all our readers need all of that information and email. It's a fantastic we're going to test it over three months and we're going to use short copy versus long copy and see which one converts people and gets them to register for the event.
She's still not happy that we're testing it because she wants to be right, but I'm excited to see what the results are going to be. We also do stakeholder interviews, focus groups. That sounds like a lot of money. No, you have your audience, you create a list of people, send them an email asking if they would have 20 minutes of your time. And we host the stakeholder interviews and focus groups ourselves.
Is it perfect? No is it cheap? Yeah does it tell us a lot of information also? Yes and then we have just a general inbox where we encourage people to come to us with questions. We'll see. OK that the quote, teach a man to fish, but let's be inclusive. Let's teach everyone to fish with every new thing that we introduce.
We document it said that before documentation is so helpful because people learn in different ways. Have you ever been to an internal training session or like one from HR and then three months later they're like, why are you having trouble with your expenses? We did a training three months ago and I didn't have expenses three months ago and I need to figure out how to do my expenses. Now Same thing with everything else that we're trying to do.
So we have a content SEO playbook where people know about what keywords to use, which H1, H2, H3, then all of your guys's world of metadata, which is incredibly fascinating, bring them all together. And then we do regular training and regular check ins. So I don't need all of these. I have different members of the organization lead them. And it's been really helpful. And so, yeah, as you can see on the right, we have regular training sessions.
That's Erin Noel, our social media manager, talking about how you can build your social presence. That applies to everyone in our organization. I want my executive publisher to have a social media presence. I want our forefront editor to have social media presence. I can't mandate it because it's their personal life, but I can encourage them and I can also give them a training so that if they want to opt in, they know how to do it. Don't worry, there are actually free trainings for how to optimize your social media presence if you presence if you want it on Health Affairs.
And we also hire folks and vendors outside of publishing. These are some of the organizations that we work with that are traditionally not from publishing. I know blueconic is getting more involved. Rocketmill Reese, you heard from him earlier. That's a performance marketing agency. House of kaizen is a subscription. I don't know what their brand is.
They helped us optimize their subscription experience, so they helped us do a little tear down and figure out what was wrong along the way. And get really overwhelmed. Didn't know what to do. And then we found our way out. So here we are. Long dash was formerly Atlantic 5857, which is the consulting arm of the Atlantic.
They wrote a strategic plan for us. Blueconic is a CDP and echologics. If anybody works at atypon, please cover your ears. If anyone is an add upon client, please open your ears. We wanted to figure out how to do front end development ourselves using page builder. So we did. And we invested some time and we trained an entire development team that works out of India and as cheap as chips.
And they developed our entire website for us using page builder so that we didn't have to get in that queue of return tickets. OK I'll just put that out there. And and then the last two slides is with all of this, with everything that we've done and we will happily talk about this any other time, we had to build a scalable Roi strategy. So when I built the digital department and then we extended it to the rest of the strategic plan, it had to pay for itself.
We were in a fortunate position where we got a grant from Robert Wood Johnson foundation, kind of like an innovation grant. Everyone here wants an innovation grant. If you don't have it or a fund and not have to give your money to the publisher. But we use that to invest in product development and digital transformation so that we could show the value of what we were doing.
There are so many ways where we've increased our revenue because of our digital transformation, including diversifying our revenue portfolio. So our advertising program three years ago was at a certain amount. We are 300% above where we were three years ago and we're trending to be at 500% above where we were next year. What are we doing? We are selling space to our audience.
We have a very niche audience that people want all of you to also. And we also set an annual and campaign performance targets for our team. So I manage my team like a marketing agency. They have goals to hit and if they don't hit their goals, don't worry, they usually like they're not in trouble, but figure out how to help them reach their goals. And all of my goals include all of their goals.
So we're all holding each other accountable. And yes, it's even things like making sure we get a certain number of views on our content, what we do right, the content is OK, and most importantly, we balance our mission and our revenue and our decision making. So sometimes my team will say, I want to work on this project and I say, OK, is it going to make us more money? Should we do it?
It's nice to have. We're a nonprofit. Everyone wants to do the nice to have mission driven stuff. So it's a mindset shift. And lastly, this is the secondary submission that we put in to present. It was not selected, but this is very, very important. Our health equity work. So we were actually we put together this blueprint to how to incorporate health equity in everything that we do, including digital transformation.
And so we invested a lot last year alongside our disability and health issue to be able to tell the world that we are mostly accessible. I would not say fully accessible because we've had some technology barriers, but we are mostly accessible and making sure that we elevate voices of underrepresented individuals and populations. And by really lifting up all of the work that we're doing as an organization.
So my partner in justice is our director of Health equity, Dr. Raven watts, and we've done a lot to figure out how to embed these processes across the organization so it doesn't sit-in a silo and so that everyone is considering this as part of the process. I encourage you to read this article again, happy to talk about it. And before we get to questions covered a lot of good stuff.
And it sounds like we've done so many incredible things and we have failed a lot coming into this industry is a digital native or however you want to label me, it's fine was a culture shock. And did not succeed. A lot of the times I didn't know what I was doing half the time. I would say it took 12 to 18 months until I even understood most of the acronyms that you all know that are part of your DNA.
I have different acronyms. We should go together and come up with our own glossary, but we've failed as well as succeeded and we learn from our failures and we kind of figure out how to pick ourselves up and move forward. And that comes from both the digital transformation. And then change management across the board. I've talked a lot and I've talked really fast. You can find me on LinkedIn and also anywhere else after this.
Laura is going to start with some questions and then we'll open it up to everybody else. Thank you, Abby and Reese from abroad. Unfortunately, we're not going to be able to patch reach Reese in for the Q&A, but Paul and Patty are here and ready to answer questions. And I'm going to start off. But if you folks have questions, let us know.
We've got a few that we're going to go through to start. So first question here, where have you had success with scaling your digital work within the business? Paul you want to start that? But on the scaling side, we got a lot of scale out of creating a network out of java, plus the archives journals about 10 years ago and a lot of the scale all started with that.
It's not just a branding exercise. It impacted how we created content and editorial, how we facilitated peer review, how we tagged our content, how we pull it together. It allowed us to have standard templates, standard modes of operation that allow us to add a new journals when we have strategic need, create new content marketing channels that are by topic rather than by journal. It enhances seo, it enhances everything.
But I think the most important thing is it gave everyone a Jama network, which there's a lot of people at Jama network, a single thing you can hold in your head. That's the concept of what you're building and what you're doing over many years. And that's really, really hard to get into. The entire operating apparatus, like everyone on the same page, a concept that you can kind of Judge things against.
Does it go with or against this single thing and times where it's a struggle or it's failing, it's usually because we get off the same page and trying to get back on is part of how you get scale and more sustainable operations and higher traffic audience understands it if you understand yourself. Right so there are clearly challenges in scholarly publishing that consumer, the consumer world, doesn't face.
Probably vice versa, too. But we have some unique approaches in scholarly publishing for sure. And patty, you've talked about some of them already, but there's many more. So so what would you say are those main challenges of that sort and how does it complicate your work? And what should scholarly publishing be doing to kind of reduce those hurdles?
Well, from. From think. Is this working? Yeah OK. From my personal experience, I think mentioned it, it's learning the language and all the intricacies. Like some things feel very, very, very complicated that could be simple or explained in a way that's like less alienating.
And so how do you make that more inclusive for people that don't work in scholarly publishing or might not be a vendor who's traditionally from scholarly publishing and figuring out how to be able to look at it as a solution even if it sits outside the remit. I think also a challenge I find is that we often think about. Um, siloed audiences like our authors or our funders or our researchers when if we want to do what we're doing right, we need to think about kind of content consumers at large who might find this information interesting, even if they're not like in our instance, a health policy wonk.
The world thought that health policy was really interesting for two or three years. And we saw that and we missed our opportunity by not launching this strategic plan two years prior. But it showed us that people were interested in this topic even if they hadn't been before. I think it's a really interesting question because. I am usually set up as being the progressive in a discussion, but in this time I'm like 22 years embedded in this industry.
And like when you first start working in journals publishing, you're like, well, why can't we just do it? Like, well, the New York Times did it this way. And blah blah, blah. And you rattle off all these consumer things that people can do really easily through scale, but they have really large audiences to shrink down into tiny units. And we start with a small audience, even like physicians, that's 1 million targets, right?
Like that's, that's really, really small. So how do you get scale and revenue from that? And, and content isn't as easy to just content mill when it comes to the types of education educational materials we create, which a journal article is educational, it's based on valid information, and any mistake in that information can damage your brand for longer than some small businesses ever even stand.
And that integrity, you know, it goes a long way. So it's expensive to create the content and you're bringing it to a small unit of users and oftentimes the users aren't the payer. So you have IP ranges that open up content access and you're not allowed to ask for someone's information to open the contact access. And when you start like I'll hire people that are new to the industry to work on the authentication wall and they just want to like take it all away and make it simple.
Just charge for access. But then we wouldn't make nearly the revenue if we did it in the consumer way off of the tiny audience we have. There's no long tail baked in, so it creates these dilemmas, which I think staffing wise, it's really good to have a mix of people with consumer knowledge and the scholarly publishing knowledge to work together and continually bump heads because you find answers in between.
Yeah and want to actually talk to that point too, because it's again, it's just different wasn't from this industry, but a lot of people here stay in their job for decades. That that is eye opening to me. Like I'm so glad you love your job so much. But also, digital folks don't mean if you look at the tech industry, you're changing jobs every 2 to three years.
That's normal. And part of it is because you're trying to learn continually where you go to a consultancy, where you work with a lot of different clients, but then there are people that have been at the same brand doing the same ish work, maybe move around a little bit for 10 or 20 years and think there's something that's missing of if, if, for example, this isn't going to happen, I'm not propositioning you.
But if Paul came to health affairs, he would bring a wealth of knowledge that I don't have and that other people in the organization. But if you don't ever leave, you're kind of just only giving that information to Jama where it's like if you can spread out that talent and that experience, it would probably benefit the entire industry. Just my hypothesis.
I will say it's interesting to me too. I've always been in this industry, although have moved around, um, and coming from a sales and a marketing background, what I have always tried to instill in folks that I'm working with is people are people and what works in terms of marketing techniques and sales techniques and digital, you know, looking at or your data and what's happening on your websites is the same.
It's not so different in scholarly publishing as it is in other areas, and we can learn from those outside organizations for sure. Any questions from the audience before I move on? OK um, so when and how do you decide to partner with external experts and how do you upskill your team in the long term, particularly with limited resources?
Who wants to go first? I'm happy to take this one first. And then mine is kind of like we've it's almost like a startup within the organization, right? So we didn't have a team. Now we're like eight people. Um, and then had to recruit a team and March 2020 and do you remember, no one was looking for a job.
Everyone was just like, not moving. We have no idea what's happening in the world. I was like, yeah, but I really need a social media manager and someone to overhaul our analytics. So in that instance, ended up bringing an agency to set things up so that we could have and then I could bring in more junior to mid-level people and teach them how to do the work. So I tried to bring on the vendors or consultants to do like the setup and creating the infrastructure.
Sometimes I will have the internal team do it, but it's really difficult to find somebody that knows that inside and out sometimes. Um, and then, yeah, we've got a mix now of. Traditional no, don't want. Outside of scholarly publishing digital people. And then we do have a mix of people that come from a deep background of scholarly publishing. And it's awesome because now we're kind of both able to learn from each other.
I would say if we have a big project that we need, you know, outside eyes on and skills, then we'll hire for that. We try to do anything that has to do with making critical accountability decisions directly in house. Maybe with augmented help like think customer research we use outside firms for. But it goes through a filter of questioning at that earns us a reputation for being hard to work with.
But we're really, really tenacious about making sure we understand where we're going and that our employees are what we're building value in, rather than a series of interconnected vendors, which can happen in this industry easy. And it's hard to make those decisions. But we really try to focus on making sure consultants are in gap areas where it's not based to our intellectual property or our concerns as a publisher.
Um, and, but so we're getting the best of both worlds, a really deep bench of people who our IP plus external experts that we can bring in when we need to fill gaps. Great so in addition to kind of the consulting, bringing in that additional skill set or more bandwidth, there's the technology. And you know, we all organizations that will only build and those who are really open to buying.
Usually there's a good there needs to be a mix somewhere along the line. So how would you say that you make those decisions in terms of building or buying, and then how do you onboard and bring that up to speed? What's the process like? What could others learn about making those decisions and bringing in that assistance or not? You go first because you probably build way more than we do?
No, it depends. We we built? If so. So Publisher is a publication. We sell a publication, its content plus a customer list plus a brand. That's your IP. Those things you need on your turf replicated not out in a vendor's hands, not in a like your website is not a CMS.
Every website is a CMS of every kind. The thing that you need on your soil is something you own and control that holds your IP safe in case of downtime or anything like that. And it allows you to scale and replicate your content with ease across multiple repositories. You can sell it multiple times. Getting into that landscape takes a lot of work, but the workflow systems, the repositories for our content that keep it safe, that's what we build the stuff.
That's the layer that users see. That's just that needs to change all the time. It's fungible. We're always going to be behind. We're always going to be improving and transforming. Now we're going to stop. So those we rent journals, we build very, very little. Some other products like education, a little bit more complicated because the transcription services and things like that, that's the IP and that we build.
And so we do a little bit more building on the education side. But ironically, the bigger brand gets less internal build and we rely on vendors. But the, the integrations between those vendors, those are us. And so we just sort of have this rationale of making sure we're in ownership and control of the things that are critical to our mission critical aspects.
Yeah, same. Think we take a similar approach? We try to not have to bring in new tech if we don't have to. But if it makes her job easier, we do it. And then when we do, there's always a named owner. If you ever bring in a new vendor or tech and five people are interested in it, that's great. But we need a clear owner and decision maker and other people are kind of like informed or consented to.
He's a racy often to be able to say like, who's going to be the decision maker on this? And if a decision isn't made, where do we bring it to get that decision made? But that person owns the vendor relationship from billing through regular status meetings to implementation to bringing any kind of challenges and also owning the roadmap. So if we have eight bits of tech across my team, different people have our owners of different tech, but they have to know what's happening in the other kind of tech channel to make sure that we're all working cohesively together.
Great just looking around to make sure there are no audience questions. All right. So in terms of product development and optimizing user experience, what are your lessons learned? I still have no idea what a compendium is. No idea. I don't. I don't get it.
I'm like, how do you sell it? What is it? How? like, how is this different? And it's a very industry word for product development, I'd say. Get a little bit like get some feedback and just try it. So again, that 80 over 20 rule, but ask some of your users what they might think about it. You would probably be surprised how vocal your community is.
Your community is probably really vocal just thinking about it of like, here's what you should do, here's what you shouldn't do, and then taking that feedback on board, but also creating a loop for that feedback. So again, the learnings over the first three years is it kind of came in. A lot of passion and let's do this, let's do this, let's do this.
And you just have to trust me. No one's going to trust me. Who the heck was I? And we did a few things and we got proof of concept, which was great. But then as time went by, we figured out a whole plan for how the team likes to work. So first we get buy in from the leadership team. We're going to do this thing, here's the watertight plan.
And then we get later, we get buy in from the senior managers and then from the full team. And you have to create an opportunity for a feedback loop. And most importantly, you have to create an emergency communications plan that the team know about. So an example that I'll use is that we put an email capture on our blog at the time was a blog. You can't put an email capture that's free content.
It's against our mission. It's like, OK, but like after they view three articles in a 30 day period, we're going to let's just try it. We're going to let's just try it. No, you can't do it. For all these reasons, we tested six well and ended up being six different variants in the end, but we tested different variants and we actually told the whole company, if someone messages you about this email, capture, please send them to me and I will manage it.
Over a 12 month period, we collected over 60,000 emails. Most of them were accurate. Some were like, I hate pop UPS at gmail.com. That's fine. We didn't have two factor authentication because it was a test. How many complaints do you think we got out of the 60,000? Internal or external five. We got 5 internal, we got 2000 external, we got 5.
Like I don't want to give you my email address and we're like, hey, we're a nonprofit. We're trying to figure out get to our users better and all this other stuff. And we're like, Oh yeah, you can have my email address. It's like, OK, Thank you. So stuff like that where you got to kind of put that full plan together for people to be less scared of that change and you think, great all.
Um, I would. I would say that. You know. So when you. No, no organization is going to do things the same way. And should. You're going to hear lots of. Speeches that say what you should do and you should do and you should do.
You need to listen to all of it and try to do as much as what makes sense for who you are. And having a staff that's creative enough to test and apply and do everything that this discussion is talking about also takes like a certain tenacity of finding the right things that you can do and don't like. In my team I have like a block against the word should. If someone says should.
They've kind of called themselves out. They've entered some dreamland. Like we're in a system of constraints for all sorts of reasons. Some are regulatory, some are nonprofit, some are just cost boiling it down to what we can do that will allow us to make like the 80% progress that we need to make so that something changes, something gets enhanced, and we move forward.
That's really the thing that I've learned. Like we're not we're not in a land of infinite resources. So and none of us are. So how do we work within that? And I think that the biggest wins do come from the smallest things. Like it's we're specialty societies because we found a lot of affinity to content that's from specialists and we live in that world.
Even even Jama breaks down into a series of specialties that we serve with different article types and formats. And um, that same thing can be said for all of you. You're all specialists. You all know your thing better than anyone else, and usually the opportunity for your brand to differentiate itself and succeed is inherent in something around your house. It's not in other people's there's not a secret sauce out there that you won't make yourself.
So, you know, the biggest features that we've launched that have registered thousands or changed like our scale, they were tiny features. They were really small things you never would have said about doing. But you saw them, you did them, you tried it. And it's the one thing that didn't fail. In fact, it surprised you how much you didn't fail. And there's just a lot of little things like that that, I don't know, maybe once a year you do something that surprises yourself, but it's fun.
Great Yeah. Should and we've always done it that way. Yeah, those are the two. All right. We're at time. Thank you all. Encourage you to connect with Paul and Patty and Reese on LinkedIn and continue the conversation. Thank you.
Form Title Download
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode