Cadmore media player playing audio Eddy Fan, MD, PhD, discusses how to use an article about quality improvement.
Audio Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the audio. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Transcript
Result Count
of
Search
Segments
Result Count
of
Search
Name:
Eddy Fan, MD, PhD, discusses how to use an article about quality improvement.
Description:
Eddy Fan, MD, PhD, discusses how to use an article about quality improvement.
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/a4c99dbd-eb80-4aad-8d6c-40fd51055df8/thumbnails/a4c99dbd-eb80-4aad-8d6c-40fd51055df8.jpg?sv=2019-02-02&sr=c&sig=FKYcCJdlhRK1HoQ8%2FnhHUN1%2F9OfVvpygfafBh13BBiU%3D&st=2024-10-16T01%3A43%3A29Z&se=2024-10-16T05%3A48%3A29Z&sp=r
Duration:
T00H10M51S
Embed URL:
https://stream.cadmore.media/player/a4c99dbd-eb80-4aad-8d6c-40fd51055df8
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/a4c99dbd-eb80-4aad-8d6c-40fd51055df8/14633761.mp3?sv=2019-02-02&sr=c&sig=1EaekVPlrHFMfvMCco20c0jRstyrYdYb8TcHDtTaDuU%3D&st=2024-10-16T01%3A43%3A29Z&se=2024-10-16T03%3A48%3A29Z&sp=r
Upload Date:
2022-02-28T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
[ Music ] >> Dr. Demetrius Kyriacou Hello and welcome to JAMAevidence, our monthly podcast focused on court issues and evidence based medicine. I'm Dr. Demetrius Kyriacou of Northwestern University and Senior Editor of JAMA. Today we are discussing how to use an article about quality improvement. >> Dr. Eddy Fan Quality improvement studies can reveal important interventions that could lead to enhanced and more consistent application of efficacious treatments which in the end can translate into improvements in care and patient outcome.
>> Dr. Demetrius Kyriacou That is my guest today, Dr. Eddy Fan, from the University of Toronto in Ontario, Canada, and author of the chapter on How to Use an Article about Quality Improvement, in the User's Guide to the Medical Literature. Dr. Fan, welcome. >> Dr. Eddy Fan Thank you. >> Dr. Demetrius Kyriacou So, Dr. Fan, can we begin with the formal definition of quality improvement for our listeners? >> Dr. Eddy Fan Sure. Quality improvement typically looks at interventions that attempt to change clinician behavior, with the idea that these changes in behavior can lead to typically a more consistent, appropriate, or efficient use of established and efficacious clinical interventions which then would translate into improved care and patient outcomes.
And so here, the idea is the likelihood of behavior change, rather than the specific efficacy of the interventions that are used, is generally in or at or near equipoise in the clinical world. >> Dr. Demetrius Kyriacou How does research and quality improvement differ from research in clinical medicine? >> Dr. Eddy Fan The main difference is usually the intervention being examined and the way in which they are studied or evaluated. When you think about traditional clinical research, looking at say the efficacy of intervention, a therapy or a drug, you usually want to evaluate that intervention in a very well controlled, almost sterilized environment, with all the known and unknown factors controlled, increasing the likelihood that patients receive that specific intervention and then figuring out what the actual efficacy of that intervention is.
In contrast, quality improvement interventions are often designed to enhance the implantation of some proven therapy or intervention, and it usually takes place in the real world setting where you're using clinically available data that's collected routinely already such as clinical registries, medical databases, and that sort of thing. And the other major difference is that quality improvement interventions are frequently context dependent. They're usually very complex and multifaceted. And they usually seek to address barriers and facilitators to the implementation of these proven therapies.
>> Dr. Demetrius Kyriacou How does one evaluate the validity of a quality improvement study compared the validity of a clinical trial or observational study? >> Dr. Eddy Fan The good news is that most of the tools are quite similar. In fact, in our user's guide we make a direct link to previous chapters in the User's Guide to the Medical Literature, including those on therapy harm and clinical decision support systems. So, looking at validity typically follows the same sorts of categories, in terms of did the intervention and control groups start with the same prognosis?
Were patients randomized? How was randomization carried out? Were patients in the study group similar? Was prognostic balance maintained in the study? To what extent was the study blinded? And were the groups treated equally? Some specific characteristics that are germane to assessing articles about quality improvement include an understanding of the unit of analysis. So, typically again quality improvement interventions target clinicians, and is important to understand whether the study used the appropriate unit of analysis.
So, was the clinician, or the clinician group, the unit, in understanding that the analysis undertaken targeted the right group. Another important characteristic is data quality. Again, because many quality improvement studies used routinely collected clinical data. It's important to understand what the validity and reliability of that data is. How much missing data there might be present that may affect the ability to make inferences during analysis. And finally, very important is the idea of was follow up complete or sufficiently long?
And this speaks to the point that changes in practice or clinician behavior are sometimes very short lived. May typically occur just around the time of the intervention and very few quality improvement projects report the sustainability more than a year out from the intervention. So, it'd be very important before you launch into an export of an intervention with the use of resources, time, and energy to ensure that any successful intervention is sustainable. >> Dr. Demetrius Kyriacou So, the intervention can be primarily targeted at clinicians instead of the patients?
>> Dr. Eddy Fan Yes. I would say that in a majority of quality improvement studies, the main intervention is targeted at changing clinician behavior or system behavior. So, the proper unit of analysis needs to be made. >> Dr. Demetrius Kyriacou How does one use an article about quality improvement in their own clinical setting? >> Dr. Eddy Fan So, once you've appraised a quality improvement study and found that it seems valid and it's something that you want to export to your site. As we mentioned context is very important.
Context is all those intangible things about your local environment where you're hoping to implement this new quality improvement intervention that includes your resources, the leadership, the culture of your practice, your institution, and those traditions that you really can't always account for in some kind of experimental study. And so, you really need to understand what the quality improvement study investigators did and the nature of their intervention and determine how exportable that is to your site. And often understanding your local barriers or facilitators to how that might be achieved are very important.
And so, you might think about local opinion leaders, what resources you have to bring to bear on implementing these things locally, may impact your ability to translate the findings in the study to your local environment. The best success is probably obtained given that these interventions are usually complex, multifaceted, and interprofessional in nature is to harness a team of people to help with implementation and that typically occurs at the level of administration, other clinicians, other professionals in the healthcare team that may be a part of that intervention to ensure the maximal likelihood of implementation and ultimately of sustainability.
>> Dr. Demetrius Kyriacou Dr. Fan, let's provide a clinical scenario for our listeners. You, the medical director, of an intensive care unit discover that mortality has increased for patients with sepsis. You are considering a quality improvement initiative to improve care and outcomes of your patients. However, you are concerned that are many quality improvement studies have weak designs, poor data quality, and often overestimate potential benefits. Before beginning, you decide to identify and evaluate existing quality improvement studies.
Dr. Fan, if you were the medical director of this intensive care unit, how would you approach this problem? >> Dr. Eddy Fan So, you begin the search for an answer by looking at the available medical literature and in fact, there is a recent before/after study looking at educational [inaudible] program for sepsis in 59 ICUs in Spain. This program used a program to train their clinicians to recognize and treat severe sepsis by implementing evidence based guidelines from the surviving sepsis campaign, looking specifically to treatment bundles.
Across these study ICUs, they found that implementing the bundles led to improved adherence to these bundle components over the 4 month implementation period. And importantly, hospital mortality also significantly decreased during the study intervention time. Unfortunately, looking a year after the intervention in a subset of ICUs, they found that adherence to the bundles had returned to baseline levels, but hospital mortality had remained similar to the post-intervention period. So, using the User's Guide to appraise the study, you find that the investigators decided to implement two interventions in the surviving sepsis campaign guidelines, that received strong recommendations as supported by the available medical literature.
So, it seems like processes of care that were important to implement. And the study did lead to increased adherence and a significant decline in hospital mortality. However, because of the study design being of uncontrolled before after study, it decreases your confidence in the potential causal association between the implementation of the quality improvement intervention and the observed outcomes. It's also a bit challenging because the intervention wasn't clearly defined.
There were no data on costs or any other unintended consequences, but the decrease in hospital mortality is compelling and if it was really the results of implementing this educational intervention, it suggests that any potential unintended consequences were actually likely to be small compared with the potential benefit. So, in the end using the User's Guide to assist in your appraisal of this study, although the confidence is low that the intervention actually improves mortality, the intervention itself being educational in nature has a relative low cost and a low potential for harm and unintended consequences.
And so, applying this educational intervention in your own hospital setting in concert with other clinicians, members of the interventional team, and your hospital administration may help to lead to improvements in patient important outcome. >> Dr. Demetrius Kyriacou Dr. Fan, do you have any other important points that you'd like to make regarding quality improvement? >> Dr. Eddy Fan I think it's important to recognize that as opposed to traditional clinical research, the vast majority of studies evaluating the effectiveness of quality improvement interventions is typically low and so it's just important for consumers of the medical literature to understand these potential limitations and the need for more robust study methods in quality improvement research.
>> Dr. Demetrius Kyriacou Thank you, Dr. Fan, for your comments and insights. More information on this topic is available in the User's Guide to the Medical Literature and on our website, jamaevidence.com, where you can listen to our entire roster of podcasts. Thanks for listening and we'll be back soon with another episode of JAMAevidence. [ Music ]
Segment:0 .
[ Music ] >> Dr. Demetrius Kyriacou Hello and welcome to JAMAevidence, our monthly podcast focused on court issues and evidence based medicine. I'm Dr. Demetrius Kyriacou of Northwestern University and Senior Editor of JAMA. Today we are discussing how to use an article about quality improvement. >> Dr. Eddy Fan Quality improvement studies can reveal important interventions that could lead to enhanced and more consistent application of efficacious treatments which in the end can translate into improvements in care and patient outcome.
>> Dr. Demetrius Kyriacou That is my guest today, Dr. Eddy Fan, from the University of Toronto in Ontario, Canada, and author of the chapter on How to Use an Article about Quality Improvement, in the User's Guide to the Medical Literature. Dr. Fan, welcome. >> Dr. Eddy Fan Thank you. >> Dr. Demetrius Kyriacou So, Dr. Fan, can we begin with the formal definition of quality improvement for our listeners? >> Dr. Eddy Fan Sure. Quality improvement typically looks at interventions that attempt to change clinician behavior, with the idea that these changes in behavior can lead to typically a more consistent, appropriate, or efficient use of established and efficacious clinical interventions which then would translate into improved care and patient outcomes.
And so here, the idea is the likelihood of behavior change, rather than the specific efficacy of the interventions that are used, is generally in or at or near equipoise in the clinical world. >> Dr. Demetrius Kyriacou How does research and quality improvement differ from research in clinical medicine? >> Dr. Eddy Fan The main difference is usually the intervention being examined and the way in which they are studied or evaluated. When you think about traditional clinical research, looking at say the efficacy of intervention, a therapy or a drug, you usually want to evaluate that intervention in a very well controlled, almost sterilized environment, with all the known and unknown factors controlled, increasing the likelihood that patients receive that specific intervention and then figuring out what the actual efficacy of that intervention is.
In contrast, quality improvement interventions are often designed to enhance the implantation of some proven therapy or intervention, and it usually takes place in the real world setting where you're using clinically available data that's collected routinely already such as clinical registries, medical databases, and that sort of thing. And the other major difference is that quality improvement interventions are frequently context dependent. They're usually very complex and multifaceted. And they usually seek to address barriers and facilitators to the implementation of these proven therapies.
>> Dr. Demetrius Kyriacou How does one evaluate the validity of a quality improvement study compared the validity of a clinical trial or observational study? >> Dr. Eddy Fan The good news is that most of the tools are quite similar. In fact, in our user's guide we make a direct link to previous chapters in the User's Guide to the Medical Literature, including those on therapy harm and clinical decision support systems. So, looking at validity typically follows the same sorts of categories, in terms of did the intervention and control groups start with the same prognosis?
Were patients randomized? How was randomization carried out? Were patients in the study group similar? Was prognostic balance maintained in the study? To what extent was the study blinded? And were the groups treated equally? Some specific characteristics that are germane to assessing articles about quality improvement include an understanding of the unit of analysis. So, typically again quality improvement interventions target clinicians, and is important to understand whether the study used the appropriate unit of analysis.
So, was the clinician, or the clinician group, the unit, in understanding that the analysis undertaken targeted the right group. Another important characteristic is data quality. Again, because many quality improvement studies used routinely collected clinical data. It's important to understand what the validity and reliability of that data is. How much missing data there might be present that may affect the ability to make inferences during analysis. And finally, very important is the idea of was follow up complete or sufficiently long?
And this speaks to the point that changes in practice or clinician behavior are sometimes very short lived. May typically occur just around the time of the intervention and very few quality improvement projects report the sustainability more than a year out from the intervention. So, it'd be very important before you launch into an export of an intervention with the use of resources, time, and energy to ensure that any successful intervention is sustainable. >> Dr. Demetrius Kyriacou So, the intervention can be primarily targeted at clinicians instead of the patients?
>> Dr. Eddy Fan Yes. I would say that in a majority of quality improvement studies, the main intervention is targeted at changing clinician behavior or system behavior. So, the proper unit of analysis needs to be made. >> Dr. Demetrius Kyriacou How does one use an article about quality improvement in their own clinical setting? >> Dr. Eddy Fan So, once you've appraised a quality improvement study and found that it seems valid and it's something that you want to export to your site. As we mentioned context is very important.
Context is all those intangible things about your local environment where you're hoping to implement this new quality improvement intervention that includes your resources, the leadership, the culture of your practice, your institution, and those traditions that you really can't always account for in some kind of experimental study. And so, you really need to understand what the quality improvement study investigators did and the nature of their intervention and determine how exportable that is to your site. And often understanding your local barriers or facilitators to how that might be achieved are very important.
And so, you might think about local opinion leaders, what resources you have to bring to bear on implementing these things locally, may impact your ability to translate the findings in the study to your local environment. The best success is probably obtained given that these interventions are usually complex, multifaceted, and interprofessional in nature is to harness a team of people to help with implementation and that typically occurs at the level of administration, other clinicians, other professionals in the healthcare team that may be a part of that intervention to ensure the maximal likelihood of implementation and ultimately of sustainability.
>> Dr. Demetrius Kyriacou Dr. Fan, let's provide a clinical scenario for our listeners. You, the medical director, of an intensive care unit discover that mortality has increased for patients with sepsis. You are considering a quality improvement initiative to improve care and outcomes of your patients. However, you are concerned that are many quality improvement studies have weak designs, poor data quality, and often overestimate potential benefits. Before beginning, you decide to identify and evaluate existing quality improvement studies.
Dr. Fan, if you were the medical director of this intensive care unit, how would you approach this problem? >> Dr. Eddy Fan So, you begin the search for an answer by looking at the available medical literature and in fact, there is a recent before/after study looking at educational [inaudible] program for sepsis in 59 ICUs in Spain. This program used a program to train their clinicians to recognize and treat severe sepsis by implementing evidence based guidelines from the surviving sepsis campaign, looking specifically to treatment bundles.
Across these study ICUs, they found that implementing the bundles led to improved adherence to these bundle components over the 4 month implementation period. And importantly, hospital mortality also significantly decreased during the study intervention time. Unfortunately, looking a year after the intervention in a subset of ICUs, they found that adherence to the bundles had returned to baseline levels, but hospital mortality had remained similar to the post-intervention period. So, using the User's Guide to appraise the study, you find that the investigators decided to implement two interventions in the surviving sepsis campaign guidelines, that received strong recommendations as supported by the available medical literature.
So, it seems like processes of care that were important to implement. And the study did lead to increased adherence and a significant decline in hospital mortality. However, because of the study design being of uncontrolled before after study, it decreases your confidence in the potential causal association between the implementation of the quality improvement intervention and the observed outcomes. It's also a bit challenging because the intervention wasn't clearly defined.
There were no data on costs or any other unintended consequences, but the decrease in hospital mortality is compelling and if it was really the results of implementing this educational intervention, it suggests that any potential unintended consequences were actually likely to be small compared with the potential benefit. So, in the end using the User's Guide to assist in your appraisal of this study, although the confidence is low that the intervention actually improves mortality, the intervention itself being educational in nature has a relative low cost and a low potential for harm and unintended consequences.
And so, applying this educational intervention in your own hospital setting in concert with other clinicians, members of the interventional team, and your hospital administration may help to lead to improvements in patient important outcome. >> Dr. Demetrius Kyriacou Dr. Fan, do you have any other important points that you'd like to make regarding quality improvement? >> Dr. Eddy Fan I think it's important to recognize that as opposed to traditional clinical research, the vast majority of studies evaluating the effectiveness of quality improvement interventions is typically low and so it's just important for consumers of the medical literature to understand these potential limitations and the need for more robust study methods in quality improvement research.
>> Dr. Demetrius Kyriacou Thank you, Dr. Fan, for your comments and insights. More information on this topic is available in the User's Guide to the Medical Literature and on our website, jamaevidence.com, where you can listen to our entire roster of podcasts. Thanks for listening and we'll be back soon with another episode of JAMAevidence. [ Music ]
Form Title Download
Media
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html audio player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the audio player are generally tabs, one of which shows a scrolling transcript of the audio and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the audio player is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the audio by opening up an explore dialog which contains an about tab which contains rich metadata about the media, a segments tab which contains further information about the audio, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
E: Explore Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Form Title Embed Media
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the media.
You can press A to open the Accessibility Dialog.
This describes many other keyboard shortcuts for media controls.
Exit Clip Mode
The video is currently set-up to show just a clip from 0:00 to video end. You have requested to see a portion of the media outside of the clip.
Click OK to switch to the full media or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode