Cadmore media player playing audio Surprising Results of Randomized Trials: Gordon Guyatt, MD, surprising results of randomized trials.
Audio Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the audio. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Transcript
Result Count
of
Search
Segments
Result Count
of
Search
Name:
Surprising Results of Randomized Trials: Gordon Guyatt, MD, surprising results of randomized trials.
Description:
Surprising Results of Randomized Trials: Gordon Guyatt, MD, surprising results of randomized trials.
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/10cdaa80-8f77-4696-874a-9627698c3efc/thumbnails/10cdaa80-8f77-4696-874a-9627698c3efc.jpg?sv=2019-02-02&sr=c&sig=s9kIQXmDhxUVTiKNEXhEc%2FEVoWe6%2Ft21F92HQXiQ584%3D&st=2025-01-15T12%3A36%3A35Z&se=2025-01-15T16%3A41%3A35Z&sp=r
Duration:
T00H10M38S
Embed URL:
https://stream.cadmore.media/player/10cdaa80-8f77-4696-874a-9627698c3efc
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/10cdaa80-8f77-4696-874a-9627698c3efc/6830503.mp3?sv=2019-02-02&sr=c&sig=xWy7zFbNlXc1ICsEEaY8O8%2FKhjgVb23rJ3qgL68OYkU%3D&st=2025-01-15T12%3A36%3A35Z&se=2025-01-15T14%3A41%3A35Z&sp=r
Upload Date:
2022-02-28T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
>> I'm Joan Stephenson, Editor of JAMA's Medical News and Perspectives Section. Today I'll be talking again with Dr. Gordon Guyatt, this time on the issue of surprising results of randomized trial. This is a topic he covers in Chapter 9.2 of Users' Guides to the Medical Literature. Dr. Guyatt, why don't you introduce yourself to our listeners? >> Hi, I'm Gordon Guyatt. I practice as a Secondary Care Internist in Hamilton, Ontario, and I'm a Professor of Medicine at McMaster University. >> Dr. Guyatt, why do surprising results occur with randomized control trials?
>> So there are some major reasons. I'll highlight two of them. One is we sometimes think as scientists we understand biology very well, and we don't understand the biology as well as we think we do. So we make predictions about how things will happen, and they don't happen that way. Let me give you one example. Years ago, cardiologists noted that antiarrhythmic drugs in acainide and flecainide decreased, virtually obliterated warning ventricular arrhythmias, which were associated with sudden death after myocardial infarction.
The cardiologists were sufficiently convinced that eliminating the warning arrhythmias would eliminate the lethal arrhythmias, that they persuaded the Food and Drug Administration to license the drugs, which were widely distributed. Randomized trials were subsequently done, which unfortunately showed that the drugs increased rather than decreased arrhythmic deaths. So here the biology seemed compelling, but things worked exactly the opposite of what they were supposed to.
And it's estimated that more Americans died of these drugs than died in the Vietnam War, showing just how wrong things can be if we get too confident about our understanding of biology. A second major reason randomized trials may surprise us is that we see individuals who take a particular medication or in the example I'll give, a vitamin, and they do better. And the natural thing is to attribute it to the intervention when that may not be the case at all.
So for instance, antioxidant vitamins in observational studies have been associated with decreased cancer and decreased cardiovascular events. When randomized trials were done, unfortunately no beneficial effect. And the reason is the people who took them did better, wasn't the antioxidant vitamins at all, it was that people destined to do better, healthier people, people who took care of themselves, people with higher socioeconomic status, various things associated were doing well, they were the people who took antioxidant vitamins.
So those are two major reasons our lack of understanding of biology and naive interpretation of observational studies that lead to surprising results of randomized trials. >> In what ways might evidence be weak? >> Evidence might be weak for a variety of reasons. One as we've just highlighted, is that study designs may be weaker. So observational studies are weaker than randomized trials. But even in randomized trials, there may be limitations, small sample sizes, the results of one study may contradict the results of other studies, decreasing our confidence in the estimates.
And the studies that are done may not be applicable to the population. So for instance, we have older and older people in our society, 85 and 95-year-olds, and studies done in younger people may not be applicable. So there's a number of reasons why we may be less confident in results of which the theme of this one is study design. >> What are some examples of randomized control trials refuting conclusions from non-human studies?
>> Well, there's a whole series of pre-experimental work in cancer for instance, where in cell cultures and laboratory experimental situations, drugs appeared very effective in terms of their impact on neoplastic cells, making the investigators think that they were very likely to work in randomized trials. When randomized trials were done, unfortunately no beneficial effect.
It also it happens when biological organisms. For instance, there was drugs that appeared to be very effective against the herpes zoster virus that ultimately proved completely ineffective when tested in human beings. So again, the generalization from the laboratory situation to the clinical situation is something we have to be very careful about. >> In randomized control trials, what circumstances can make surrogate endpoints misleading?
>> So by surrogate endpoints, we mean endpoints that are associated with things that are important to patients, but are not themselves important to patients. So for instance, in people with diabetes, high blood glucose is associated with microvascular complications, including problems with eyes, peripheral neuropathy, and kidneys, and macrovascular complications like stroke and myocardial infarction. And because of the association, that is lower blood glucose, less of these bad events, it is natural to assume that if you lower the blood glucose, you will decrease the incidence of those events.
Unfortunately, often that connection does not work out. And diabetes is one where in a variety of situations we have lowered blood glucose without any beneficial effects on the micro and macrovascular complications of diabetes. In a second example HDL, the good cholesterol, is associated with lower levels of coronary risk. And there's no doubt, the higher your HDL, the lower your coronary risk.
When people have introduced drugs however that increase the HDL very nicely, no benefits in terms of coronary risk, and on one occasion a increase in coronary events. Once again, we have an understanding of the biology, and we have an association between the surrogate and the outcome, but it doesn't always mean that changing the surrogate will change the outcome. >> Why should clinicians be wary of the results of observational studies when making therapeutic decisions?
>> Well, I've already mentioned the example of antioxidant vitamins and cardiovascular risk and cancer, and they illustrate the general phenomenon of why commissions should be wary of observational studies. And that is, that apparent associations may not be causal and may be a result of prognostic differences. So with the antioxidant vitamin example, people who are healthier or have more health-inducing activities, or have higher socioeconomic status, which we know is highly associated with better outcomes, are the ones who take antioxidant vitamins.
In guidelines that were promulgated and widely followed, we gave out hormone replacement therapy to women for many years or encouraged them to take it on the basis of reducing cardiovascular risk from observational studies. As it turns out, it isn't that hormone replacement therapy lowers cardiovascular risk, it's that women destined to do better in terms of cardiovascular events were the ones who took the hormone replacement therapy. So the problem with any of these observational studies is that the people who use the drugs may be intrinsically different than the people who don't, and that may be related to the prognosis.
So people do better with the drugs, but it turns out that's not because of the drugs, it was because the people who took the drugs were destined to do better. >> And of course, the way to tease that out is to do a randomized control trial. >> That's right. In all the examples that I've given here, we've found out about the truth by doing randomized trials, which illustrates the point that at the moment in the comparative effectiveness initiatives, there are folks who believe that we can rely on the observational studies.
Sometimes they work out right, but sometimes they don't. And if we never do the randomized trials in the examples that I'm using, for instance if we'd never done the randomized trials of flecainide and acainide, people would still be dying of lethal arrhythmias. If we hadn't done the randomized trials of antioxidant vitamins, we'd still be handing out the vitamins to people, and certainly with the lipid soluble ones, they've been associated with substantial adverse effects.
So unless we do the randomized trials, we're going to get caught. >> Is there anything else you would like to tell our listeners about surprising results of randomized trials? >> Well, just that they should not come away thinking that any randomized trial answers the question. So randomized trials may not measure the right outcome. For instance, if they're measuring surrogates as problematic, they sometime yield a non-reproducible result. So you shouldn't jump at the first randomized trial.
Wait until there's a solid body of evidence. And if the trials are small, they're also unreliable. So randomized trials are the gold standard, but we need large, reproducible randomized trials that focus on the right outcomes. >> Thank you, Dr Guyatt, for your insights into surprising results from clinical trials. For additional information about this topic, JAMAevidence subscribers can consult Section 2 of Chapter 9 in Users' Guides to the Medical Literature.
This has been Joan Stephenson of JAMA, talking with Dr. Gordon Guyatt for JAMAevidence.
Segment:0 .
>> I'm Joan Stephenson, Editor of JAMA's Medical News and Perspectives Section. Today I'll be talking again with Dr. Gordon Guyatt, this time on the issue of surprising results of randomized trial. This is a topic he covers in Chapter 9.2 of Users' Guides to the Medical Literature. Dr. Guyatt, why don't you introduce yourself to our listeners? >> Hi, I'm Gordon Guyatt. I practice as a Secondary Care Internist in Hamilton, Ontario, and I'm a Professor of Medicine at McMaster University. >> Dr. Guyatt, why do surprising results occur with randomized control trials?
>> So there are some major reasons. I'll highlight two of them. One is we sometimes think as scientists we understand biology very well, and we don't understand the biology as well as we think we do. So we make predictions about how things will happen, and they don't happen that way. Let me give you one example. Years ago, cardiologists noted that antiarrhythmic drugs in acainide and flecainide decreased, virtually obliterated warning ventricular arrhythmias, which were associated with sudden death after myocardial infarction.
The cardiologists were sufficiently convinced that eliminating the warning arrhythmias would eliminate the lethal arrhythmias, that they persuaded the Food and Drug Administration to license the drugs, which were widely distributed. Randomized trials were subsequently done, which unfortunately showed that the drugs increased rather than decreased arrhythmic deaths. So here the biology seemed compelling, but things worked exactly the opposite of what they were supposed to.
And it's estimated that more Americans died of these drugs than died in the Vietnam War, showing just how wrong things can be if we get too confident about our understanding of biology. A second major reason randomized trials may surprise us is that we see individuals who take a particular medication or in the example I'll give, a vitamin, and they do better. And the natural thing is to attribute it to the intervention when that may not be the case at all.
So for instance, antioxidant vitamins in observational studies have been associated with decreased cancer and decreased cardiovascular events. When randomized trials were done, unfortunately no beneficial effect. And the reason is the people who took them did better, wasn't the antioxidant vitamins at all, it was that people destined to do better, healthier people, people who took care of themselves, people with higher socioeconomic status, various things associated were doing well, they were the people who took antioxidant vitamins.
So those are two major reasons our lack of understanding of biology and naive interpretation of observational studies that lead to surprising results of randomized trials. >> In what ways might evidence be weak? >> Evidence might be weak for a variety of reasons. One as we've just highlighted, is that study designs may be weaker. So observational studies are weaker than randomized trials. But even in randomized trials, there may be limitations, small sample sizes, the results of one study may contradict the results of other studies, decreasing our confidence in the estimates.
And the studies that are done may not be applicable to the population. So for instance, we have older and older people in our society, 85 and 95-year-olds, and studies done in younger people may not be applicable. So there's a number of reasons why we may be less confident in results of which the theme of this one is study design. >> What are some examples of randomized control trials refuting conclusions from non-human studies?
>> Well, there's a whole series of pre-experimental work in cancer for instance, where in cell cultures and laboratory experimental situations, drugs appeared very effective in terms of their impact on neoplastic cells, making the investigators think that they were very likely to work in randomized trials. When randomized trials were done, unfortunately no beneficial effect.
It also it happens when biological organisms. For instance, there was drugs that appeared to be very effective against the herpes zoster virus that ultimately proved completely ineffective when tested in human beings. So again, the generalization from the laboratory situation to the clinical situation is something we have to be very careful about. >> In randomized control trials, what circumstances can make surrogate endpoints misleading?
>> So by surrogate endpoints, we mean endpoints that are associated with things that are important to patients, but are not themselves important to patients. So for instance, in people with diabetes, high blood glucose is associated with microvascular complications, including problems with eyes, peripheral neuropathy, and kidneys, and macrovascular complications like stroke and myocardial infarction. And because of the association, that is lower blood glucose, less of these bad events, it is natural to assume that if you lower the blood glucose, you will decrease the incidence of those events.
Unfortunately, often that connection does not work out. And diabetes is one where in a variety of situations we have lowered blood glucose without any beneficial effects on the micro and macrovascular complications of diabetes. In a second example HDL, the good cholesterol, is associated with lower levels of coronary risk. And there's no doubt, the higher your HDL, the lower your coronary risk.
When people have introduced drugs however that increase the HDL very nicely, no benefits in terms of coronary risk, and on one occasion a increase in coronary events. Once again, we have an understanding of the biology, and we have an association between the surrogate and the outcome, but it doesn't always mean that changing the surrogate will change the outcome. >> Why should clinicians be wary of the results of observational studies when making therapeutic decisions?
>> Well, I've already mentioned the example of antioxidant vitamins and cardiovascular risk and cancer, and they illustrate the general phenomenon of why commissions should be wary of observational studies. And that is, that apparent associations may not be causal and may be a result of prognostic differences. So with the antioxidant vitamin example, people who are healthier or have more health-inducing activities, or have higher socioeconomic status, which we know is highly associated with better outcomes, are the ones who take antioxidant vitamins.
In guidelines that were promulgated and widely followed, we gave out hormone replacement therapy to women for many years or encouraged them to take it on the basis of reducing cardiovascular risk from observational studies. As it turns out, it isn't that hormone replacement therapy lowers cardiovascular risk, it's that women destined to do better in terms of cardiovascular events were the ones who took the hormone replacement therapy. So the problem with any of these observational studies is that the people who use the drugs may be intrinsically different than the people who don't, and that may be related to the prognosis.
So people do better with the drugs, but it turns out that's not because of the drugs, it was because the people who took the drugs were destined to do better. >> And of course, the way to tease that out is to do a randomized control trial. >> That's right. In all the examples that I've given here, we've found out about the truth by doing randomized trials, which illustrates the point that at the moment in the comparative effectiveness initiatives, there are folks who believe that we can rely on the observational studies.
Sometimes they work out right, but sometimes they don't. And if we never do the randomized trials in the examples that I'm using, for instance if we'd never done the randomized trials of flecainide and acainide, people would still be dying of lethal arrhythmias. If we hadn't done the randomized trials of antioxidant vitamins, we'd still be handing out the vitamins to people, and certainly with the lipid soluble ones, they've been associated with substantial adverse effects.
So unless we do the randomized trials, we're going to get caught. >> Is there anything else you would like to tell our listeners about surprising results of randomized trials? >> Well, just that they should not come away thinking that any randomized trial answers the question. So randomized trials may not measure the right outcome. For instance, if they're measuring surrogates as problematic, they sometime yield a non-reproducible result. So you shouldn't jump at the first randomized trial.
Wait until there's a solid body of evidence. And if the trials are small, they're also unreliable. So randomized trials are the gold standard, but we need large, reproducible randomized trials that focus on the right outcomes. >> Thank you, Dr Guyatt, for your insights into surprising results from clinical trials. For additional information about this topic, JAMAevidence subscribers can consult Section 2 of Chapter 9 in Users' Guides to the Medical Literature.
This has been Joan Stephenson of JAMA, talking with Dr. Gordon Guyatt for JAMAevidence.
Form Title Download
Media
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html audio player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the audio player are generally tabs, one of which shows a scrolling transcript of the audio and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the audio player is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the audio by opening up an explore dialog which contains an about tab which contains rich metadata about the media, a segments tab which contains further information about the audio, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
E: Explore Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Form Title Embed Media
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the media.
You can press A to open the Accessibility Dialog.
This describes many other keyboard shortcuts for media controls.
Exit Clip Mode
The video is currently set-up to show just a clip from 0:00 to video end. You have requested to see a portion of the media outside of the clip.
Click OK to switch to the full media or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode