Name:
Understanding How to Use Likelihood Ratios in Clinical Practice: A Conversation With the Rational Clinical Examination Editor, David L. Simel, MD, MHS
Description:
Understanding How to Use Likelihood Ratios in Clinical Practice: A Conversation With the Rational Clinical Examination Editor, David L. Simel, MD, MHS
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/ee050aeb-4d29-414e-8613-9db0572550ab/thumbnails/ee050aeb-4d29-414e-8613-9db0572550ab.jpg?sv=2019-02-02&sr=c&sig=dunvoRiLtUrO%2BZFpQTAiObmmbGWvqg6CLeG5bKsA9aA%3D&st=2023-09-21T08%3A13%3A14Z&se=2023-09-21T12%3A18%3A14Z&sp=r
Duration:
T00H14M47S
Embed URL:
https://stream.cadmore.media/player/ee050aeb-4d29-414e-8613-9db0572550ab
Content URL:
https://asa1cadmoremedia.blob.core.windows.net/asset-110d49ef-1c9b-4b7b-9829-9a38771b335f/13312144.mp3
Upload Date:
2022-02-23T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
>> Many clinicians don't really understand the concept of likelihood ratios, but they're really helpful when trying to use a clinical test to determine if a patient has a disease or not. To get a better understanding of the proper way to use likelihood ratios, we spoke to Dr. David Simel, Professor of Medicine at the Duke University School of Medicine and the Editor of the "Rational Clinical Exam Series". So let's take one of the features of the acute coronary syndrome that shows up in the table in the "Rational Clinical Exam" chapter on acute coronary syndrome that I would've thought would perform pretty well which is typical chest pain and typical, I guess, means typical for having acute coronary syndrome, but the positive likelihood ratio for that is only 1.9. So, Dr. Simel, could you explain that to us?
What does that number mean, and how do we use it clinically? >> Sure, Ed. Well, when you're looking at tests from the clinical exam, you want to look first at sensitivity and specificity. And you get some general idea of whether a test is a good test or bad test. There are many physicians who look at sensitivity and specificity and kind of stop from there at that point. If you look at typical chest pain, it has a sensitivity of 66%, and the specificity is actually the same also, 66%.
And so you look at those and you might think this isn't a very useful test. The problem is you don't really know what those values mean for sensitivity and specificity because you don't really know how to apply them to your individual patient. So, let me explain. The sensitivity only applies to the patients who have disease or, in this case, acute coronary syndrome. And the specificity only applies to those patients who don't have acute coronary syndrome.
So if you're sitting there trying to figure out whether or not your patient has acute coronary syndrome, you don't know which of these values apply. The likelihood ratio, though, gives you a way out. And so what first appears is this doesn't sound like a big number, 66% as opposed to, say, a toss-up of 50%, we see from a different perspective when we look at the likelihood ratio. Now if the likelihood ratio is positive, that means the likelihood ratio that applies to a patient who has typical chest pain is 1.9. Well, that just seems like an arbitrary number.
What does that 1.9 mean? And what that number means is that the odds increase 1.9 fold or two times compared to a patient who doesn't have typical chest pain for acute coronary syndrome. So the question then becomes, well, is that a lot? Is that good? And what people who use likelihood ratios tend to do, like those who use sensitivity and specificity, is sort of think good test/bad test. And the higher a positive number, the better a test is when it's positive.
The lower a number, the better a test is when it's absent or negative. So the question then becomes is 1.9 or 2 a large number? And I guess I'll ask you, Ed. What do you think? >> So I'm influenced by my reading that says that it's really a very powerful indicator if the positive likelihood ratio is more than 10. So, I would say, given that, that 1.9 is okay but not great. >> Well, so our listeners will think that I set you up to give me that response, and I really didn't but it is a perfect set up.
And if you look at the literature, there are many authors who have suggested exactly that. It needs to be greater than eight. It needs to be greater than 10, and on the negative side, it needs to be less than .1. It needs to be less than .2. And none of those perfectly capture the importance because the value of the likelihood ratio depends on where you start. So if you start with a low probability of disease, a positive likelihood ratio of eight or 10 is not necessarily going to be useful.
Let's think about acute coronary syndrome. About 10% of patients who come to the emergency room with chest pain are going to have an acute coronary syndrome. And if typical chest pain had a likelihood ratio of eight, that only gets you to a probability of 47%. So, really, almost kind of a toss-up. If you start though with a -- say a much higher number of pre-test probability, let's say a pre-test probability of 20%, so instead of 10% of patients, 20% of patients with chest pain are going to have an acute coronary syndrome, that likelihood ratio of eight gets you up to 67%.
And so now we're talking about bigger numbers here and you can see that the likelihood ratio of eight, its value really depends on where you start at the pre-test probability. >> So in this case, the likelihood ratio is close to two. >> Right. >> So how does that work? >> Well, in this case, the likelihood ratio of two doesn't get you real far. It gets you from a pre-test probability of 10% to 18%. So in thinking in the mnemonics that Dave Sackett taught us, if we go back to sensitivity and specificity, specificity is the result that we think about for a positive test and the mnemonic is SpPIn, S-P-P-I-N.
So if you have a high specificity test, a positive test tends to rule in. And a result of a likely ratio of two is just not high enough to really rule in. Going from 10% to 18% is just not something we can hang our hat on. And most people who see patients with chest pain know that typical chest pain isn't that great for figuring out who has acute coronary syndrome. But the reality is, is they might overestimate the importance of that and they might behave as if it had a likelihood ratio of much higher.
If they did, they'd be wrong. >> So you came up with those numbers at the top of your head pretty quickly. How does the average clinician do these calculations? >> Well, I think the average clinician doesn't have to do the calculations. You can just look at them and think good test/bad test. And you need to think about -- well, where am I starting from? What's the pre-test probability? And, you know, the pre-test probability for this is about 10%, 15%, 20% so kind of low. And then I realize that the higher a value, the higher a likelihood ratio, the better it is at ruling in.
And so as I go from two to three to four to five to six up to 10, I realize it's going to be increasingly important. Now in this particular case, what I'm really most interested in is figuring out who doesn't have an acute coronary syndrome. So here I'm most interested actually in lower values. And now by lower values, I mean values less than one. And in this case, as I think of tests having likelihood-- or the absence of typical pain having a likelihood ratio of .5, .4, .3, .2, the lower it is, the more valuable it is in ruling out or making an acute coronary syndrome less likely.
In this particular case, the absence of typical chest pain is a likelihood ratio of .52, so it has the odds. And the probability would be all the way down to 5% if we started at 10%. Now if I want to do the math, what I typically will do myself, as I'm reading an article, is I will do the math quickly in my head by rounding off. And what I've learned how to do is convert the pre-test probability to odds.
And I do that liberally with rounding off. So if I tell you that the pre-test probability is 80%, the odds of that we can intuit as being 4-to-1. And how do we do that? Well, there is an 80% probability of having disease and a 20% probability of not having disease so .8 divided by .2 is 4-to-1. And I can round off at 20% probability, so .2 divided by .8 is .25. Now, correct me, Ed, if I make a mistake in my head but that's quickly how we get the prior odds.
And then we just multiply that by the likelihood ratio and that gets us our post-test odds. Now most of us don't go to a patient and we don't say I'll lay 5-to-1 odds on you that you have a disease. We might do that at the race track, but we don't do that with patients. So I quickly do the math in my head, and I convert back to a probability. And if I'm rounding off, what I do to go from odds to probability is it's odds over 1-plus odds.
So rather than go to a patient and say I'll lay 5-to-1 odds on you that you're having acute coronary syndrome, it's 5 divided by 1 plus 5 or 5/6 and that's a pretty high probability of disease. So I've just learned to quickly do them in my head. But in the end, after I've done them, I sort of think good test/bad test for my clinical situation. >> So what's interesting about this particular "Rational Clinical Exam" article on acute coronary syndrome is that none of these tests that were taught are classic presentations for cardiac disease work very well.
Is that correct? >> Well, they don't work particularly well in isolation. It's a problem because we can overestimate how good a test is. So for example, if a patient has crushing chest pain, we can overestimate the importance of that and too frequently go to an approach to ruling out an acute coronary syndrome when maybe everything else goes against it. And what we do is -- to overcome that is we look to see if there are combinations of signs that might be better.
And, in fact, there are some combinations of signs that are better. And those combinations have a variety of eponyms or acronyms. And two of the best are the TIMI Score, T-I-M-I and the HEART Score, H-E-A-R-T. I think most of your listeners will have heard about the TIMI Score. Maybe fewer non-cardiologists will have heard of the HEART Score but these have much better discriminatory value for detecting the person at higher risk or lower risk.
And they can also stratify to figure out the person with intermediate or indeterminate risk. >> So the numbers in the paper are somewhat confusing. I don't -- at least the draft that I have, there is a likelihood ratio for HEART and TIMI for high risk is 13 for HEART, 6.8 for TIMI. The low is .2. Are those positive likelihood ratios in that table? >> Yeah. So when we're using a combination score that has various levels, the likelihood ratios are not what we call likelihood ratio positive/likelihood ratio negative.
It's just a likelihood ratio associated with the result of the test. So a likelihood ratio of 6.8 for the TIMI Score is a likelihood ratio for a high-risk TIMI which would be a TIMI Score of five to seven. Now, the TIMI Score can go from zero to seven. So the likelihood ratio for a TIMI Score of zero to one is 0.31. And we can give you the TIMI Score for increments up to five to seven.
So there's just one likelihood ratio associated with the particular TIMI result. >> Okay. So this likelihood ratio then means if you have a score in this range, it's the likelihood of having the disease. It's not for ruling it out. >> That's right. And that's one of the nice things about likelihood ratios is that no matter whether you're talking about a positive test or a negative test or a test on some ordinal scale, it always is the likelihood rate, the likelihood of having disease.
So as it gets greater than one, there is an increased likelihood of disease. As it becomes less than one and approaches zero, there is a lower likelihood of having disease. >> I see. So the way it plays out for this is if you have a TIMI of five to seven, your likelihood of having an acute coronary syndrome, the likelihood ratio is 6.8. >> That's right and so it goes up a lot. >> Yeah. And then on the other end of the scale, zero to one, the likelihood ratio is .3. >> Yeah. So it goes down a lot and that's useful because if we start with our pre-test probability of 10 and you've got a low-risk TIMI Score of zero to one, the likelihood ratio for that is .31.
So you go from a 10% probability of having an acute coronary syndrome down to a 3.4% probability. And, routinely, admitting 100% of those patients is going to mean that 97% of the time you didn't need to do that. That's a lot of resources to spend on someone who's very low risk and so maybe we should be sending those patients home from the emergency room rather than keeping them for 24 hours to see what happens.
>> This is Ed Livingston, Deputy Editor of "Clinical Reviews and Education" for JAMA. I have been speaking to Dr. David Simel, the Editor of the "Rational Clinical Exam Series". We discussed the use of likelihood ratios, sensitivity, and specificity as applied to clinical decision-making. The bottom line is that sensitivity and specificity refer to tests themselves and how often the test is positive or negative when a patient does or doesn't have the disease. However, sensitivity and specificity tell you little about how a specific sign, symptom, or lab value relates to the presence or absence of a disease in a patient for whom you don't know whether they do or don't have the disease.
When seeing patients in real life and trying to determine what's wrong with them, likelihood ratios are much more useful than sensitivity or specificity. There is an entire library of podcasts that help explain the various concepts found in "JAMAevidence". Look for them at jamaevidence.com. Please provide feedback to me regarding these podcasts using my Twitter handle @ehljama.