Evidence Based Medicine
Evidence based medicine refers to the practice of basing medical decisions on experimental data rather than other factors such as a physician’s personal preference or popularity of a certain treatment in a particular geographic area. Because this type of evidence based medicine involves comparison of the risks and benefits of different medical treatments, it requires understanding of probability to make informed decisions. Unfortunately, probability and statistics are not something that is well understood, even by most of us doctors.
Keep in mind that the cost benefit equation is not necessarily the same for the patient and the doctor. If a plane crashes, the pilot will likely die as well as the passengers. This is not the case with the doctor patient relationship. This may contribute to the fact that aviation safety has developed checklists and safety systems that have until recently been absent from most hospitals and doctor’s offices.
Therefore, it is important that you as the patient understand how to ask the right questions and correctly evaluate the answers, so that you may work as a team with your doctor to preserve your health and quality of life.
How We Phrase the Question Matters
For example, consider the examples below taken from the wonderful book Reckoning with Risk by Gerd Gigerenzer (2002). He gave the following problem to 48 physicians in a variety of specialties ranging from radiologists to dermatologists. Out of the 48 physicians, only 2 could come up with a reasonably correct answer:
The probability that one of these women has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent that she will still have a positive mammogram. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?
Gigerenzer recommends rewriting the problem in frequencies expressed as people per population rather than percent probability as a tool to make it easier to think about:
Eight of every 1000 women have breast cancer. Of these 8 women with breast cancer, 7 will have a positive mammogram. Of the remaining 992 women who don’t have breast cancer, some 70 will still have a positive mammogram. Imagine a sample of women who have positive mammograms in screening. How many of these women actually have breast cancer?
Phrased this way, it is easier to see that only 7 of the 77 women who test positive (70+7) actually have breast cancer, which is 1 in 11 or 9 percent (below, left) . A much easier calculation than the probability method (below, right).
It is alarming to note that when Gigerenzer gave this problem to 48 physicians, the median estimate of how many people actually had breast cancer given a positive screening test was 70 percent, quite a large difference from the correct answer of 9 percent. This is especially important because it is on the basis of probabilities such as these that treatment decisions are made.
Relative Risk Reduction Versus Absolute Risk Reduction
If a treatment reduces the number of people who lose vision from a particular eye disease from 6 to 4 in every 1000 people, then the relative risk reduction is 33.3 percent (6 divided by 4). However, the absolute risk reduction is 2 in 1000, or 0.2 percent. Reporting risk reduction in relative terms is popular because the numbers look bigger, but it doesn’t take into account the actual risk involved. For example if the treatment reduces the number of people who lose vision from 6 to 4 in 10,000, the relative risk reduction is still 33.3%, but now the absolute risk reduction is only 0.02%. This is why it is important to know the overall prevalence of a disease in the population you are considering, because as the disease becomes rarer, the same relative risk reduction of a treatment reduces your absolute risk by less.
Sensitivity Versus Specificity
Sensitivity of a test is the percentage of people who actually have a disease that will test positive for the disease when given the test. Basically, how “sensitive” is the test at picking up the disease. As doctors, we are very interested in sensitivity of a test because we do not want to miss any case of a disease such as glaucoma or cancer where treatment is usually more effective if the disease is caught early. Add to this the possible legal ramifications of a missed diagnosis in American society, and our fee-for-services model of health care in this country, and it is clear that there are strong incentives for your doctor to perform testing with high sensitivity. However, as the sensitivity of a test is increased, and we get closer to diagnosing everyone with a disease, we increasingly misdiagnose people without the disease as having the disease. This is where the specificity of the test comes in, as described below.
Specificity of a test is the percentage of people who don’t have the disease the test is looking for that will test negative for the disease when given the test. Basically, how good is the test at correctly identifying people who don’t have disease as not having the disease, and not incorrectly identifying them as having the disease. This is an important aspect of medical testing for patients to ask their doctors about because an incorrect positive test result for someone, who in reality does not have the disease being tested for, can have emotional, financial, and physical costs from treatment or further testing for a disease that is not actually present.
Positive Predictive Value
The positive predictive value of a test tells you how likely it is that you actually have a disease in the case that you have a positive result on a test. The positive predictive value is found by dividing the number of people that the test correctly identifies as having the disease by the total number of people the test says have the disease (correct or not).
Working as a Team With Your Doctor
As doctors we bring many years of training and experience in medical diagnosis and treatment to the equation, and it is not always easy to discuss the uncertainty inherent in medical diagnosis and treatment with us. Nor do we always like to admit the degree of uncertainty that exists in diagnosis and treatment. However, you as the patient, are the one who has to live with the choices made. So, it is important you understand the risks and benefits of these decisions, and that they are made on the basis of the best available evidence.
Below are the types of questions you and your doctor should discuss relating to evidence based medicine when considering medical testing and treatment. This is not necessarily meant to be a script for you to talk with your doctor, but is something we use to guide the conversation here at Portland Eye Care. Ideally, the answers to these questions should be phrased in frequencies (i.e. 4 people in every 1000).
Among 1000 people like me, how many of them will have this disease? (prevalence)
If I do have the disease, what are the chances that the test will correctly identify me as having it? (sensitivity)
If I don’t have the disease, what are the chances that the test will correctly identify me as not having it? (specificity)
If I get a positive test result, what are the chances that I actually have the disease? (positive predictive value)
Resources for Further Research
Cochrane Library The Chocrane Library is a collection of reviews of the available evidence based medicine literature on a particular disease or question. The wonderful thing about these reviews is they have been created by a team that evaluates not only the results of studies, but also the quality of the studies themselves. They also include a plain language summary. If you click on the Cochrane Library link at cochrane.org it will take you to a page that has summaries of the Chochrane reviews, if the summary doesn’t include enough information for you, the articles are available for purchase, but you can also usually access them for free from an academic or public library that subscribes to the site.
Pubmed.Gov This is a database of research articles maintained by US National Library of Medicine National Institutes of Health. It is great evidence based medicine resource in that you can find research articles on most any topic. The downside is that the articles are in their raw form and will be up to you to interpret, both in their results and the quality of the research itself.
National Eye Institute The website of the national eye institute contains many articles and information specific to eye health as well as information on ongoing clinical studies. It is also a great source for evidence based medicine statistics to find out how prevalent a certain condition is within a group of people.
Interpreting Medical Statistics A guide by Dr. Adelman on this website explaining P-Values and Confidence Intervals, both of which come up in most scientific articles published.