Enchancing Appreciation of Evidence-Based Science
If you like to read studies published in professional journals then you probably understand that a drug, vaccine or new treatment under consideration may be given to one group of people (the exposed group) while being withheld from another similar group of people (the control group). The two groups of people are then compared to determine if the drug or treatment had an effect, perhaps a reduction in pain or the likelihood of contracting a disease. Some people who read studies are content to simply know the gist of what the authors of the paper concluded — the treatment appears to be effective, for example — without understanding the statistical methods used to quantify the results. However, study outcomes can be expressed several ways including odds ratio, relative risk, and absolute risk. Confidence intervals and p-values are often provided as well. Although these terms may seem intimidating to the casual reader or anyone who has math anxiety, they are not difficult to grasp and will enhance your appreciation of significant findings in new studies.
Odds and Odds Ratio (OR):
The odds of something occurring, such as contracting a disease or being hospitalized, are the number of times the event or outcome happens divided by the number of times it does not happen within a group. (The probability of something occurring is calculated by dividing the number of times an event happens by the total number of possible outcomes.) For example, in a recent study of influenza-related hospitalizations, of 34 children who were diagnosed with laboratory-confirmed influenza after receiving the trivalent inactivated influenza vaccine (TIV), 11 required hospitalization for their severe illnesses. Thus, the odds that a TIV-vaccinated child will be hospitalized after being diagnosed with influenza are 11/23 (the number of times the event happened divided by the number of times it did not happen). Of 226 children who did not receive TIV prior to being diagnosed with laboratory-confirmed influenza, 26 required hospitalization — odds of 26/200.
An odds ratio (OR) measures the odds that an event or outcome will occur following a particular exposure, compared to the odds of the event or outcome happening without that exposure. Odds ratios can be calculated from cross-sectional and case-control study designs. In the example above, the exposure refers to trivalent inactivated influenza vaccination and the event is hospitalization. To determine the odds ratio we must divide 11/23 (.47826) by 26/200 (.13), which yields an OR of 3.67 (rounded down in the study). If the OR = 1, then there is no difference in risk between the two groups. If the odds ratio is less than 1, then there is less risk in the exposed group compared to the unexposed group. If the odds ratio is greater than 1 (as in the example), then there is greater risk in the exposed group compared to the unexposed group.
With a confidence interval (CI) ranging from 1.6 to 8.4 and a p-value of 0.003 [odds ratio (OR), 3.67; 95% CI, 1.6, 8.4; p = 0.003] the results may be considered highly significant. (Statistical significance is assumed if a) the confidence interval does not include the number 1 — both numbers must be below 1 or above 1 — and b) the p-value is no greater than 0.05.) This means that we can be very confident that when children are diagnosed with influenza, those who were previously vaccinated with TIV have a substantially elevated risk of hospitalization when compared to children who did not receive the vaccine. In fact, according to the authors of the paper, "Trivalent inactivated influenza vaccine did not provide any protection against hospitalization in pediatric subjects, especially children with asthma. On the contrary, we found a threefold increased risk of hospitalization in subjects who did get trivalent inactivated influenza vaccine" .
Note: In every experiment comparing an exposed group and a control group there is a possibility that there is no difference between the groups: the drug or treatment may have no effect. This presumed lack of a difference between groups is called the null hypothesis. P-values measure how likely the sample data support the null hypothesis despite findings in the study that might appear to show an effect. A low p-value provides strong evidence that the sample population in the study accurately reflects the full population and the null hypothesis can be rejected.
Confidence intervals consist of a range of values that are likely to contain the true odds ratio or relative risk. They provide us with an estimate of the precision of the results, given a particular level of confidence. The desired level of confidence is set by the researcher, usually at 95% or 99%, yielding significance levels of 0.05 or .01, respectively. In a study with a p-value of 0.01, we can be 99% certain that there is a true relationship between a particular exposure and an outcome but there is a 1% chance that the "significant" finding doesn't actually exist.
Relative Risk or Risk Ratio (RR):
The relative risk or risk ratio (RR) is the risk of an event in an experimental group divided by the risk in a control group. It can also be thought of as the ratio of the probability of an outcome in an exposed group to the probability of the outcome in a comparison, non-exposed group. Relative risks can be calculated from prospective and retrospective cohort studies, as well as randomized controlled trials. For example, in a recent study assessing whether vitamin D supplementation can prevent influenza in school children, 18 of 167 children who received 1200 IU of vitamin D daily through winter contracted influenza whereas 31 of 167 children who received a placebo contracted the disease. To determine the relative risk we must divide 18/167 (10.8%) by 31/167 (18.6%), which yields an RR of 0.58. Since the relative risk is less than 1, there is less risk in the exposed (vitamin D supplementation) group compared to the unexposed (placebo) group.
To calculate how much the relative risk declined due to the vitamin D treatment, simply subtract the relative risk (0.58) from 1, which yields 0.42 or a 42% reduction in the risk of influenza. With a confidence interval ranging from 0.34 to 0.99 and a p-value of 0.04 [relative risk (RR), 0.58; 95% CI: 0.34, 0.99; p = 0.04] the results are significant. According to the authors, "This study suggests that vitamin D3 supplementation during the winter may reduce the incidence of influenza A, especially in specific subgroups of school children" .
Absolute Risk Reduction (ARR) and Number Needed to Treat (NNT):
Study results are often reported as relative risk (RR) reductions rather than absolute risk reductions (ARR). This can make some treatments seem more beneficial than they actually are. The absolute risk of a disease is your true risk of developing it over a certain time period. For example, the study above showed that about 18.6% of school children will contract a case of influenza during winter. However, if they are supplemented with vitamin D then only about 10.8% will get the disease. By subtracting 10.8% (the treatment group event rate) from 18.6% (the control group event rate) we can determine that the absolute risk reduction is 7.8% (.078).
The inverse of the absolute risk reduction is the Number Needed to Treat (NNT), which is the number of patients that must be treated to prevent one additional bad outcome. To determine the number needed to treat to prevent one case of influenza per winter season, we must divide 1 by the ARR (1/.078) which equals 12.8, and we round up to 13. Although there was a 42% reduction in relative risk, there was a 7.8% reduction in absolute risk and about 13 school children need to receive vitamin D supplementation to prevent one case of influenza. In large populations, this can have a tremendous effect. For example, if one million school children are supplemented with vitamin D we may be able to prevent more than 75,000 cases of influenza!
Commonly used statistical measures are tools to comprehend the true value of new treatments and significant findings in published studies. For most clinical trials, especially those in which the event is rare, odds ratio and relative risk can be interchanged as a measure of the relative risk of an event. However, odds ratios are not equivalent to relative risks because they are a ratio of odds; relative risk is a ratio of probabilities. Absolute risk reduction is the difference between the treatment group event rate and the control group event rate expressed as a percentage. Its inverse is the number of patients that need to be treated to prevent one additional bad outcome. Confidence intervals provide an estimate of the precision of the results while small p-values add credibility to the significance or correctness of study findings. Other statistical measures such as hazard ratios and rate ratios are utilized by scientists when necessary, but by mastering the few statistical terms described in this paper, your understanding and appreciation of evidence-based science will be greatly enhanced.
by Neil Z. Miller
The Children’s Medical Safety Research Institute (CMSRI) is a medical and scientific collaborative established to provide research funding for independent studies on causal factors underlying the chronic disease and disability epidemic.