In day to day language, efficacy and effectiveness are essentially interchangeable. If you google the definition of efficacy you get the following: the ability to produce a desired or intended result. Do the same for effectiveness and you will find: the degree to which something is successful in producing a desired result. Remarkably similar, right? However when discussing research, effectiveness and efficacy represent two distinct scientific approaches that require precision in language. The difference between the two lies in what question the trial aims to answer and how it is designed.
An efficacy trial (sometimes known as an explanatory trial) is concerned with the question “does the studied intervention have an actual effect in a controlled setting?” To answer this question confidently, the trial must be designed in a way that maximizes internal validity. To achieve this, the design must have strict inclusion/exclusion criteria to obtain a homogenous sample, a rigid treatment protocol that is delivered in a standardized manner and a no-treatment and/or placebo group to serve as a control. With these design parameters, you are best able to reduce noise in order to detect a treatment’s signal (or lack of thereof). However, to achieve the level of internal validity needed to demonstrate efficacy, you must sacrifice some external validity. It can be helpful to imagine internal validity and external validity as a see-saw. As one rises, the other must come down. The low levels of external validity typical of efficacy trials make it difficult to know whether if and how a treatment’s effect will generalize to actual clinical practice.
A good example of an efficacy trial is Dr. Lorimer Moseley’s “Graded motor imagery (GMI) is effective for long-standing complex regional pain syndrome: a randomized controlled trial.” Dr. Moseley studied a very specific population, those with upper limb CRPS1 following a wrist fracture, with a tightly regimented protocol that was delivered in a standardized manner by a single clinician (himself). The experimental group was compared to what was essentially a waiting-list control group to best capture the effect of GMI. What Moseley’s trial demonstrated is that a GMI program is efficacious for those with CRPS1 following a wrist fracture. However, these results may not generalize to someone who has CRPS1 of the foot, or someone with CRPS1 following a cerebrovascular accident. The effect might also be altered if the program is not performed with the same frequency, or if the program is delivered by someone not as well trained in the protocol as Dr. Moseley.
An effectiveness trial (sometimes known as a pragmatic trial) is concerned with determining the effect of an intervention in an environment that resembles true clinical practice. This means accounting for variables such as adherence, availability of resources, differences between clinicians, patient variability and other factors. To do so, these pragmatic trials must be designed to maximize external validity. Typical characteristics of a trial with high external validity include broad inclusive criteria, minimal restrictions placed on clinicians treating within the trial, the allowance for multimodal treatments and the use of several different therapists in different clinical environments. In effectiveness trials, the outcome of the experimental intervention is often compared to the current standard of care rather than a placebo or no treatment group. This comparative effectiveness design can best inform us if a novel treatment performs better than comparable interventions.
Cook et al’s “Early use of thrust manipulation versus non-thrust manipulation: A randomized clinical trial” serves as an excellent example of a pragmatic trial. This trial collected a heterogeneous sample from 16 different outpatient clinics, utilized 17 different clinicians who were allowed to utilize their clinical decision making to select the most appropriate thrust or non-thrust technique, and incorporated a tailored multimodal treatment plan following the initial two visits. Such a study design allows for a high level of external validity which can give clinicians confidence that the results will generalize to clinical practice. However, like a see-saw, as external validity goes up internal validity must come down. This trial informs us that thrust and non-thrust techniques have comparable effectiveness, but the trial is unable to inform us if either technique demonstrates efficacy.
When discussing efficacy and effectiveness trials, it would be myopic to conclude that either study design is better. They simply ask different questions. Efficacy trials are best used to study new interventions and elucidate if the treatment has a specific effect. As such, they do not fully represent clinical practice. This is on purpose. Effectiveness trials are best suited to capture the multifactorial nature of rehab interventions and inform us how new treatments compare to currently available interventions and standards of care. As such, they are unable to determine the specific effect of a singular intervention. This too is on purpose. To best inform our clinical decision making, we need both approaches to research. We must also be acutely aware of each approach’s unique strengths and limitations.
Photo Credit: 'Dictionary' from Greeblie (flickr.com)