Today's article is a guest post from Jason Eure, a physical therapist practicing in Virginia who likes to write a lot of words but doesn't usually share them. Personally, I am really glad he decided to put this post up here because he's a fantastic clinician and thinker who we can all learn something from. You can follow him on Twitter @jmeure
Internet discussions often remind me of the movie Groundhog Day. The theme may change (dry needling, manual therapy, posture, taping, etc) but the ensuing discussion will almost always follow a remarkably similar path. Invariably, the topic of expertise and its role in clinical decision making is weaved into the discussion to either argue for or against inclusion of certain treatment modalities. This point generally coincides with a “Sackett’s Stool” analogy where one party berates the opposition for being overly reliant on a single information source for guidance when it comes to practice, quickly followed by an equally condescending retort regarding the uselessness of expertise to provide reliable data. Both parties likely exit the conversation with bolstered perceptions of their pre-existing beliefs, and I’m left disappointed in myself for reading another thread with the hope that it may be different than the norm. This stereotyped discussion above is a mine-field of topics to explore; however, I want to tackle the interplay between research, experience, and values.
A Stool Worth Sitting On?
In 1996, David Sackett published an article titled “Evidence based medicine: what it is and what it isn’t.1” Here, he and his co-authors articulate a fairly nuanced approach to optimize outcomes, stating that evidenced based medicine:
is not ‘cookbook’ medicine. Because it requires a bottom up approach that integrates the best external evidence with individual clinical expertise and patients' choice, it cannot result in slavish, cookbook approaches to individual patient care.
Interestingly, although Sackett and the stool analogy are inseparable in most discussions,the image of a stool (or ‘pillars’) isn’t evoked once in this original text. This visual was created later and served as a representation of the three major components contributing to the clinical decision making process: clinical experience, clinical research, and patient preferences. Unfortunately, this image has altered the intended message and created an overly-simplified narrative.
It is important to note that each feature, typically presented as separate stool legs, is not mutually exclusive from the other. There will always persist a dynamic interplay between the three which guides clinical decision making. To discuss them as separate spheres (as done to an extent in this article for simplicity's sake) only perpetuates this notion and adds confusion to the topic.
In 2015, Roger Kerry and his colleagues published an article titled “Evidence based on what?” where they discuss various forms of external evidence and how it is critical to understand what information is reasonable to extrapolate from each type of study design2. The same basic principle needs to be applied when it comes to Sackett’s stool. The image inspires the incorrect assumption that each “leg” gives us information of equally-weighted value to the same question. Each aspect of the decision making process is essential to the development of a treatment plan for its own unique reasons.
The discrepancy often manifests when determining if something is efficacious versus effective or when determining if something has specific effects. Causality cannot be stated unless controlling for all other variables which may influence the observed outcome. In non-physical sciences, although it is impossible to establish complete causality, we afford ourselves the best opportunity to filter the signal from the noise by employing the scientific method. Well-conducted, randomized, placebo-controlled trials epitomize this process through maximizing the internal validity of the contextual factors being studied and reducing potential causative influences to the smallest possible number. We cannot and should not make claims in regard to efficacy of any intervention or modality if this standard is not met; thus, rendering external evidence as our only viable method of reliably determining specific effects.
But where does experience play a role in this process? And what about effectiveness? The common argument ranges from “I know this treatment works because I see the positive effects” to “I don’t care ‘how’ it works, if it makes my patients better, that’s all that matters.” It may appear justifiable to only value the clinical outcome or attribute some causal factors to an intervention after consistently observing positive results within a clinical setting. However, this thought process is dangerous. Why? How could amassing years of experience and observing hundreds to thousands of data points lead to worse clinical decision making?
Experience vs Expertise
Experience and expertise are not synonymous with each other, and there is not a simple linear relationship between the two. This was notably demonstrated in the field of medicine through a study conducted by Choudhry et al. where they showed
Our systematic review of empirical studies evaluating the relationship between clinical experience and performance suggests that physicians who have been in practice for more years and older physicians possess less factual knowledge, are less likely to adhere to appropriate standards of care, and may also have poorer patient outcomes. These effects seem to persist in those studies that adjusted for other known predictors of quality, such as patient comorbidity and physician volume or specialization. The results are somewhat paradoxical since it is generally assumed that clinical experience enhances knowledge and skill and, therefore, leads to better patient care.3
This contrast between experience and expertise has also been demonstrated specifically within the field of physical therapy by Resnik and Jensen, where they conclude:
Our findings challenge a basic assumption that extensive experience as a physical therapist is essential for the development of physical therapist expertise.4
This ‘paradoxical result’ is hard to accept at face value until examining why we may fall prey to deteriorating decision making skills over time. Choudhry reasons that
Our findings have many possible explanations. Perhaps most plausible is that physicians' “toolkits” are created during training and may not be updated regularly. Older physicians seem less likely to adopt newly proven therapies and may be less receptive to new standards of care.3
So why are we resistant to change? There is an abundance of psychological research which may help explain cognitive rigidity and the distinguished gap between our perceived and actual clinical influence. Once we have adopted specific philosophical stances toward common problems, it tends to become harder to alter our perspective. Ownership of an object or idea typically leads to the overvaluing of that property (known as the endowment effect). Thus, when presented with new or contradictory information from our owned belief, it becomes easier to dismiss or ignore when contrasted against our excessively-weighted, biased perspective. We continually compound upon this opposition on a daily basis when we confirm our rationalizations and justifications for treatment. Even the most well-intentioned individuals view the world through skewed lenses due to a phenomenon known as confirmation bias, a type of selective thinking where an individual attends to information which confirms their beliefs to a significantly greater amount than information which contradicts their current narrative. Additionally, not only are we narrowing our attentional focus on outcomes which augment our current outlook, we are doing so on an already limited sample group. Many patients who are dissatisfied with their progress and/or do not respond appropriately are lost to follow-up. This creates a survivorship bias where we are only considering patients who complete their treatment plans as originally prescribed- a population which, as noted above, most likely consists of individuals who have demonstrated some favorable response over the course of time. Thus, we entrench ourselves further into flawed clinical patterns while feeling soundly justified because we “see it work.”
When the circumstances above are taken into context with a human tendency to overestimate their ability to control events (illusion of control), make false assumptions of causality due to sequential nature of events (post hoc fallacy), and wrongfully develop preferences for and ascribe greater validity to things which the individual has been exposed to frequently (familiarity principle)- it becomes clear that experiential learning is not a reliable source to provide information regarding specific effects or efficacy of treatment interventions.
Important tangent: Philip Tetlock has extensively examined the psychological constructs which may predispose individuals to faring worse when given more data points. In short, those who thought probabilistically, were open to the notion that they may be wrong, and were willing to alter their stance when presented with new information, demonstrated superior predictive abilities compared to those with opposing characteristics5. These traits were echoed by Jensen in her paper examining expertise within the physical therapy realm, showing positive differences for those who “understand their own limitations, appreciate what they did know as well as what they needed to learn, and demonstrate a well-developed ability for self-reflection and reassessment of their own practice.4” However, it is crucial to understand that adopting such traits does not allow experience to answer questions of causality or efficacy- it simply mitigates negative effects of the cognitive biases outlined previously. (See more in Philip’s book)
A Case for Experience
Despite the limitations noted, one simply cannot become an expert without experience. Specific to the field of physical therapy, there are many avenues which experience can enhance clinical performance and subsequent outcomes.
First, circling back to the concept of the three stool legs as dynamically interconnected components; experience influences a clinician’s ability to critically appraise, assimilate, and integrate research into practice. External evidence is not infallible and has been linked to issues of publication bias, conflicts of interest, fraud, outcomes switching, academic paywalls, and misrepresentation of data to list a few (see this post for a commentary). Navigating these issues to decipher meaningful data requires extreme skill itself. Even more is necessary to appropriately weigh the evidence provided against the established body of literature and apply it clinically. Due to the highly controlled nature of research trials, the immediate application of concepts within vastly different contexts becomes problematic. The cost of the high internal validity required to determine efficacy significantly limits external validity (the transferability of results from a specific population to others). Even pragmatic trials, attempting to use multi-modal treatment approaches within relatively heterogeneous treatment groups, are unable to completely replicate the context of an individual patient. We do not treat in a vacuum, and our patients commonly present with variable co-morbidities, psycho-social factors, socio-economic statuses, required daily stress demands, time/resource constraints, and preferences (to be expanded upon later) which render replication of many treatment paradigms impossible as outlined within a study. This is where experience is invaluable. The ability to alter variables without losing the central intent of the provided evidence is enhanced through exposures to different scenarios.
Second, while arguing for or against the incorporation of movement screens and manual interventions is beyond the scope of this article, experience plays a pivotal role in these realms. The contribution of deliberate practice in any activity requiring specific perception-action capabilities or neuromuscular coordinative capacity is self-evident.
Third, and potentially most important, life experiences can develop and enhance many social skills which are imperative to successful rehabilitation. Recent evidence suggests that patient expectations and therapeutic alliance play integral parts in optimizing treatment outcomes6,7,8. Thus, instilling confidence in your patients and establishing a strong rapport can mean the difference between a positive and negative conclusion. While social traits are largely heritable and are innate constructs of an individual’s personality, it does not mean we cannot improve upon our given characteristics. Every social interaction is an opportunity to learn and adapt for the better. Obviously, patient personality types vary greatly and a therapist must be flexible to discover and incorporate the appropriate educational and motivational techniques required to suit the individual- a skill that can only be acquired and enhanced through a multitude of real-world experiences.
Important tangent: Improving all of the aforementioned skills through experiential learning is contingent upon the individual being self-aware. If someone displays the inflexible traits discussed earlier, does not consider the possibility that there is room for improvement in their abilities, and/or is unaware of the feedback their surroundings provide them; there will be no incentive to change. This is seen frequently on social media forums where an individual will constantly use the same aggressive argumentative style for every discussion despite never obtaining the desired outcome.
The Value of Patient Values
Finally, where do patient preferences fit in this model? In a 2014 systematic review, Price et al found that patient’s perception of their experience was positively linked to better clinical outcomes, better patient safety within hospitals, and less healthcare utilization9. Although the body of evidence suggests this trend, the evidence is not unequivocal. A group of researchers from UC Davis found higher patient satisfaction was associated with increased mortality even when controlling for baseline health measures10. They argue that physicians frequently accede to patients’ requests for discretionary services that are of little or no medical benefit, and these discretionary services may lead to iatrogenic harm via overtreatment, labeling, or other causal pathways. So what are we to make of such seemingly opposed messages regarding attending to patient desires? An unmentioned factor in Price’s work was the connection between patient satisfaction and subsequent adherence to their medical plan of care. They demonstrated a 19% risk increase of non-adherence when accounting for satisfaction regarding communication skills alone9. Even the best treatment plan will fail if unfollowed. Therefore, the totality of evidence suggests that we must account for patient satisfaction to ensure adherence, but not to the extent that we allow it to supersede our clinical expertise. Granting the patient a choice, by providing equivocal (or near-equivocal) options for interventions, invokes a sense of ownership within the therapy process and can aid in patient satisfaction and ultimately compliance.
But what about when a patient explicitly asks for a treatment which has shown little to no clinical utility? In theory, it is easy to draw lines in the sand which denote beneficial treatment modalities from ones which we believe are unjustifiable or simply pander to the patient’s demands. In reality, this conversation is filled with grey and the demarcation line is moveable and blurred. At best, we can educate the patient through an explanation of the consensus of available evidence, using appropriate social skills tailored to the preferential interaction style of the patient (again, exemplifying the dynamic nature of the ‘stool legs’), and allow them to make a decision based upon the available information.
A Distorted Narrative
The notion of a stool being an accurate symbol of evidence based medicine is flawed. Continuing to employ this simplified heuristic perpetuates an incorrect notion that external evidence, clinical experience, and patient preferences provide equal insight to the same problems. Each medium of knowledge has inherent value, along with inherent limitations, which dictate the role it can play within the clinical decision-making framework. Picture the development of a treatment plan as a gradual refinement process. External evidence will provide probabilistic information regarding efficacy of various treatment interventions to elicit desired responses, experience will allow for this information to be adapted based upon the immediate and goal physical, task, and environmental constraints, and patient values will further refine that selection to improve adherence.
To sum up:
- External evidence, experiential data, and patient preferences do not provide equal value in regards to treatment decisions.
- Innate human characteristics render experience as a poor source to generate claims regarding efficacy and effectiveness.
- Think probabilistically, with the understanding that you can always be wrong, to blunt inherent bias in our thought processes
- Expertise cannot be gained without experience; however, simply amassing years of experience does not automatically lead to expertise.
- Patient adherence is paramount to positive clinical outcomes. Treatment plans should value patient preferences but not sacrifice best practices in order to do so.
- The analogy of a three-legged stool (or three pillars) is a mischaracterization of what each information source is capable of reliably providing. Throw the stool in the woodchipper.
I encourage all readers to read the references provided for further insight into the topic, as well as to read commentaries by Steven Novella, Erik Meira, and Kenny Venere for a more comprehensive understanding of the application of science within healthcare.
1. Sackett David L, Rosenberg William M C, Gray J A Muir, Haynes R Brian, Richardson W Scott. Evidence based medicine: what it is and what it isn't BMJ 1996; 312 :71
2. Anjum, R. L., Kerry, R. and Mumford, S. D. (2015), Evidence based on what?. Journal of Evaluation in Clinical Practice, 21: E11–E12. doi: 10.1111/jep.12493
3. Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005 Feb 15;142(4):260-73
4. Resnik, L., & Jensen, G. M. (2003). Using Clinical Outcomes to Explore the Theory of Expert Practice in Physical Therapy. Physical Therapy, 83(12),1090-1106. Accessed June 30, 2016. Retrieved from http://ptjournal.apta.org/content/83/12/1090.
5. Mellers, B., Stone, E., Atanasov, P., Rohrbaugh, N., Metz, E. S., Ungar, L., Bishop, M. M., Horowitz, M., Merkle, E., Tetlock, P. The psychology of intelligence analysis: Drivers of prediction accuracy in world politics. J Exp Psychol Appl. 2015 Mar;21(1):1-14. doi: 10.1037/xap0000040. Epub 2015 Jan 12.
6. Dunn, Warren R. et al. 2013 Neer Award: predictors of failure of nonoperative treatment of chronic, symptomatic, full-thickness rotator cuff tears. Journal of Shoulder and Elbow Surgery, Volume 25, Issue 8, 1303 – 1311.
7. Chester, R. et al. Psychological factors are associated with the outcome of physiotherapy for people with shoulder pain: a multicentre longitudinal cohort study. Br J Sports Med. bjsports-2016-096084Published Online First: 21 July 2016 doi:10.1136/bjsports-2016-096084
8. Fuentes, J., Armijo-Olivo, S., Funabashi, M., Miciak, M., Dick, B., Warren, S.,Rashiq, S., Magee, D. J., & Gross, D. P. (2014). Enhanced Therapeutic Alliance Modulates Pain Intensity and Muscle Pain Sensitivity in Patients With Chronic Low Back Pain: An Experimental Controlled Study. Physical Therapy, 94(4), 477-489. Accessed July 31, 2016.http://dx.doi.org/10.2522/ptj.20130118.
9. Anhang Price R, Elliott MN, Zaslavsky AM, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev 2014; 71(5): 522–554
10. Fenton JJ, Jerant AF, Bertakis KD, Franks P. The Cost of Satisfaction: A National Study of Patient Satisfaction, Health Care Utilization, Expenditures, and Mortality. Arch Intern Med. 2012;172(5):405-411. doi:10.1001/archinternmed.2011.1662.