Survivorship Bias and Physical Therapy

13707046604_cafde37890_b

Everyone has hopefully had a “eureka” moment in the clinic— The patient no one was able to help before, who by some stroke of genius (or more likely, luck), you are able to assist in a way that is truly remarkable. These successful outcomes are likely to stick with you. They are salient, positive and downright enjoyable to think of. Our memories of positive outcomes stay with us because most of us want to be successful clinicians. So naturally, it makes sense to try and re-create what we did, or to emulate what others did to succeed. In reality though, this does not work out so well.

For example, imagine a patient comes into your clinic with longstanding lateral elbow pain. Previous approaches have not helped much but you think rubbing some cabbage on it may help. After two sessions of therapeutic cabbage rubbing, the patient explains to you “my pain is gone!” This is great news and likely something that will be in the back of your mind the next time a patient with lateral elbow pain comes in to your clinic. You then might think, “Aha! Last time I cabbaged a patient like this and they had zero pain in two visits”. This recall of a previous success influenced your decision making. Perhaps your line of reasoning helps produce a good result this time too.

These successes will continue to take up space in your mind and continue to influence the way you make decisions. But, what about all those times therapeutic cabbage rubbing did not help? What about those ones who did not see a remarkable change in their pain? Surely you aren’t curing everyone with your approach. As it turns out, we humans really are not so balanced in our decisions that are based off personal recall of experiences. Those successes that take up so much space in our mind bias our judgment and impact our decision making and not always for the better. The middle of the road outcomes, the patients that stopped coming, those instances simply fade from memory.

Further, positive outcomes in the clinic tend to be unpredictable and chaotic. There is an incredible amount of noise, which can make it difficulty to tease out any meaningful signal after the fact. As a result, we can be easily deceived into thinking we understand how and why we achieved such success, when in reality the positive outcome was a result of any number of things, much of which we might be oblivious to.

Enter survivorship bias —

Survivorship bias is “a cognitive bias that occurs when someone tries to make a decision based on past successes, while ignoring past failures.

What this leads to is a form of selection bias in our decision making. If we only remember the positive outcomes, we are not exactly making a balanced and well informed decision, are we? As David McRaney writes “You must remind yourself that when you start to pick apart winners and losers, successes and failures, the living and dead, that by paying attention to one side of that equation you are always neglecting the other.” If you only try to emulate what successful people have done, you are missing the bigger part of the picture and likely ignorant of the fact that a lot of what made these unique cases successful is a lot of luck, randomness and circumstance. In the words of Daniel Kahneman “If you group successes together and look for what makes them similar, the only real answer will be luck.”

Survivorship bias is why it is important to inform (not dictate) our decision making with systematic approaches and tools that seek to limit our propensity to only recall successes. Things like evidence from controlled trials, population data and robust outcome tracking are able to capture successes and failures much more accurately than our personal experiences and observation of others. That is not to say these things are without their own forms of bias and limitations, but that is beyond the scope of this post. We must also keep in mind where our observations, experience and expertise can truly provide value. You can read more about that here.

Are you interested in learning more about survivorship bias? The podcast by David McRaney at You Are Not So Smart is a great start. It was the inspiration for this post and where the quotes from McRaney himself and Kahneman were found. Give it a listen, maybe purchase his books. They are quite enjoyable and will certainly make you less dumb.

Photo courtesy of Nina Sivatheesan over at flickr

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s