Evidence From a Value-Added Approach On an Online Therapy Platform
By Jeffrey Swigert
We construct direct measures of therapist quality using a value-added approach similar to those used in the economics of education. We then use these measures and the quasi-random assignment of patients to therapists on an online therapy platform to explore the causal effect of being assigned to a high quality therapist on mental health assessment scores, therapy engagement, treatment compliance, and the likelihood that a patient opts to switch providers. We find evidence that… Using therapist value-added as a guide, we attempt to identify best practices and methods employed by the highest quality therapists in improving patient mental health using natural language processing on archived text therapy transcripts.
Executive Summary
“Therapist quality can be effectively quantified and measured.”
What is therapist quality? This is an important (and perhaps controversial) question that begs several more questions related to what it means to be an effective therapist. The better we can understand what makes for effective psychotherapy, the better enabled we will be to provide treatment to patients. This project attempts to shed new light on aspects of therapist quality by constructing direct, data-driven measures of therapist quality and exploring their effect on a variety of therapeutic outcomes of patients.
We utilize patients' improvement on psychometric assessment scores as a means of quantifying their therapists' effect. Drawing upon similarities that exist between the setting of mental health therapy in the healthcare industry and teaching in the education sector, we borrow the value-added approach that has been fruitfully applied to measure teacher quality. Economists like Lovenheim and Turner (2018) define a teacher’s value-added as “his or her contribution to student test score gains.” Likewise, we define a therapist’s value-added as their ability to increase observed patient scores on a variety of psychometric assessments.
Thanks to a rich database provided by a large, private online therapy platform that has matched more than 5 million patients to roughly 5,000 mental health care providers, as well as cutting-edge econometric methods, we are able to arrive at precise estimates for therapist-level value-added measures.
This project is currently underway, as we are cleaning the data and preparing it for rigorous analysis. We're working hard to develop efficient models of therapist quality so as to isolate those improvements in psychometric assessment scores that can be attributed to the patient's therapist, and not to extraneous factors.
The applications of these findings will be numerous, and the possibilities are exciting. With quantitative measures of therapist quality, we will be able to identify what makes a therapist a good therapist, and can from there recommend best practices to improve online psychotherapy at large.
Contributing Research Fellows: Jared Rowley, Jeff Rowley, Mitchell Zufelt, Zhousheng Zhou, and Cheyenne Lawrence
Comments