An approach to predicting patient experience through machine learning and social media analysis
One of the goals of healthcare is to provide a positive experience for every patient during their medical journey. Patient experience is positively associated with clinical effectiveness and patient safety. Information relating to patient experience can therefore be used to drive strategy for transforming practices as well as to drive an overall system transformation. The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is the first national, standardised and published survey of patients’ perspectives of hospital care in the United States, and represents an objective measure of patient experience.
Various factors are associated with patient experience, including complaints about a healthcare provider, the courtesy of support staff, the time providers spend with a patient, the length of waiting time, the clarity of discharge information, the cleanliness of the treatment area, as well as the patient’s gender, language, education and state of health. Patients whose doctors provide information in a comprehensible manner are more likely to recognise their health problems, understand their treatment options, change their behaviour accordingly and follow their medication schedule.
There is growing recognition of the importance of training providers to improve their interactions and communication with patients. Identifying the most effective providers can help prioritise training and education. Focusing communication training on the highest-priority providers can facilitate the dissemination of best practices throughout the facility, thereby improving communication between providers and patients and ultimately the patient experience.
This study aims to identify two new interconnected approaches to improve patient experience.
- The first approach makes it possible to identify patients who are at risk of having a negative experience. As part of the prediction component, the risk of a negative patient experience was determined based on two types of features: the patient’s personal demographics and the specific providers who care for the patient.
- The second approach tests the hypothesis according to which, within each network, doctors who are highly connected probably have the most influence on patient experience. Social media analysis is used to determine the “connectedness” of each provider.
Predictions from both approaches will be implemented in production in the hospital to improve patient experience. In the first approach, if the model predicts that a negative patient experience is likely to occur (in the provider’s domain), the hospital will intervene by bringing a customer service specialist to the patient’s room. The customer service intervention will mitigate the risks of a negative patient experience. Thus, service recovery in real time will be facilitated. In the second approach, when influential providers are identified, they will receive special training in their bedside manner. The goal of both these approaches is to facilitate improvement of the inpatient experience.
Separate models were built to predict the responses to each of three questions in the HCAHPS survey:
- During your hospital stay, how often did the doctors treat you with courtesy and respect? (Courtesy/respect)
- During your hospital stay, how often did doctors explain things in a way that you could understand? (Understand)
- During your hospital stay, how often did doctors listen carefully to you? (Listen)
The response variable was taken from the HCAHPS survey responses to doctor communication questions. For each question, patients had four possible responses:
- Always
- Usually
- Sometimes
- Never
The responses to three questions from the patient survey in the area of communication with providers were retrospectively validated on out-of-sample data. The predictive model developed can be used as the basis to build a risk stratification tool to predict patient experience. Such a risk stratification tool could allow for earlier assessment and intervention (service recovery), which would mitigate the risk of negative survey responses.
The performance of the prospective validation for questions 2 and 3 is slightly lower than the performance of the retrospective validation due to the fact that some newer providers were not included in the training data. This issue will be resolved in production by retraining the model on a daily basis.
Additional patient and clinical features, including additional demographics, comorbidities, and laboratory results, will likely improve predictive performance and will be studied for future versions. Future models could be improved further. Going forward, the authors plan to quantify the business impact of production models through A/B testing.
The opinion of the SPX Virtual Contents Editorial Board:
« This article goes a step beyond the patient experience, in the sense that it does not discuss its value, but once accepted it explores new methods to value it. Through a machine learning model, with just three questions from the HCAHPS survey, it identifies patients at risk of having a negative Patient Experience and this can initiate actions to improve it. On the other hand, analyzing the social networks of professionals, those who can most influence the Patient Experience are identified. »
« This article also questions the overall approach towards Patient Experience in regions where this dimension has been partly integrated within healthcare providers’ remuneration schemes. One could argue whether we’re looking here at an innovative way to support an actual improvement of patient experience, or at a potential shift towards looking at the best means to mitigate risks to get low HCAHPS ratings while maximizing chances to get positive ones. »
