HCOs Must Be AI-Driven to Succeed

According to a survey by the Society of Actuaries, 93% of healthcare organizations (HCOs) believe leveraging predictive analytics is key to the future of their business. This perspective is undoubtedly warranted. As an industry, healthcare is evolving, and to succeed in a system centered on value-based care, HCOs must be able to harness the ever-increasing stream of data to produce better outcomes with greater precision. Instead of treating patients as adverse events and complications occur, HCOs will need to anticipate the future and work proactively. To this end, AI is essential to preemptively surface high-risk patients, predict exactly what they’re at risk of and why (e.g., an unplanned admission due to COPD exacerbation), and help care teams to determine effective interventions with a long enough runway.

The benefits of AI extend far beyond singular use cases and permeate every aspect of healthcare. Leveraged effectively, AI has the potential to impact HCO business operations, clinical programs, marketing efforts, patient engagement, sales, contracting, rate setting, digital health implementation, and more. There is not a single facet of business or care delivery that doesn’t stand to benefit from AI-based insights, and the HCOs scaling it the fastest are beginning to solidify competitive advantages.

And yet, healthcare is lagging behind other industries when it comes to realizing the value of AI. A staggering 87% of all data science projects never make it into production, and HCOs implementing AI must do so while also accounting for healthcare-specific challenges. They must have the tools to handle everything from defining episodes of care with dynamic healthcare terminologies to addressing algorithmic biases which may exacerbate health disparities – all while ensuring adoption and explainability. Even if they’re among the 13% that manage to successfully deploy machine learning (ML) models, they still require comprehensive capabilities to maintain them in production. Put simply, successfully operationalizing ML models for healthcare is difficult.

The need for AI-driven insights is clear, but for many HCOs, the steps they need to take to avoid having their AI initiatives end up as failed science fair projects are unclear. How can they ensure their investments result in improved decision-making and better outcomes? Answering this question necessitates a holistic evaluation of data science processes, key considerations for healthcare-specific use cases, and an understanding of the required ML capabilities. Ultimately, HCOs must ask themselves, “What can we specifically focus on to operationalize models, streamline adoption across the organization, and drive impact at scale?”

Start Small and Get SMART

HCOs that try to boil the ocean from the outset are bound to fail. Unrealistic expectations regarding the initial scope of AI projects set the initiative up for failure and leave stakeholders and data scientists disappointed. In turn, this failure can leave organizations hesitant to attempt subsequent AI projects. To demonstrate success and begin realizing improvement, it’s imperative that data scientists work closely with both clinical and business stakeholders to determine specific use cases and ensure models are developed with that context in mind. This means establishing a complete understanding from all parties of how models will be used in production, how results will impact outcomes, and how improvement will be measured. Moreover, it means establishing consensus about the specifics from the outset. Data scientists and stakeholders need to thoroughly answer the question: “What problem are we trying to solve and how?”

SMART (Specific, Measurable, Achievable, Realistic, and Timely) is an acronym used in the goal-setting process that does an excellent job of framing real-world problems in a machine learning context. This framework enables stakeholders and data scientists to consider the problem they’re attempting to address practically and set clear goals and expectations rather than attempting to solve broad issues with vague criteria for success. For example, consider the following two statements that both attempt to frame the same AI initiative:

“We need to reduce our inpatient admissions for asthma.” VS “Compared to the national average of 5.5 per 10,000, our inpatient admission rate associated with poorly managed asthma is 8.4. By implementing the asthma management program recently published in Annals of Allergy, Asthma & Immunology we expect a 15% reduction in asthma-related admissions within six months of the initial diagnosis of asthma.”

The first statement is vague and would need to be substantially fleshed out to begin addressing the problem at hand. Conversely, the SMART statement provides a much clearer foundation, with specific reference points, measurable goals, a set time-period, and expectations for effectiveness. It also drives constructive discussion and pinpoints specifics to provide essential guidance for designing and training models. As a result, data scientists can purposefully conceptualize and build models to target specific pain points, help facilitate intervention programs, and accurately reflect the needs of care teams.

AI projects will never reach deployment without clear communication, specificity, and defined goals that are realistically achievable. Rather than tackling a broad, widely interpretable issue, such as inpatient admissions, organizations need to actively engage in identifying specific use cases and collaboratively making them actionable.

Ensure Explainability

Explainability is crucial to promote adoption and enable the shift to value-based care. Clinical stakeholders simply won’t use model results if they aren’t able to verify the integrity of predictions and intimately understand why they were made. To this end, “black box” models are insufficient, delivering no more than minute improvements when integrated in clinical workflows.

To trust models, clinical stakeholders must be able to unpack exactly which variables were used and which were the most impactful in making predictions. This level of specificity is necessary for validation at the population health level, (e.g., understanding how frailty contributes to risk of fall-related injuries) but stakeholders need even greater explainability at the individual level. They need the ability to identify which patients are at high risk for a given outcome, the modifiable risk factors that had the greatest impact on this prediction, and which of these patients are most likely to benefit from intervention efforts.

Diagnosis-code level insight enables clinical stakeholders to deliver personalized interventions that maximize value. For example, a given patient may be flagged as high risk for heart failure in the coming months, but it may be difficult to understand what distinguishes them from the larger population without granular explainability. However, if AI-based models are able to surface the raw data from this patient’s health history, such as a decline in their left ventricle ejection fraction, care teams will better understand impactability. Ultimately, this specific evidence is essential to personalize interventions and improve health and financial outcomes.

Managing, Monitoring, Auditing

Deployment is a new beginning, not an ending. Once models are in production, data science teams have a tendency to adjust them less regularly than they did in development and monitor performance infrequently. This is a critical mistake; as time progresses, accuracy degradation, feature drift, and unforeseen bias will inevitably begin to jeopardize model performance. Moreover, data scientists must also keep deployed models constantly updated and running reliably in production amidst frequent changes across the healthcare industry, such as shifting code systems, program eligibility requirements, and chronic condition prevalence. Case in point, a model to predict respiratory illness that was trained on 2019 data and never retrained would be wildly inaccurate due to the COVID-19 Pandemic.

To evolve along with healthcare demands that HCOs have the capability to seamlessly manage deployments, monitor performance, and audit models in production. This requires ML Ops. ML Ops is a set of practices that enhances the DevOps procedures used to deploy traditional software with the unique requirements of ML systems. It provides a suite of capabilities to not only manage deployments, but automate model performance monitoring and effortlessly retrain models, with support for audit and governance. This enables HCOs to better manage the explosion of healthcare data, anticipate and address algorithmic bias, maintain model accuracy, and explore new strategic opportunities.

The Future of Healthcare

The ability to operationalize ML models and improve outcomes from their predictions is quickly becoming a given. HCOs must adapt to value-based care, the inundation of available data, and the rise of alternate payment models. Today’s healthcare leaders will simply be left behind if they are too slow to invest in the required technologies and adopt end-to-end model development processes. Going forward, success will be predicated on the ability to contextualize models for specific healthcare use cases, establish trust in predictions, and rapidly adapt to emerging challenges.

For a more thorough analysis of healthcare-specific data science challenges, please download our white paper: Six Mistakes You Can Avoid In Healthcare Data Science.