LAS VEGAS—The American Heart Association is launching an AI Assessment Lab in partnership with Dandelion Health to validate predictive AI for cardiovascular conditions and related diseases.
The group hopes its 100-year-old history as a medical specialty group will lend credibility to algorithms that have met safety and efficacy standards so health systems feel confident using them in practice.
It’s the latest move to lend credibility to a technology that lacks standard reporting guidelines and that hospitals have largely not figured out how to deploy and effectively monitor.
The lab will assess algorithms that predict the risk of heart conditions or events. They will also review models for related conditions like kidney disease or diabetes, widening the aperture of model developers that could go through the lab. Dandelion will review algorithms both pre- and post-FDA clearance, if applicable.
“There are algorithms now that can predict your likelihood of heart failure or your likelihood of a heart attack,” Jennifer Hall, chief of data science and analytics, said in an interview. “And so we’re very excited about that, but at the same time, want to provide a framework so patients understand what’s happening, that there’s a safe system in place to rigorously assess those algorithms and help hospital systems understand when it’s a good idea to implement those algorithms.”
The AI Assessment Lab mirrors the AI Assurance Lab concept that the Coalition for Health AI was once pursuing. However, the effort is smaller and more defined than CHAI’s grand concept, which is essentially defunct.
Hall said AHA set up a committee of AI experts from big and small health systems to advise it on AI initiatives and has formed a strategic set of committees on AI. The panel of experts will review the algorithms after they are assessed by Dandelion to make any relevant clinical recommendations, Elliot Green, CEO and co-founder of Dandelion, said.
In addition to an assessment report on the model’s technical performance, the AI Assessment Lab will produce an impact assessment that quantifies its economic impact for future customers. Dandelion will create the impact report using data on the cost of the condition the model is targeting and how often the model correctly predicts an event
Dandelion can also tailor the impact assessment and validation to run on data that is similar to a specific health system that the vendor may be trying to sell into. Through its network of de-identified patient data, Dandelion can create a proxy dataset that reflects the population of the health system.
It’s not exactly local validation, which is a gold standard in AI implementation. However, it’s an effort to help diffuse AI innovation to systems that don’t have the capability to do local validation and give smaller health systems an idea of how an algorithm would likely perform in their environment.
“What we can do is … find the subset of our groupings that help people understand a little bit more about what could happen within their healthcare systems,” Green said. “So it just gives you that extra level of surety as to the performance of an algorithm with regards to your own population.”
Model developers will pay for the review by the lab. Hall said that thus far, AHA has a few models in the pipeline to be assessed through the Assessment Lab.
Eventually, AHA and Dandelion will embark on post-deployment monitoring of the models they assess as they are implemented into hospitals. The groups said the capability will likely be rolled out next year, once more of the algorithms have been deployed.
The AHA also hopes to bridge the digital divide by giving small community hospitals a way to understand the economic impact of implementing an algorithm and spreading the benefits of AI to rural and underserved communities.
The AHA also funds research in data science and AI, maintains a database of 14 million de-identified patient records for research use, and has a quality improvement program that tracks hospitals’ adherence to AHA standards in 3000 hospitals nationwide.
“We write the guidelines,” Hall said. “When you go to the doctor and they tell you you need to take a statin, that’s because the American Heart Association wrote the guidelines that told the doctor what to do.”
She continued: “Our volunteers and clinicians—healthcare providers trust us, and we felt that, you know, if we could provide this service, that it may go a long way for helping in this really changing landscape of AI to figure out what are the best practices that they can do for their hospital system and for patients,” Hall explained.
The AHA went through an “intensive process” to select a vendor to run its AI Assessment Lab, Hall said. It ultimately chose Dandelion because its customer base is representative of all corners of the U.S. and mirrors the demographic makeup of the country. Hall also touted Dandelion’s careful handling of data.
Publisher: Source link
