How Cleveland Clinic Is Speeding Up Clinical Trial Recruitment

Cleveland Clinic’s research organization is deploying a solution from AI startup Dyania Health that uses medically trained large language models (LLMs) to speed up and scale patient identification for research studies.

Lara Jehi, M.D., chief research information officer at Cleveland Clinic, and Eirini Schlosser, Dyania’s CEO and co-founder, sat down with Healthcare Innovation last week to discuss why Cleveland Clinic was so impressed with pilots of Dyania’s Synapsis AI platform that it chose to invest in the company. 

As an example of the potential, Cleveland Clinic’s pilot evaluated Dyania’s Synapsis AI alongside two experienced research nurses for a melanoma trial. On average, the technology identified an appropriate trial patient in 2.5 minutes with 96% accuracy, compared to 95% accuracy in 427 minutes by a melanoma-specialized nurse and 88% accuracy in 540 minutes by an oncology research nurse.

Healthcare Innovation: Dr. Jehi, could you start by briefly describing your role as chief research information officer? Do you oversee the informatics infrastructure for the research efforts across Cleveland Clinic? 

Jehi: Yes. I would say it is bridging research and technology across the health system, whether it is supporting the current needs or developing the plans for future directions in research. That includes accelerating operations through innovative tools like the Dyania platform, but also stuff that is much further out, like doing quantum computing and accelerated discovery and that type of a thing.

HCI: Eirini, could you talk about the founding of Dyania? 

Schlosser: I’ve been an entrepreneur for about a decade, specifically focused on natural language understanding within AI as that has evolved over the course of time. I founded Dyania at the end of 2019 very specifically focused on solving a very pertinent problem in medicine. We had some early investors who owned hospitals and they were really keen to understand the value that could be unlocked within electronic medical record data specifically for clinical research. I dove in with them to just understand the manual effort that went into the review of charts and all the particular use cases across any given healthcare system that would be necessary to make sense out of the data. That led me to jump into the space of trying to automate chart review. 

From day one, I would say, to a degree, we were gluttons for punishment, because we went after what is probably one of the more complex use cases of that technology, which is finding patients for clinical trials. And it’s particularly challenging because you have patient clinical characteristics that are changing quite rapidly over time, and so you’re looking for an exact order of events, which is almost like looking for the day that a patient’s stars align and they meet inclusion/exclusion criteria. 

So it’s not just about answering a one-off question about a patient, such as does a patient have Type 2 diabetes? Rather, we’re looking for patients who have had a very specific order of events and would be potential candidates, and one day they might not meet criteria and 10 days later, they might meet the criteria. 

Obviously, there are many other applications that we are very quickly moving into and are already active in, but the clinical trials space was where we started out with Cleveland Clinic.

HCI: Dr. Jehi, could you talk about some of the limitations in the traditional way that a health system identifies potential patients for clinical trials?

Jehi: I don’t think it would come as a surprise to anyone that the process is a very inefficient, archaic, frustrating, painful, excruciating exercise for everybody involved — all the way from pharmaceutical companies who are funding this exercise to healthcare systems who are trying to execute it, to research coordinators, patients, you name it. I mean, the whole cycle is very inefficient and slow because it is very manual, and it is very manpower-intensive. 

It also lacks a feedback mechanism. The pharmaceutical company spends years designing that protocol based on their best guess of what how this trial should run, and they distribute it across the study sites, and the only way to figure out if this was a good protocol or not is that they sit and wait for all of those sites to be recruiting patients. That is inefficiency number one, I think, and inefficiency number two is on the healthcare system side.

Before we make a decision to join a clinical trial or not, we do some preliminary feasibility assessments internally. Those can include different types of queries from structured data within our electronic health record, but most of the time, to tell you the truth, it is a gut check. And then we sign up, and invest a lot of money in setting up the infrastructure for the trial and hiring coordinators. The principal investigator is dedicating their time, and from that point on, it’s a manual review with those coordinators looking at the electronic health record to find the patients.

The end result s that more than half of our trials are left lacking. We were very much aware that we were only meeting 51% of our enrollment goals across our clinical trials portfolio. At Cleveland Clinic, we run 1,000 clinical trials at any point in time, so we have a lot of activity in that space. So when you are 50% efficient and that is one-third of your whole volume for clinical research, that’s not a good place to be.

HCI: You mentioned that these assessments were looking at structured data. Does the LLM, in this case, allow you to look both at structured and unstructured data in a in a way that the manual chart reviews did not?

Jehi: Yes, that is the differentiator that we’re talking about here. The study criteria for clinical trials are typically a lot more complex than what you can get from looking at structured data. Structured data might give you that somebody is this old, they’re this sex, they have this diagnosis, and maybe they have this lab value. But clinical trials require criteria to be defined that have to do with whether this patient has taken medication A, but not medication B, and definitely not within the last six months. So with criteria this complex, even with the best platform that accesses structured data in the electronic health record, it is never going to have this level of granularity. We tried many other solutions — both internally developed or with outside parties. And throughout all those pilots, what we would end up finding is that those queries that we are getting from platforms like this are neither sensitive nor specific, so we might as well have saved ourselves all of that pain and just have the coordinators manually do it. 

I get approached by many companies, but Dyania from day one was different because their technology was different. So that’s why I decided it’s worth it to give them a chance and work with them on the pilots that we did with them, because the technology was convincing.

HCI: I read that in addition to the pilots, Cleveland Clinic has invested in Dyania as well. Was that the idea from the beginning? And did that involve brining in the innovation arm of the health system?

Jehi: My strategy is that whenever we’re looking at vendors and partners that we want to work with, before we commit, they really have to demonstrate that they can deliver on what the promise is. So we worked very closely with the Dyania team to set up the pilots that we ran to make sure that their technology works in our environment, right? So we started with those pilots, and it became clear very early on that the Dyania technology is different.

I wasn’t sure if we had a mechanism to invest in outside companies, so I reached out to our innovation team at the clinic to ask. Initially, the answer I got was no, but I kept pushing. Then several months later, the innovation team came back to me and said we may have a way to make this work. So then we worked with them, and they were tremendous. So, of course, the innovation team should get the credit for the investment and the partnership.

HCI: Eirini, does it matter which EHR the customer is using? And is there quite a bit of customization that needs to happen at the local level so that the Synapse AI platform can do its work?

Schlosser: I think there are a few parts of this question that are worth addressing. So first, taking a step back, I think one of the big areas is not just that we’re accessing EMR data holistically, inclusive of both unstructured and structured data. But what has not even been possible for a single LLM off the shelf is the ability to deduce multi-data, multi-note source conclusions. So what do I mean by that? Information could be spread across dozens of notes and labs, and all of that information could be driving a singular conclusion. So if you think about how a physician might read a medical history, they might read all of it and say, based off of all of these factors, I am going to apply medical reasoning and logic and deduce this conclusion. Off the shelf, your standard foundational models are built to process one note. They have a context window and answer a question about that note. We spent the majority of the pandemic with about 25,000 physician hours annotating data to train our own models. That really allowed this to be quite different. I think that gave us a head start, but it also gave us more of a logical head start to focus on some of the more medically nuanced problems. 

it’s not necessarily related to which EMR system a healthcare system is using, but rather how the data has been organized. In large healthcare systems that are often acquiring other hospitals, the data needs to be normalized in a way that is consistent. So if you’re looking at hematocrit values, if the data is not mapped or labeled in the same way in terms of the field names, that can be quite complex. 

HCI: What’s the plan for getting this into other health systems? And can Cleveland Clinic help with that?

Jehi: The clinical trial recruitment and the optimum utilization of our clinical delivery resources — those challenges are universal across all healthcare systems. So we are very committed to helping the community as a whole. Anything that we can do to help Dyania advance and build that technology disseminated in other places, we will do because otherwise it will be very inefficient for them to start from scratch. As a chief research information officer, I can tell you, I am not going to be spinning up infrastructure for 10 different vendors that do the same thing. I’m only doing it once, and my counterparts across the other healthcare systems are the same way, so we have to work together.