Interviewees: Carolyn Lam,1,2 Li-Ming Gan3,4,5
1. National Heart Centre of Singapore, Singapore General Hospital, Singapore
2. Cardiology, Duke-NUS Graduate Medical School, SGH Campus, Singapore
3. Early Clinical Development, Research and Early Development, Cardiovascular, Renal and Metabolism (CVRM), BioPhamaceuticals R&D, AstraZeneca, Gothenburg, Sweden
4. Department of Molecular and Clinical Medicine, Institute of Medicine, Sahlgrenska Academy At the University of Gothenburg, Sweden
5. Department of Cardiology, Sahlgrenska University Hospital, Gothenburg, Sweden
Disclosure: Lam is supported by a Clinician Scientist Award from the National Medical Research Council of Singapore; has received research support from Bayer and Roche Diagnostics; has served as consultant or on the Advisory Board/Steering Committee/Executive Committee for Abbott, Actelion, Amgen, AnaCardio AB, Applied Therapeutics, AstraZeneca, Bayer, Boehringer Ingelheim, Boston Scientific, Cytokinetics, Darma Inc., EchoNous Inc, Impulse Dynamics, Ionis Pharmaceutical, Janssen Research & Development LLC, Medscape/WebMD Global LLC, Merck, Novartis, Novo Nordisk, Prosciento Inc, Radcliffe Group Ltd., Roche Diagnostics, Sanofi, and Us2.ai; and serves as co-founder and non-executive director of Us2.ai. Gan has declared no conflicts of interest in relationship to this product.
Acknowledgements: Medical writing assistance was provided by Jennifer Taylor, London, UK.
Support: The publication of this feature interview was supported and reviewed by EchoNous.
Disclaimer: The opinions expressed in this article belong solely to the named interviewees.
Citation: EMJ. 2021;6[4]:40-45.
INTRODUCTION
December marks the launch of Us2.ai’s automated cardiac analysis platform1 as a cloud-based service on the EchoNous Kosmos device (Seattle, Washington, USA). The world’s first fully automated solution for both 2D and Doppler images, cleared by the U.S. Food and Drug Administration (FDA) for 23 parameters (data on file), will now be available on the only handheld ultra-mobile tool offering diagnostic grade imaging with continuous-wave Doppler capability.
In these interviews with the EMJ, Carolyn Lam, Senior Consultant Cardiologist and Director of Women’s Heart Health, National Heart Centre of Singapore, and Li-Ming Gan, Chief Physician, Department of Cardiology, Sahlgrenska University Hospital, Gothenburg, Sweden, and Vice President and Head of Early Clinical Development, Cardiovascular, Renal and Metabolism, AstraZeneca, Gothenburg, Sweden, shared their experiences of using artificial intelligence (AI) tools in cardiology and their vision for the future.
Carolyn Lam
Use of Artificial Intelligence in Medicine
Lam pointed out that the use of AI in medical imaging is not new: “If we were to walk into a hospital now and get a chest X-ray, I think you’ll be surprised at how much AI is used to pre-screen that X-ray to pick out, for example, nodules and other abnormalities, before the radiologist reads it and signs off on the report.”
The purpose is to offset manual tasks that inherently involve human error. “We can never be fully reproducible in manual measurements; this refers not only to intra-observer variability (difference in repeated measurements by the same observer), but also to inter-observer variability (differences in measurements between observers),” she said. “AI affords a standardisation, with complete reproducibility, improved precision and increased efficiency. Regardless of which doctor you meet, you get some basic level of screening and detection.”
Rather than replacing doctors, AI frees up time for doctors to spend with patients to look at complex and rare cases. Lam noted that not all AI works by a black box approach where it’s hard to fully understand how results were derived, and doctors are expected to ‘trust’ the AI. Rather, other AI approaches recapitulate what the human does. Doctors examine the echo view, perform segmentation, trace the border of the heart, take measurements, and compare them against guidelines to determine if they are normal or not to decide what action is needed. Us2.ai recapitulates that process, with full explainability in the report, showing end-users exactly which views were used and how annotations were made to derive a particular measurement, further allowing users to edit the measurements if needed. This keeps doctors in full control, while relieving them of the tedium of repetitive manual tasks and increasing their efficiency manyfold.
Introducing Us2.ai and EchoNous Kosmos
Lam explained that Us2.ai, referring to ‘us’ and ‘ultrasound’, is the software, whilst EchoNous is the hardware. The software completely automates the analysis, measurement, and reporting of an echocardiogram. The hardware is the handheld device used to acquire the images from the patient. The Kosmos is mobile, the probe plugs into a tablet, and is the only mobile hardware that can acquire not just 2D but all Doppler images including pulse wave and continuous wave. It also comes with AI guidance to help the user point the probe in the correct way.
“Together the combined product is truly a match made in heaven,” said Lam. “Because you have the best-in-class mobile hardware with AI to guide acquisition. And once you get those best quality images, Us2.ai gives you a full report, completely automatically.”
She acknowledged that a criticism of AI in general is that it appears to work only in the population it was derived from. Us2.ai, however, has been externally validated in large real-world cohorts in Taiwan, Singapore, Canada, and the USA (data on file).
Vision for Automated Cardiac Analysis
“The short-term vision that we started with was to alleviate my carpal tunnel syndrome from sitting in a dark room doing 250 clicks and spending 30 minutes for each study just reading it and reporting it,” said Lam. “I did it that way because that’s how I was taught. We spend so many years training that we don’t question that we have to spend hours just doing that.” The experience sparked an ambition to automate those manual tasks, improve the workflow in the echocardiography lab, and make the measurements more reproducible.
The moonshot vision is to democratise echocardiography (ultrasound of the heart) so that it is no longer the preserve of cardiology institutes but is performed by general practitioners (GPs), pharmacists, community nurses, and even patients. “My vision has definitely grown as I’ve seen it come to life,” said Lam. “It’s not that far away that [patients] could even do a medical selfie of [their] heart with this mobile AI combination.”
There were many challenges to meet along the way. Particularly memorable was showing the product to other clinicians, especially echocardiologists. Lam recalls, “I was so nervous because I anticipated that my colleagues reading ‘echos’ everyday may be insulted by ‘AI created to what they do’. Indeed, I have met scepticism at first mention; yet, thankfully once they see the software in action, they become converts. Because, trust me, none of us like doing that manual stuff. We like to be in control, which our software gives you. We like full explainability, you show me what you did and only if I agree I accept it, which again our software provides. Doctors like that. But we don’t like doing the manual stuff, which the software removes.”
U.S. Food and Drug Administration Validation Study
The landmark study that led to FDA clearance for Us2.ai was performed at a top echo core lab and included 600 echo studies from the USA (data on file). The bar set by the FDA was that the automated readings for 23 variables had to be completely interchangeable with human readings. The variables are measurements deemed clinically important by international societies (European Association of Cardiovascular Imaging [EACVI],2 American Society of Echocardiography [ASE]3) for a comprehensive transthoracic adult echocardiogram. The primary performance metric used in the study was the individual bioequivalence co-efficient (IEC), where 0 means that the variance between a machine read and a human is the same as between two humans. “I want to remind you that these were top expert human readers so that’s already a very small variance,” added Lam.
The performance success requirement set by the regulators was a non-inferiority margin of IEC <0.25, and this was achieved for all 23 measurements with IEC for some variables near 0 or even negative. “That was a real eye opener to me because that means that the variance between a machine and a human was actually less than between human and human for quite a number of the measurements,” she said. “And that’s the way it’s going to go because algorithms will keep improving. It’s really quite amazing.”
“That’s not even touching on the time-saving,” Lam continued. Performing the full suite of 23 measurements in each patient took human readers an average of 40.0 minutes compared with 1.2 minutes for Us2.ai. “You can imagine for 600 studies it’s a massive gain because the software just had to run for 12 hours. We turned it on, went to sleep, and the results were all available the next day. That’s the potential.”
In addition to the automated measurements being accurate, i.e., interchangeable with top human reader measurements, they were also reproducible. Lam described how human readers can only pick one or two frames of a full echo study to perform the measurements on. “We cannot possibly measure every frame of every video in one patient study, it’s just humanly impossible. But the software does. And in measuring everything all the time it is completely reproducible. In other words, as long as the algorithm hasn’t been updated and you give us the same study it will produce the same results 100% of the time. Whereas, you cannot say that about a human being. If you give me the same study twice, I will for sure have some variability in my manual measurements; it depends which frame of which cardiac cycle I choose, for instance.”
Improving Outcomes for Patients
Lam highlighted that the combination of automated AI and a handheld device has the potential to facilitate the early diagnosis of patients with heart failure, both by enabling GPs to perform imaging and by increasing the efficiency of echo labs so that patients are not waiting for months for a test. “The sad fact is that one in six elderly patients with heart failure have their diagnoses missed at a GP’s, the reason being they tend to be elderly, and the symptoms are non-specific (breathlessness, swollen ankles), and GPs don’t have access to echo now since it’s restricted to cardiology centres. On the flipside, 80% of patients get their first diagnosis of heart failure with an unplanned hospitalisation, despite the fact that many of them have had symptoms for up to 5 years before presenting. Those months when they have had symptoms and the diagnosis was missed really represent opportunities lost to treat the condition, prevent hospitalisation, and change the disease trajectory with effective medications and devices that we already have.”
She also envisages nurses performing echo in patients’ homes, which will save time travelling and waiting for their appointment. “COVID has taught us that patients don’t want to be in hospital and, humbly, they don’t even need to be in hospital. Tools like the Kosmos and Us2.ai combination are essential for home-based care of heart failure.”
Li-Ming Gan
Path into Using Artificial Intelligence Tools
As a clinical specialist in non-invasive cardiology with a special focus on ultrasound-based cardiovascular imaging, Gan has been doing clinical diagnostics and research using ultrasound for more than 20 years. He was introduced to the Us2.ai AI tool through a longstanding collaboration with Lam on non-invasively assessing heart failure with preserved ejection fraction (HFpEF) and microvascular function. In the pioneering PROMIS-HFpEF study, they demonstrated that microvascular impairment is the main feature of HFpEF.4 “Despite their very different clinical presentation, the heterogeneity in clinical features is not reflected in the phenotype as the majority of these patients suffer from microvascular disease,” he said.
Multiple different cardiac features in ultrasound imaging can reflect microvascular disease but non-invasive coronary flow measurements require a challenge test, high-end machinery, and skilled staff. The idea then arose to use AI to automatically predict microvascular function based on ordinary echo images. The PROMIS-HFpEF study also revealed the key features that are highly associated with impaired microvascular function of the heart, for example right ventricular motion and left atrial motility, and these can be readily measured using conventional echo.4
The ultimate aim is to describe a more specific phenotype in each patient so that treatment can be personalised. “HFpEF is a syndrome meaning diagnosis can be difficult,” said Gan. “With AI-based assessment you automatically get all these echo measures which helps with diagnosis. Going forward that should also make recruitment into clinical trials much easier.”
A Comprehensive and Reproducible Tool That Saves Time
It may take up to 1 hour to measure all relevant cardiac parameters after image acquisition, which in itself is a labour-intense session and requires a skilled and well-trained operator, noted Gan. With the EchoNous AI-assisted acquisition, he has noticed a dramatic improvement regarding how quickly he can teach beginners to perform echo examinations with impressive image quality. Also, with the Us2.ai cloud-based analysis tool, the 1-hour measurement time is down to less than 1 minute for image upload. AI performs the measurement with extremely good reproducibility and accuracy.
Gan admits he was hesitant in the beginning because of previous experience with other AI algorithms that were in-built into commercial machines but never performed like an experienced operator. That meant adjustments were always needed afterwards. “I have worked with this Us2.ai cloud-based AI tool for a while now, and I do see the performance is excellent,” he said. “Just by uploading nine images you get 23 FDA-endorsed cardiac parameters, meaning they are of very high quality. And when I look back at how the software measures the original images, I very seldom have anything extra to add. That means it’s in agreement with how a well-trained operator would do it in real life.”
Gan pointed out that because most of the measures, including cardiac output and left ventricular outflow tract velocity, are done automatically, they can be performed over multiple heartbeats, which is too time-consuming for manual acquisition. The AI software then calculates an average of all the signals. “I do believe the final value from AI is much more robust compared with manual measurement,” he said. “And even more fascinating, if we do measurements on a conventional set of data, it takes usually half an hour to an hour depending on the number of variables. But the software can provide the full list of measures instantaneously after the images have been uploaded.”
The reproducibility has also been impressive, added Gan. In a pilot validation study, his team trained medical students who had no experience with echocardiography. Over 2 days, these beginners were shown the basic images they needed to acquire and how to use the Kosmos AI-guided tool, which shows where to place the probe to get the best image. After the newly trained medical students acquired the images from 20 patients, two experienced operators repeated the measurements manually on the same day using a gold standard high-end machine. The question was, could a beginner, after brief training with AI-guided acquisition and with fully automated AI measurements, mimic or predict what could be acquired by an experienced operator? “The coefficient of variation was less than 3% for ejection fraction which is amazing,” said Gan. “I never expected that. It was almost better than my own intra-observer variability. This was a real eye opener because we all know echo is highly variable and user dependent. And now an AI-powered machine could do that so reproducibly and in so great agreement with expert operators. The AI-guided tool is useful for standardising images and can be used by both beginners and experienced operators. We need to figure out whether this tool can bring down variability even further.”
The pilot study provided evidence that echo-beginners can acquire accurate measurements of ejection fraction using AI-powered software. Gan is also interested to know whether AI can reduce inter-day variability among skilled operators. And on top of that, can it lower variability of more challenging measures such as left artrial motility, left ventricular strain, and right ventricular motility.
Implementation of Artificial Intelligence Tools
Gan sees numerous uses of AI-based echocardiography analysis. One, which can be put into practice now, is to diagnose cardiac conditions. “I will most likely use this machine in everyday clinical practice instead of a stethoscope because you see all the cardiac parameters you want to know instantaneously,” he said. “Instead of guessing who is having aortic murmur, insufficiency, etc, you see it directly with such a machine.”
A second use is as a screening tool for clinical studies, where a research nurse could be trained to measure ejection fraction and diastolic parameters and thereby recruit patients instead of waiting for an experienced sonographer. Third is to assess changes in cardiovascular parameters, both in a clinical setting to follow progression of disease and to capture clinical trial endpoints. “Trial endpoints are the highest end of validity,” noted Gan. “In these studies, we care about a few percentage points improvement of ejection fraction and are keen to keep the variability as low as possible, whereas in the clinic we are happy with 5% as a clinically meaningful change.”
This precision would help trialists recruit the desired patient population, which could improve the likelihood of detecting any effect of a new therapy. “This is particularly relevant for Phase 2 trials that require a substantial number of subjects to address questions such as would a drug improve any of these cardiac parameters,” he added. “Some of the inclusion criteria for HFpEF studies are highly variable and many are cumbersome to measure. One of the reasons for the variability of the measures is that the sites use different machines, some of them high-end, some of them low-end, and it is not standardised. So even if the analysis is done by a core lab the acquisition itself implies a lot of variability. With this package of EchoNous and the AI-powered software you can standardise the acquisition across all sites and take an average value from multiple measures. I think this approach will most likely dramatically improve accuracy and inter-site variability.”
The reduction in variability could have a huge impact because when studies are designed, the calculations to determine the patient population required to demonstrate efficacy are dependent on the variability inherent in having multiple centres, different skills and operators, and different machines. Gan said, “It’s an obvious mathematical equation that if you reduce the standard deviation by half, you need only half the number of patients which is a dramatic saving for the clinical trial and a dramatic improvement in trial quality. It could be the case that in the past we threw out a drug because we were not able to detect any efficacy just due to the noise of the method.”
Closing Remarks
While Gan believes that automated echocardiography will become invaluable in clinical practice and in research, he also envisages it being used to increase heart failure awareness in the community. “We know heart failure is dramatically underdiagnosed with some patients struggling with their symptoms for years before being hospitalised, diagnosed, and treated,” he said. “The problem is that it takes a very long time before patients get an echo of the heart. That means GPs and others measure a lot of circumstantial surrogate markers for a potential heart problem. Imagine this AI tool being implemented in a GP clinic.” Early access to automated echo is being piloted in the AstraZeneca Heart Bus project in the Gothenburg region, which screens people in the community with dyspnoea. “This type of point of care in our diagnostics to exclude heart failure and other conditions will probably become a successful approach in the future,” he said.
Both Gan and Lam noted that some level of caution is needed in the early stages of introducing this technique. “There is no question that this can dramatically improve the workflow of physicians, especially cardiologists,” said Gan. “But we still need to explore how the level of the beginner and their willingness to learn may or may not influence the outcome.”
“AI in general will always struggle with the very rare cases because it needs to learn from something and datasets are not infinite, so I think that we should recognise those limitations,” said Lam. “We should embrace the advantages in automating highly manual tasks, but then never see it as replacing doctors who are still needed for that human touch and ability to manage the complex cases. Another crucial point is that we should always hold AI up to the same bar that we do for therapeutics: it should be validated and tested with the same rigour.”
