Exploring the Weaknesses and Strengths of an Innovative Technology
As IT healthcare analysts we are biased towards excitement for generative AI, but also cautious in its integration in the business at all costs, especially when we refer to healthcare organisations. It’s impossible not to be impressed, excited and terrified when you’re shown the latest technology.
Researchers use it to investigate genes and DNA to identify patterns and make predictions regarding disease progression in nanoseconds, instead of normally wasting human years. A first generation of generative AI has already been considered to facilitate and automatise many clinical processes: an effective case is the personalisation of care plans.
For example, generative AI algorithms can be used to refine and further personalise engagement with patients directing them to the right resources across multiple clinical systems, improving their experience and optimising their pathways.
Nevertheless, what is still missing is to understand whether, when and how healthcare organisations really need generative AI and when the decision is out; they need to define how to govern it and its risks.
Download eBook: Generative AI in EMEA: Opportunities, Risks, and Futures
The Potential Risks of Generative AI in the Healthcare Industry — Regulations will Be Needed
Governments, public authorities, industry experts, academia should have deep discussions to develop policy frameworks that both regulate potential harms and unlock benefits. They should access a collective debate and forge a collective path forward.
As already seen for AI technologies, also for generative AI, without the right rules and protections, this is going to get seriously out of hand, and quickly. And for the healthcare market, these words resonate more and more for several reasons:
- First, regulation plays a key role when generative AI is touching sensitive medical data and its intersection with the benefit for the healthcare community and us all. A simple example would be the use of personal medical data to conduct drug discovery and clinical trials.
Is it “right” to share our personal healthcare data with healthcare professional scientists to get innovative care treatments and drug discovery for the entire population? While this issue of protecting sensitive patient data from being disclosed without the patient’s consent has already been raised with the adoption of AI-based applications. In the case of generative AI, it’s even more difficult to manage.
For instance, patients’ consent can’t be easily exercised in the case of an unlearning process. Removing selected data points from a model might affect the performance of the model itself.
- Second, the risks of abuse are extensive because the accuracy of the responses from these generative AI tools largely depends upon the data used to train them. Without a real and human understanding of the healthcare topic under the analysis, these models create and predict what’s statistically likely or looks good, but not necessarily true.
This will cause reasonable concerns for their use in clinical practice, which necessarily needs immediate regulation.
- Third, the IT infrastructure underpinning generative AI requires huge investments from healthcare organisations. To perform efficiently and effectively, these large language models need continuous training on real-world health data. But this requires major investment in clusters of compute, storage, networking, and systems infrastructure software.
Furthermore, resources are needed to manage, optimise, scale, and secure the entire infrastructure and associated applications to prevent privacy breaches and ensure business continuity.
Register for the Webcast: Generative AI in EMEA: Opportunities, Risks, and Futures
The Potential Benefits for the Healthcare Industry Are Significant
Despite the concerns surrounding generative AI, its potential benefits for the healthcare industry cannot be overlooked. By harnessing this technology, the healthcare sector can:
- Improve workforce experience:
- Streamlining clinical documentation, generating patients’ histories, referrals, etc. suggest order entry.
- Helping to explain to patients their medical conditions in simpler terms and in an empathetic way.
- Analysing patient data, identifying patterns and make predictions regarding disease progression, treatment response, and suggesting treatment plans.
- Improve quality care:
- Improving patient experience by answering basic questions, explaining medical terms, scheduling appointments, directing them to appropriate resources.
- Helping to collect more accurate health data from different sources (wearables, conversations, EHR) to support personalised health recommendations.
- Enriching digital therapeutics solutions capabilities, expanding the capabilities of remote care and treatment.
Generative AI holds immense promise for healthcare, but we must strike the right balance between innovation and safeguarding patient interests. Collaborative efforts involving governments, providers, industry experts, and academia, are crucial to develop policy frameworks that address concerns, ensure data privacy, validate accuracy, and optimise the integration of Generative AI in healthcare.
Are you more worried or more excited about generative AI? Please share your thoughts with us, and in the meantime, we invite you to read our latest research on the topic.
If you are interested in knowing more about IDC Health Insights’ upcoming research, please contact Silvia Piai or Adriana Allocato.
