There is growing concern among some healthcare technology leaders that they may need to take a step back to ensure that the use of artificial intelligence – especially generative AI – is safe, appropriate, secure and morally sound.
Their potential benefits are enormous when used in combination with human guidance, leading to early diagnosis, improved disease prevention and overall well-being with the properly ‘tuned’ prediction algorithms. But some have raised alarm that the use of AI is already widening the digital divide, creating even more bias and furthering inequality.
Gharib Gharibi, who holds a PhD in computer science, is director of applied research and head of AI and privacy at TripleBlind, an AI privacy technology company. He has a strong opinion – based on his own research and experience training large language models – that AI should be viewed as enhanced intelligence, which can only be successful with human interaction and support. Healthcare IT news spoke with him to get his perspective on this and other topics.
Q. You say there is a growing digital divide and bias due to the misuse of generative AI in today’s healthcare environment. Please explain.
A. Generative AI, and AI algorithms in general, are programs that generalize based on data; and if the data already used is biased, so will the AI algorithm.
For example, if a generative model is trained on medical images collected from a single source, located in a geographic area with a predominant ethnic population, the trained algorithm will most likely not work accurately for other ethnic groups (assuming the patient’s ethnicity is a good predictive factor). variable).
Generative AI in particular has the ability to create synthetic patient data, simulate disease progression, and even generate realistic medical images for training other AI systems. Using biased, single-source data to train such systems can therefore mislead academic research, misdiagnose diseases, and generate ineffective treatment plans.
While diversifying data sources, both for training and validation, can help minimize bias and generate more accurate models, we must pay close attention to patient privacy. Sharing healthcare data can pose significant privacy concerns, and there is an immediate and significant need to find the right balance between facilitating data sharing and protecting patient privacy.
Finally, there is the ongoing debate about regulating AI in healthcare to reduce intentional and unintentional misuse of this technology. Some regulation is needed to protect patient safety and privacy, but we must also be careful about it, because too much regulation will stifle innovation and slow the creation and adoption of more affordable, life-saving AI-based technologies .
Q: Please talk about your research and experience training large language models and based on that your opinion that AI should be seen as enhanced intelligence, which can only be successful with human interaction and support.
A. My experience and research interests lie at the intersection of AI, systems and privacy. I am passionate about creating AI systems that can simplify human lives and expand our tasks accurately and efficiently, while at the same time protecting some of our fundamental rights – security and privacy.
Today, AI models themselves are designed to work with human users. Although AI systems, such as ChatGPT, can generate responses to a wide range of prompts, they still rely on humans to provide these prompts. It still has no goals or ‘desires’ of its own.
Today’s main goal is to help users achieve their goals. This is particularly relevant in healthcare, where the ability to process sensitive data quickly, privately and accurately can improve diagnosis and treatments.
However, despite the powerful capabilities of generative AI models, they still generate inaccurate, inappropriate, and biased responses. It might even leak important information about the training data, violating privacy; or easily fooled by conflicting input samples to generate wrong results. Therefore, human involvement and supervision is still crucial.
Looking ahead, we will witness the emergence of fully automated AI systems, capable of performing extensive, complex tasks without human intervention. These advanced generative AI models can be assigned complex tasks, such as predicting all potential personalized treatment plans and outcomes for a cancer patient.
It could then generate comprehensive solutions that would otherwise be impossible for human experts.
The immense data processing capabilities of AI systems, which far exceed the cognitive limits of humans, are crucial in this context. Such tasks also require calculations that would take a human lifetime or longer, making them impractical for human experts.
Finally, these AI systems are not subject to fatigue and do not get sick (although they face other types of problems such as concept drift, bias, privacy, etc.), and can work relentlessly around the clock and deliver consistent results. This aspect alone could revolutionize industries where constant analysis and research are crucial, such as healthcare.
Q. What are some of the guardrails you think need to be put in place regarding generative AI in healthcare?
A. As we move towards a future where generative AI becomes more integrated into healthcare, it is essential to have robust guardrails to ensure the responsible and ethical use of these technologies. Here are some key areas where safeguards should be considered:
1. Data Privacy and Security. AI in healthcare often involves sensitive patient data, so robust data privacy and security measures are critical. This includes using and improving current privacy-enhancing methods and tools, such as blind learning, Secure Multiparty Computation (SMPC), federated learning, and others.
2. Transparency. It is important that healthcare providers and patients understand how AI models make predictions. This may include providing clear explanations of how the AI works, what its limitations are, and what data it is trained on.
3. Limiting bias. Steps must be taken to prevent and correct AI biases. This includes diverse and representative data collection, techniques for detecting and mitigating bias during model training, and continuous monitoring for bias in AI predictions.
5. Regulation and accountability. There must be clear rules for the use of AI in healthcare, and clear accountability when AI systems make mistakes or cause harm. This may include updating existing medical regulations to take AI into account, and creating new standards and certifications for AI systems in healthcare.
6. Equal access. As AI becomes an increasingly important tool in healthcare, it is critical to ensure that access to AI-enabled care is equitable and does not exacerbate existing health disparities. This could include policies to support the use of AI in underserved areas or among underserved populations.
Setting up these guardrails requires collaboration between AI scientists, healthcare providers, regulators, ethicists and patients. It is a complex task, but necessary to ensure the safe and beneficial use of generative AI in healthcare.
Q. What are some of the data management techniques that you believe will help providers avoid biased results?
A. Reducing bias in privacy-preserving, explainable AI systems requires careful and effective data management, design, and evaluation of the entire pipeline of AI systems. In addition to what I already mentioned, here are several techniques that can help healthcare providers avoid biased results:
1. Miscellaneous data collection. The first step to avoiding bias is to ensure that the data collected is representative of the diverse populations the AI will serve. This includes data from individuals of different ages, races, genders, socio-economic statuses and health conditions.
2. Data pre-processing and cleaning. Before training an AI model, data must be preprocessed and cleaned to identify and correct any sources of bias. For example, if certain groups are underrepresented in the data, techniques such as oversampling these groups or undersampling overrepresented groups can help balance the data.
3. Bias audit. Regular audits can help identify and correct biases in both the data and the AI models. This includes reviewing the data collection process, examining the data for potential biases, and testing the AI model’s results for fairness across different demographic groups.
4. Feature selection. WWhen training an AI model, it is important to consider what features or variables the model uses to make its predictions. If a model relies heavily on a feature that is biased or irrelevant, it may need to be adjusted or removed.
5. Transparent and explainable AI. Using AI models that provide clear explanations for their predictions can help identify when a model is relying on biased information. If a model can explain why it made a certain prediction, it is easier to recognize when it is basing its decisions on biased or irrelevant factors.
Ultimately, controlling bias in AI requires a combination of technical solutions and human judgment. It is a continuous process that requires continuous monitoring and adjustment. And it’s a task well worth doing, because reducing bias is essential to building AI systems that are fair, trustworthy, and useful to everyone.
Follow Bill’s HIT coverage on LinkedIn: Bill Siwicki
Email him: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.