A multi-university initiative including GW announced new funding for research to improve the trustworthiness of artificial intelligence earlier this month.
The initiative, the Institute for Trustworthy AI in Law and Society — which launched in May 2023 and involves four universities including GW — announced five new grants, totaling $685,000, to fund research into the ethical use of AI. Valerie Reyna, a researcher with the School of Engineering & Applied Sciences and a project lead for Cornell University on the initiative, said her work will train medical AI to use more logical principles in their decision-making to improve patient outcomes and increase trustworthiness.
“We want to, on the one hand, make sure that society is able to take advantage of these remarkable technological achievements,” Reyna said. “But on the other hand, we want to make sure that there are appropriate protections and safety elements in this.”
The initiative is made up of four universities, the University of Maryland, GW and Cornell and Morgan State universities. The five new grants are funding projects to improve the quality of AI in health, autonomous vehicle safety, social media platforms, academia and educational settings, according to a release announcing the funding.
Reyna said a key part of how researchers determine the trustworthiness of AI is by applying human psychological principles to analyze the decision-making processes of AI models. She said researchers look for psychological principles in AI decision-making to see if the technology is making consistent decisions based on logic, which can be applied to a range of AI forms from chatbots like ChatGPT and Gemini to AI medical technology.
“If I ask you a bunch of choices and your decisions contradict themselves, then you’re not really making sense,” Reyna said. “And then we can say we worry that that’s not a rational decision process. If decisions are inconsistent with one another, it’s kind of a minimum condition.”
She said her research studies physician decision-making to understand how people make decisions in health care and will use that to train the AI health models to ensure they are based on real perspectives.
Reyna said a common problem in medical research is the lack of diversity among participants and scientists, which she said this project aims to change by relying on community participation to inform their results. She said most medical research focuses on expert opinions.
“We’re very much dedicated to the importance of community participation and active participation in research,” Reyna said. “And that’s extremely important to diverse communities and people from different backgrounds and people with different needs and bringing them into this discussion and in an active way in terms of helping shape the research.”
Experts in AI said the lack of trustworthiness is a problem in the AI field because of how popular its usage has become and the number of people making AI programs in recent years — many of whom have not properly trained their models, leading them to provide inaccurate information and decrease users’ trust.
Janusz Wojtusiak, professor of health informatics at George Mason University, said the biggest issue in AI right now is distinguishing the “good stuff from the not so good stuff.” Wojtusiak said because AI has gotten so popular, the market has become saturated with newer programs that have not been tested.
“It needs to do what it’s supposed to do,” Wojtusiak said. “It needs to be extremely well tested, so we know it actually does that thing.”
Wojtusiak said a lot of AI programs can be what he calls a “black box” where users feed it information and it gives them a result, but they don’t know how it got to that answer. Wojtusiak said research into the decisions of AI can increase user trust, and experiences with AI providing inaccurate information and a lack of information can cause people to distrust.
“All you have is an app that does something, maybe correct or maybe not,” Wojtusiak said. “But we just don’t know what it is. So it’s all about the transparency on what the thing is doing, how well it’s doing, and what’s been trained on.”
Peter Szolovits, a professor of computer science and engineering at the Massachusetts Institute of Technology, said the trustworthiness of chatbots like ChatGPT has improved since they first launched. He said about two years ago, he asked ChatGPT to write a synopsis of his career, and it got most of the facts wrong, but now, the model will tell you if it does not have enough information to give you an accurate answer.
“That’s one thing that has actually helped, and empirically, that does seem to reduce the amount of hallucination done by systems, although not to zero,” Szolovits said.
Szolovits said in the medical field, AI models succeed at making broad generalizations from large data sets that are mostly accurate. He said AI models run into problems with specific case studies, where it is more likely to miss key information.
“If you’re going to make clinical decisions about an individual patient, then I’m much more nervous about models,” Szolovits said. “Because basically if it misses some important thing in my medical record or if it makes up some important thing in my medical record, that could kill me, and I’m not interested in having that happen.”
Jennifer Igbonoba contributed reporting.
