How Do We Preserve Trust in Science in the Age of AI?

To preserve trust in science in the age of AI, it is essential to prioritize transparency, ethical standards, and rigorous peer review. Scientists and AI developers should clearly communicate their methodologies, data sources, and potential biases to the public. Establishing and adhering to strict ethical guidelines ensures that AI applications are used responsibly and for the public good.

Additionally, fostering an environment of continuous scrutiny and independent verification of results through peer review and replication studies can help maintain credibility. By combining these efforts, the scientific community can uphold trust and demonstrate the reliability and integrity of their findings amidst rapid technological advancements – ChatGPT, when asked how trust in science can be preserved in the age of AI.

This compelling and yet weirdly generic argument was output by ChatGPT, one of the AIs that have recently been released to the general public. It’s a bit strange to start a discussion on trust and AI with a quote from AI, but that’s also what moderator Sibylle Anderl did in a Panel Discussion at LINO24, on Mainau Island. Anderl asked ChatGPT to briefly introduce participants.

For people unfamiliar with ChatGPT (if there were any in the crowd), it must have been pretty surprising: the AI output a perfectly convincing introduction of all panel participants. However, perfectly convincing is not perfectly true.

Hallucinations and Scepticism

For instance, when introducing Nobel Laureate and panel member David Gross, ChatGPT accurately summarized his field of work and major accomplishments  but simply made up a statement about Gross’ opinion on AI.

“My only use of ChatGPT is to try to find out what’s wrong with it, which is quite easy by the way. In fact, it told three lies here. That statement it quoted from me – I never made.”

AI hallucinations (the process where an AI generates incorrect or nonsensical information that appears plausible or coherent) are a major problem not just with ChatGPT but also with other generative models. The problem is compounded by the fact that oftentimes, we don’t really understand why it produces the output that it produces, including hallucinations.

“I think AI models are inherently hard to trust because it’s hard for people to understand what’s going on,” says Jaryd Ricardo Christie, a young researcher from the University of Western Ontario, Canada, who was also on the panel. “These models are sort of a black box. So it’s hard to really understand what’s going on under the hood. So I think what we should do as AI scientists is we should always be skeptical of these models. We should never trust them blindly.”

Despite all the interest and all the effort put into machine research, it’s still very hard (and oftentimes, impossible) to understand how these models work. Many generative models are essentially plausibility generators, but they have no real understanding of what they are generating.

“AI models, especially large language models (LLMs), are trained to come up with things that sound good … but they don’t know if it’s true or not,” says Gross. The Laureate continued to say that if AI could be trained to follow the scientific method, that would be great, but until that happens, it’s up to us humans to be skeptical of AI and ensure what it is outputting is factual and reliable.

Of course, members in the audience are very familiar with the scientific model and use it in day to day life, as well as their professional activity. But for members of the general public, things are different.

Plenty of Questions for Society

It’s one thing to ensure trust in AI in research; it’s another to ensure it for society.

“Of course, LLMs are, I think, very useful. We’re looking at creating digital assistants that can be there 24/7 to help people and that’s a wonderful thing. But we also have to worry about people not using this assistance as a crutch and not ever learning anything,” says Nobel Laureate Brian Schmidt, who was also on the panel.

Schmidt argued that the sense of accountability is crucial, and we as a society are still defining who is accountable for what. The Laureate expects AI to become “more insidious” and end up in all sorts of scientific instruments (as well as products accessible to the public, like self-driving cars for instance). Who is accountable when an error is made?

“The sense of accountability is about to become really important. There’s gonna be a transformation as this stuff [AI] gets buried into things. Accountability is going to be unclear and I encourage you to exercise extreme caution. If you have a beautiful new algorithm and you start killing patients, you are going to be accountable.”

Donna Strickland, another Laureate on the panel, echoes these ideas. However, she added that unfortunately, the scientific method is not as widely spread for the general public as it is for scientists.

“I think we have, especially in North America, a growing distrust of science,” Strickland noted. She referenced the coronavirus pandemic, which, especially in the early stages, brought a scientific process to the forefront of the entire world. When researchers were changing the recommendations for things like masking and public distancing, they were implementing and adding new data – but for many people in the public, this was interpreted as uncertainty and brought frustration. Something similar can happen with AI, Strickland notes. “Including AI in understanding this, we need the public to understand all these ramifications so they can understand what is real science and what is not.” Scientists from all fields can be a part of the solution by engaging with the public, and not just the public who already love science, says Strickland.

For the general public, it will be more important in the short term to deal with the new deluge of content that will hit society (and in some ways, is already hitting society).

“We are going to see in the next couple of years a proliferation of generated false material, whether written, video, images… to the point where you are not going to be able to tell if something is real or false unless it has a accountability digitally put in there […] of which person or organization stands behind this fact,” says Schmidt.

So any path towards maintaining trust in AI and in science should have two components: one where scientists continue to do a good job (whether using AI or not), and one where scientists engage the public.

At the Next Gen Session on Artificial Intelligence in Physics, young researchers took on the first part.

AI in Physics

Ranging from quantum machine learning to medical physics and air flow prediction, there was no shortage of AI applications in physics.

Jaryd Ricardo Christie’s research focuses on developing a deep learning model that integrates data from PET (Positron Emission Tomography) and CT (Computed Tomography) imaging to improve the prognostication of patients with resected lung cancer. This innovative approach aims to enhance post-surgical treatment decisions and patient outcomes by leveraging advanced imaging techniques and artificial intelligence. The system may identify patterns useful for diagnosis that could escape the human eye, even the eye of trained clinicians.

Christie, who was also on the earlier-mentioned panel, aims to create a reliable and accurate model to predict the likelihood of cancer recurrence in patients who have undergone surgery. In the clinic, you can’t afford to have any hallucinations or misleading conclusions. Of course, any model is bound to be imperfect, but any mistakes in this context are extremely costly. In the end, the model was forwarded to clinicians, where it helped improve their assessment of patients.

Anna Dawid-Łękowska works in another field of AI connected to trust: interpretable AI. Interpretable AI refers to artificial intelligence systems whose operations and decision-making processes are transparent and understandable to humans. Unlike “black box” models, interpretable AI provides clear insights into how specific outcomes are reached. Dawid-Łękowska works on applying interpretable AI to scientific problems at the Flatiron Institute.

The young researcher presented work where the primary goal is to create a robust, automated system that can identify and optimize laser cooling schemes for various atoms and molecules. Different atoms and molecules have unique electronic structures, making it challenging to develop a one-size-fits-all approach to laser cooling. Each species may require a tailored cooling scheme to achieve optimal results, and this is where machine learning comes in. The young researcher is working on a system that leverages large datasets and advanced algorithms to predict and refine the most effective laser parameters, significantly streamlining the experimental setup process and improving the accuracy of cooling outcomes.

Simon Wassing, from the German Aerospace Center (DLR), focuses on advancing the prediction of aerodynamic flows. He uses a novel approach that combines physics-informed neural networks (PINNs) and quantum circuits. Aerodynamic simulations, are crucial for aerospace engineering and the design of various aerodynamic structures. They are also another area where trust is crucial — you can’t afford to have any errors when dealing with planes and rockets.

The primary goal of this approach is to enhance aerodynamic flow predictions by integrating physics-informed neural networks with the computational power of quantum circuits. This hybrid approach aims to overcome the limitations of traditional computational fluid dynamics (CFD) methods. Beyond flight, the methodology could be applied to other fields requiring fluid dynamics simulations, such as automotive engineering, climate modeling, and renewable energy systems.

Accountability is Everything

Looking ahead, the integration of AI in scientific research holds great promise, yet it also demands careful consideration and proactive measures to maintain trust. The first step is maintaining scientific trust in AI and ensuring that it is used reliably.

The second step is arguably much more pressing – maintaining public trust. This is a monumental challenge, the panel agreed, but also one that brings opportunities. The panel ended by expressing cautious optimism. “I’m always an optimist,” says Strickland.

However, Schmidt concluded by once again pointing toward accountability as the key aspect. No matter what happens, we, as humans, are accountable for it, and we should accept that.

“When you start trusting a machine for accountability, that is the end.”