A Panel About AI to Ask the Oldest Question in Science

The Panel Discussion on „Understanding in the Age of AI“ opened with a clarification that turned out to be the most important sentence of the afternoon: Heiner Linke – physicist, director of Sweden’s foremost nanoscience institute NanoLund, member of the Swedish Royal Academy of Sciences, and until #LINO75 Vice President of the Council for the Lindau Nobel Laureate Meetings – had spent months looking forward to this panel. He opened it with a question he did not try to answer himself: “Who needs to understand something for it to be understood? Is it humans, is it a model, or how should that work?”

The 600 Young Scientists gathered in the Main Hall of the Inselhalle had heard a great deal about AI throughout the week. This session was different. It addressed a more fundamental question: What does it actually mean to understand something?

An Unexpected Result – And What Happened Next

Anne L’Huillier – Nobel Laureate in Physics 2023 for her work on attosecond pulses of light – began with a story from 1987. Her group had exposed a gas of atoms to a laser field and observed something no one had predicted: high-order harmonics, bursts of light at far higher frequencies than expected.

“We made an unexpected experimental discovery. And here I want to challenge AI. I mean, this is something that only can happen in a laboratory with experiments. So here I feel kind of secure.”

They simulated the phenomenon. The calculation required solving the time-dependent Schrödinger equation and coupling it with Maxwell’s equations to describe how many atoms emitted light together. After three or four years, the simulations agreed with the experiment. But agreement was not the same as understanding. – “Do we understand the phenomenon? I would say no. Understanding is something else. It’s really an intuition and an image.”

For L’Huillier, the decisive step came later, when physicists developed a mental picture of what was happening: an electron tunnelling away from an atom, being driven away by a strong laser field, returning, and colliding with the atom again – producing extremely short pulses of light. The model was simplified. It was not a complete description of reality. But it gave researchers a way to think: “From this model, this intuition that all physicists in the field develop, we could take the next step, try to measure these pulses, to use them, et cetera.”

The point was not that the model was perfectly true. The point was that it allowed scientists to move forward.

The Tie That Shouldn’t Work – And Still Does

William D. Phillips – Nobel Laureate in Physics 1997 for his work on laser cooling and trapping of atoms – continued the discussion with an object he was wearing: his tie.

It showed the famous Bohr model of the atom – electrons orbiting a nucleus like planets around the sun.

“This was revolutionary. This changed people’s thinking. And the model that Anne talked about is based on this idea, and it’s completely wrong. And everyone knows that it’s completely wrong.”

The problem is that electrons do not move around atoms like planets around the sun. Quantum mechanics replaced this picture with wave functions and probability distributions. And yet the image has persisted. “The image is so powerful we still use it. And it was such an important image that it’s still used today to gain insights.”

For Phillips, this example showed why understanding in science cannot simply be reduced to producing correct calculations. A numerical solution can give the right answer without giving scientists the intuition they need to make the next discovery.

“This question of whether we truly understand something, if we present the problem to AI and AI gives us answers that we didn’t know, because of course that’s the gold standard for any new theory, is that it will predict something that we didn’t already know.”

The question, then, is whether a new prediction generated by AI automatically creates new understanding. Phillips argued that the answer depends on “what do we mean by understanding?”. And he pointed out that this issue is not new. Science has long relied on calculations that produce correct answers without necessarily providing the conceptual insight needed to move the field forward.

As his colleague Jean Dalibard once put it: if you write down the optical Bloch equations and solve them, you may obtain the correct answer whether you understand the physics or not. The equations work. The intuition may still be missing.

The Generation That Studied Without ChatGPT

Klara Bonneau – Young Scientist and PhD researcher at Freie Universität Berlin, working on neural network models for protein dynamics – brought a different perspective to the discussion: that of a researcher who had completed most of her education before generative AI became widely available.

She explained that she had finished high school in 2016 and her master’s degree in 2022, only a few months before ChatGPT was released. „So basically I did my whole education without what is generally commonly known today is AI – large language models, you can just ask a question and they answer you. During my studies, I felt there was mostly the opinion that AI is a tool for people who don’t really want to understand the science behind it. People just wanted to throw data at it and let the AI do the heavy lifting.“

But that perception had changed. AI systems were now producing results that opened possibilities  previously considered unrealistic.

Bonneau brought two questions to the panel. The first concerned how Young Scientists should adapt to this rapid change in scientific culture. The second addressed one of the central issues of the discussion: the relationship between understanding and trust. She argued that, historically, trust in scientific models has often been connected to the ability to trace back why a prediction was made. Scientists could examine the underlying mechanisms of a model and understand why it produced a particular result. Using protein modelling as an example, she explained that researchers can usually relate a prediction to physical concepts within the model – for example, interactions between atoms and the forces that determine a protein’s behaviour.

With AI systems, she suggested, this connection becomes less direct. “And now with AI, this link is kind of broken. And so do you think we can – or should we – learn to trust something despite this lack of prediction traceability?”

AlphaFold, Crystallography, and the Question Nobody Asks the Crystallographer

John M. Jumper, Nobel Laureate in Chemistry 2024, for the development of AlphaFold, and until recently Senior Research Scientist at Google DeepMind before his announced move to Anthropic – had been visibly waiting to respond. He started not with AlphaFold, but with crystallography, a field that has long relied on experimental insight, models, and intuition to reveal the structures of proteins.

His argument: science has always relied on models and theories that were not complete explanations.

The difference with AI is not the lack of transparency itself – it is that we are more aware of it. Jumper illustrated this with a question that crystallographers know well:

“Do you understand why those conditions crystallized your protein? And they will almost invariably say, no, they have some intuitions on kind of moves that might crystallize a protein, things you might try, but why these conditions worked and not those, I don’t know.”

His point was that science has always progressed through a combination of understanding, empirical evidence, and practical tools. Researchers often use models that work reliably without providing a complete explanation of every underlying mechanism.

For Jumper, this is also how AI systems should be understood. Scientists do not simply develop a theory and then test one perfect prediction. They engage in a dialogue with nature – through experiments, simulations, and increasingly through computational models. “And machine learning developers do exactly the same thing to build a system that generalizes well. And then we’ve produced an artifact just like a simulation package.”

That artifact then becomes part of scientific practice. Researchers use it to generate hypotheses, develop intuition, and decide what experiments to perform next.

The question is not whether AI systems understand in exactly the same way humans do. The question is whether they can become useful instruments in the process through which scientists develop understanding.

Jumper emphasized that science has always involved creating local theories and practical abstractions: “We always build both these understandings and these local theories and this kind of patchwork of theories. And that’s how science progresses and that’s how we ultimately make technologies like these that work sometimes in near-perfect understanding, sometimes imperfect.”

Trust Without Full Understanding

Returning to Klara Bonneau’s question about whether scientists can trust AI systems without being able to fully trace how they produce their predictions, Jumper pointed to an important distinction: trust does not always require complete understanding.

“There is one interesting aspect in AlphaFold that is trust, not understanding, in the following sense: AlphaFold actually makes two predictions. One is the predicted structure of the protein, and the second is the predicted error in that structure.” In other words, AlphaFold does not only provide a prediction; it also provides an estimate of the uncertainty of that prediction.

This distinction allows researchers to decide when a result is likely to be reliable and when it should be treated with caution. The mechanism behind this was conceptually simple. During training, AlphaFold was taught not only to predict protein structures but also to estimate how accurate those predictions would be. The system’s predicted error was compared with the actual error, and the model was adjusted accordingly.

The result was a practical measure of confidence that researchers could use.

“That works shockingly well for AlphaFold. It became something useful for experimentalists who use it every day to say, oh, AlphaFold believes this. Oh, AlphaFold doesn’t believe this. AlphaFold is so unconfident about that, it’s probably disordered.”

For Jumper, this illustrates a broader principle: sometimes the most useful form of understanding is not knowing every internal step of a system, but knowing how reliably it performs under different conditions. Scientists can investigate AI models through experimentation, much as they investigate natural phenomena. The goal is to understand what kinds of inputs lead to what kinds of outputs – and when a system is likely to fail.

What Makes for a Good AI Paper?

One of the most interesting exchanges of the session came during the audience discussion. The question did not come from a Young Scientist but from another Nobel Laureate: John M. Martinis. His question to John M. Jumper shifted the conversation from understanding to a more practical issue: how should scientists judge whether an AI system deserves their trust?

Jumper’s answer revealed what he considers the single most important criterion for evaluating AI research. Whether he is reading a paper or listening to a scientific presentation, he first wants to know how the model was evaluated. “There’s almost only one thing I care about. For the very first to decide if it’s worth my time, I want to understand what is not in your training set, and I want to understand what is in your test set.”

For Jumper, the crucial issue is whether the test set represents a realistic situation in which the system would actually be used. A scientifically meaningful evaluation requires discipline: the model must be tested on cases that genuinely challenge its ability to generalize. In other words, the decisive question is whether the evaluation reflects the conditions under which the system will ultimately be used in practice.

The exchange was notable because it connected two scientific cultures. Martinis approached the subject from the perspective of experimental physics, where confidence in a result depends on rigorous validation. Jumper’s answer translated that same principle into the language of machine learning: the credibility of an AI system depends not only on its performance, but on how convincingly that performance has been tested.

For Jumper, this is what distinguishes serious AI research from impressive demonstrations: not simply strong results, but convincing evidence that those results will hold beyond the training data.

The Colleague Down the Hall: Fermi and the Ideal AI Assistant

Returning once more to Klara Bonneau’s question about how Young Scientists should adapt to the age of AI, William D. Phillips offered an answer rooted in the way scientists have always learned from one another.

Before becoming an independent physicist, he thought that learning something new meant going to the library and looking it up in books. But when he began doing research himself, he discovered something different. “The way you did it was you walked down the hall until you found somebody who knew something about it and asked them about it.”

Phillips described three different ways of explaining science through three famous physicists: Paul Dirac, J. Robert Oppenheimer, and Enrico Fermi.

With Dirac, he said, one might understand the explanation but leave wondering how any human being could ever have discovered such an idea. With Oppenheimer, the explanation might feel within reach – something one could imagine arriving at with enough time and effort. But Fermi represented something different: the rare ability to make a difficult problem feel obvious after the explanation has been given.

“You take the same question to Fermi, and Fermi explains it to you, and you come out and say, how could I have been so stupid not to understand how to do that problem? Now, I think what we want out of AI is Fermi. We want to somehow program the AI so that it’s like the colleague down the hall, but it’s the good colleague down the hall that will explain things to us in a way that is so obvious that we can’t understand how we weren’t able to figure that out in the first place.”

But Phillips also added an important qualification: AI should support scientific thinking, not replace the process of developing that thinking. “I don’t think we should replace that colleague down the hall with AI. We should think of AI as just, you know, maybe the colleague that’s further down the hall. Now, is this good advice? Ask me tomorrow. I’ll probably give you a completely different answer.”

A Group Discovery – and the Role of Diversity

Anne L’Huillier returned to the discussion with a reflection on how scientific discoveries actually happen. In her view, major breakthroughs are rarely the result of a single individual suddenly finding the answer. More often, they emerge through what she described as a “group discovery” – a process in which understanding develops collectively rather than individually.

She referred to discovery as a process in which different people contribute different pieces: a PhD student, a postdoctoral researcher, a professor – each bringing ideas and observations until a new picture begins to form. “And suddenly all the pieces of the puzzle fall into place.”

For L’Huillier, this collective process is one of the most exciting aspects of research.

“To me, these are really fantastic moments, when you see something new emerging. And it’s a group discovery. You need the group to be diverse. For me, diversity is so important in research.”
The remark connected back to the central theme of the discussion: understanding is not simply the possession of information. It is a process of connecting ideas, perspectives, and experiences.

The Pilot Who Keeps Training

Near the end of the session, Heiner Linke asked Klara Bonneau whether the discussion had answered her original questions. She reflected on the role AI should play in the future of science.

“AI is an impressive tool, and we would not be good scientists if we did not go with the progress. We should definitely use it. But in the meantime, we should remember what is it that we want to keep that we learned before and that was useful before, and that we want to keep training like a pilot. And a plane has autopilot today, but still keeps training in a simulator. Some basic skills that in case of need, we should always have ready.“

Etually, the panel did not provide a final definition of understanding. Instead, it showed why the question remains open. Understanding may involve intuition, models, experiments, and conversations – whether supported by humans, simulations, or AI.

What matters is not only finding answers, but developing the ability to recognize patterns, test ideas, and turn information into insight. It may be the most honest summary of where this field – and this generation of scientists – actually stands.

This blog post is part of a series showcasing highlights from #LINO75. Most of the Scientific Programme from the recent Lindau Meeting is available in the Lindau Mediatheque. Watch the full panel “What Does It Mean to ‘Understand’ in the Age Of AI?”.