Scientific models are essential tools for understanding complex, real-world systems. They allow researchers to test ideas, make predictions and uncover patterns that would otherwise be impossible to see.
In their new research published in Nature Communications, physics Professor André Longtin and his former PhD student Dr. Alexandre René developed a new technique that could improve how scientists evaluate and compare these models. The team is particularly hopeful that it could help researchers better understand neurological disorders such as epilepsy.
“Our colleagues at the University of Ottawa and other institutions study brain tissue and brain-wave activity from patients with epilepsy. They can record from the hippocampus, which is one area of the brain that often goes awry in epilepsy. Suddenly, an electrical storm will start and eventually stop, which represents a seizure. To this day, we still don’t have a clear idea of what goes on when the neural circuitry malfunctions,” says Professor Longtin, who co-directs the uOttawa Centre for Neural Dynamics and AI (CNDAI).
By fitting mathematical models to these patterns of brain activity, Professor Longtin and Alexandre René hope to find ways to better predict seizures and identify new targets for treatment, whether through surgery, stimulation or medication.
But here’s where it can get tricky: several different mathematical models can often explain the same observations, a problem that is especially common in neuroscience. Scientists must make the difficult decision of which model fits best before continuing their studies.
Finding the best fit
“Suppose two candidate models, A and B, differ in their description of the impaired brain cells. Each provides a reasonable fit to the seizure data but involves a different mechanism. Logically, even though we don’t know the true mathematical model underlying the patterns of activity, either could be correct,” Professor Longtin explains.
Traditionally, scientists compare models using different metrics to determine how well each one matches existing data and how accurately it can predict unseen data. A model with a high likelihood value is the one that best matches the existing data. However, because their data are only a limited sample of reality, the best “fit” model may not always be the most reliable one. If the experiment were repeated in the same or different labs, it would almost certainly produce slightly different results.
Seeing the bigger picture
This is where Professor Longtin and Alexandre René’s epistemic modelling framework offers a new perspective.
“My student Alex [René] really pioneered this technique. He challenged us to start from scratch and ask: ‘If similar models produce solutions that more or less fit the data, how do we decide which one to trust?’”
Instead of assigning a single score to a given model, as scientists typically would do, Professor Longtin and Alexandre René’s framework captures the full range of possible risks associated with that model.
“Instead of giving you just the risk for model A and the risk for model B, our technique gives you a probability distribution for these risks,” says Professor Longtin.
A probability distribution provides a picture of all the possible outcomes and the likelihood of each one occurring.
This allows researchers to see not only which model appears strongest, but also how confident they can be in that assessment. A model may have a slightly better likelihood value but still carry significant uncertainty, while another model may perform slightly less well but overall provide more reliable predictions.
By making model uncertainty and reproducibility visible, scientists can make more informed decisions when choosing between competing models. This deeper understanding could ultimately lead to better scientific decisions, not only in epilepsy research, but also across many fields that rely on mathematical models to drive discovery.