Today, we tap into the brilliant minds at forefront of integrating neural ordinary differential equations (neural ODEs) into AI predictions for buildings.
Read on to discover what BrainBox AI’s Mathieu Le Cam and Jean-Simon Venne, and University of Montreal’s assistant professor of computer science and operations research, Dr. Pierre-Luc Bacon, have to say about what neural ODEs are and how they’re making our AI for HVAC models even faster and more accurate.
To kick us off, Jean-Simon, could you briefly describe what Neural ODEs are?
Jean-Simon: Of course. Neural ODEs are a type of machine learning model that combines the power of neural networks with the precision of ordinary differential equations (ODEs) to predict changes over time. This is an innovative approach as it allows for more accurate and efficient predictions with less data and computational power.
So, in terms of HVAC systems, Neural ODEs can be used for continual adjustment. Instead of turning the HVAC system on and off at fixed times, think of it as a system that continuously and smoothly adjusts settings based on the current climate inside the building. Normally you would need a lot of data to do achieve the same outcome with deep learning models alone. With neural ODEs, you need just a fraction of that data to get the same predictive accuracy.
The fact that neural ODEs require less data for the same level of accuracy - is that what makes it so transformative for AI predictions?
Mathieu: Yes. So, the advantage here is massive. What we've seen so far with deep learning methods only is that they need quite a lot of data to be accumulated before you can train your model and have a forecast that's accurate enough for predictive control strategies. But with neural ODEs, where we used to need weeks of data to train our models, we can now reach the same accuracy with just a few days of data. This is because neural ODEs allow us to incorporate prior knowledge about the thermodynamics of a building into the way we shape the equations and models, so we only need to learn a few parameters that have physical interpretations. Basically, With neural ODEs, we've reduced the amount of data required to train our models by 10 times.
"With neural ODEs, we've reduced the amount of data required to train our models by 10 times." - Mathieu Le Cam
It’s also what enables us to train our models on central processing units (CPUs) as opposed to graphics processing units (GPUs). This requires less resources to train and reduces our overall energy footprint.
How does this technology impact real-world applications and reliability?
Pierre-Luc: I think an important aspect of real-world applications is the ability to interpret and verify our systems, especially if you're deploying these systems in buildings. Clients need to trust their (HVAC) systems, so you need to be able to verify and assure customers that the product is going to do the right thing.
What challenges have you faced during this project?
Pierre Luc: One major challenge is retraining with Neural ODEs to address data distribution drifts. When the data distribution changes or drifts, it means the problem changes slightly, and not all methods handle this well. This is a significant challenge with black box deep learning methods. That's why hybrid models like ours, which combine black box and physics-informed approaches, are beneficial. They are nimbler and can adapt quickly without needing to change a vast number of parameters all at once, which is very difficult to manage.
Jean-Simon: Also, what many often don't realize is that, after the first experiment, you'll need to have retraining features and safety measures in place to move into production and implementation.
Mathieu: At the same time, that’s been a good teaching opportunity for me. When you're in research and you’re working in your little bubble, you always dream that this theoretical baby you’re giving birth to is going to be used in the real world. It's a nice accomplishment, I think, to see that this is going to go to production.
Neural ODEs seem like something that may one day be integrated more widely in the AI for HVAC world. Are there any steps you’re taking to patent and implement this technology more extensively in your operations?
Jean-Simon: We're actually patent pending right now and are following the process to obtain the full patent. In parallel, we're putting Neural ODE into production. We have these existing neural network envelopes in place which are training the model and retraining it when the prediction accuracy depreciates. It's an automatic system that runs every night. We're going to begin to introduce this into our machine learning operations (MLOps) system - so that's what’s happening next.
Want to stay updated with the latest in cutting edge tech for buildings?