Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning and the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms.

An important but underrated area is the uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture, and the stochastic training process.

The goal of this paper is to clearly explain and illustrate the importance of the UQ of ML. We elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling are discussed, demonstrated, and compared. We also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian processes, and conformal prediction. Finally, we discuss the need for building a verification, validation, and UQ framework to establish ML credibility.