Neural Likelihood Surfaces for Spatial Processes with Computationally Intensive or Intractable Likelihoods (2305.04634v3)
Abstract: In spatial statistics, fast and accurate parameter estimation, coupled with a reliable means of uncertainty quantification, can be challenging when fitting a spatial process to real-world data because the likelihood function might be slow to evaluate or wholly intractable. In this work, we propose using convolutional neural networks to learn the likelihood function of a spatial process. Through a specifically designed classification task, our neural network implicitly learns the likelihood function, even in situations where the exact likelihood is not explicitly available. Once trained on the classification task, our neural network is calibrated using Platt scaling which improves the accuracy of the neural likelihood surfaces. To demonstrate our approach, we compare neural likelihood surfaces and the resulting maximum likelihood estimates and approximate confidence regions with the equivalent for exact or approximate likelihood for two different spatial processes: a Gaussian process and a Brown-Resnick process which have computationally intensive and intractable likelihoods, respectively. We conclude that our method provides fast and accurate parameter estimation with a reliable method of uncertainty quantification in situations where standard methods are either undesirably slow or inaccurate. The method is applicable to any spatial process on a grid from which fast simulations are available.
- Simulation-Based Inference Methods for Particle Physics. In Artificial Intelligence for High Energy Physics, Chapter 16, pp. 579–611. World Scientific.
- Mining Gold from Implicit Models to Improve Likelihood-Free Inference. Proceedings of the National Academy of Sciences 117, 5242 – 5249.
- Carnell, R. (2022). lhs: Latin Hypercube Samples. R package version 1.1.5.
- Statistical Inference. Duxbury Resource Center.
- High-Order Composite Likelihood Inference for Max-Stable Distributions and Processes. Journal of Computational and Graphical Statistics 25(4), 1212–1229.
- Inference for clustered data using the independence loglikelihood. Biometrika 94(1), 167–183.
- Chollet, F. et al. (2015). Keras. https://keras.io.
- The Frontier of Simulation-Based Inference. Proceedings of the National Academy of Sciences 117(48), 30055–30062.
- Confidence sets and hypothesis testing in a likelihood-free inference setting. In H. D. III and A. Singh (Eds.), Proceedings of the 37th International Conference on Machine Learning, Volume 119 of Proceedings of Machine Learning Research, pp. 2323–2334. PMLR.
- Likelihood-free frequentist inference: Bridging classical statistics and machine learning for reliable simulator-based inference. arXiv:2107.03920.
- Davison, A. C. (2003). Likelihood, pp. 94–160. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.
- Statistical Modeling of Spatial Extremes. Statistical science 27(2), 161–186.
- Asymptotic properties of the maximum likelihood estimator for multivariate extreme value distributions.
- A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the Gap? arXiv:2112.00639.
- Handbook of Spatial Statistics. Chapman & Hall/CRC Handbooks of Modern Statistical Methods. Taylor & Francis.
- Fast Covariance Parameter Estimation of Spatial Gaussian Process Models using Neural Networks. Stat 10(1), e382.
- Deep Learning. Cambridge, MA, USA: MIT Press. http://www.deeplearningbook.org.
- On Calibration of Modern Neural Networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, pp. 1321–1330. JMLR.org.
- A Case Study Competition Among Methods for Analyzing Large Spatial Data. Journal of Agricultural, Biological, and Environmental Statistics 24, 398 – 425.
- Composite Likelihood Estimation for the Brown–Resnick Process. Biometrika 100(2), 511–518.
- Stationary Max-Stable Fields Associated to Negative Definite Functions. The Annals of Probability 37(5), 2042 – 2065.
- A General Framework for Vecchia Approximations of Gaussian Processes. Statistical Science 36(1), 124 – 141.
- Keener, R. (2010). Theoretical Statistics: Topics for a Core Course. Springer Texts in Statistics. Springer New York.
- Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015).
- Kuusela, M. and M. L. Stein (2018). Locally Stationary Spatio-Temporal Interpolation of Argo Profiling Float Data. Proceedings of the Royal Society A 474(2220), 20180400.
- Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE 86(11), 2278–2324.
- Neural Networks for Parameter Estimation in Intractable Models. Computational Statistics &\And& Data Analysis 185, 107762.
- Towards black-box parameter estimation.
- An Explicit Link Between Gaussian Fields and Gaussian Markov Random Fields: The Stochastic Partial Differential Equation Approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73(4), 423–498.
- Mardia, K. V. and R. J. Marshall (1984). Maximum likelihood estimation of models for residual covariance in spatial regression. Biometrika 71(1), 135–146.
- Murphy, K. P. (2022). Probabilistic Machine Learning: An Introduction. MIT Press.
- Likelihood-based Inference for Max-Stable Processes. Journal of the American Statistical Association 105(489), 263–277.
- Platt, J. (1999). Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. Advances in Large Margin Classifiers 10(3), 61–74.
- Rasmussen, C. E. and C. K. I. Williams (2006). Gaussian Processes for Machine Learning. Adaptive computation and machine learning. MIT Press.
- Ribatet, M. (2009). A User’s Guide to the SpatialExtremes Package.
- Ribatet, M. (2020). SpatialExtremes: Modelling Spatial Extremes. R package version 2.0-9.
- Likelihood-free neural bayes estimators for censored inference with peaks-over-threshold models.
- Neural bayes estimators for irregular spatial data using graph neural networks.
- Neural Point Estimation for Fast Optimal Likelihood-Free Inference. arXiv:2208.12942.
- Shao, J. (2003). Mathematical Statistics (2nd ed.). Springer.
- Handbook of Approximate Bayesian Computation. Taylor and Francis.
- Stoev, S. A. (2008). On the ergodicity and mixing of max-stable processes. Stochastic Processes and their Applications 118(9), 1679–1705.
- Geostatistics for Large Datasets. In Advances and Challenges in Space-time Modelling of Natural Events, Chapter 3, pp. 55–77. Berlin, Heidelberg: Springer.
- Vaart, A. W. v. d. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.
- Argo Data 1999–2019: Two Million Temperature-Salinity Profiles and Subsurface Velocity Observations From a Global Array of Profiling Floats. Frontiers in Marine Science 7(700).