Neural Bayes Estimators for Irregular Spatial Data using Graph Neural Networks (2310.02600v3)
Abstract: Neural Bayes estimators are neural networks that approximate Bayes estimators in a fast and likelihood-free manner. Although they are appealing to use with spatial models, where estimation is often a computational bottleneck, neural Bayes estimators in spatial applications have, to date, been restricted to data collected over a regular grid. These estimators are also currently dependent on a prescribed set of spatial locations, which means that the neural network needs to be re-trained for new data sets; this renders them impractical in many applications and impedes their widespread adoption. In this work, we employ graph neural networks to tackle the important problem of parameter point estimation from data collected over arbitrary spatial locations. In addition to extending neural Bayes estimation to irregular spatial data, our architecture leads to substantial computational benefits, since the estimator can be used with any configuration or number of locations and independent replicates, thus amortising the cost of training for a given spatial model. We also facilitate fast uncertainty quantification by training an accompanying neural Bayes estimator that approximates a set of marginal posterior quantiles. We illustrate our methodology on Gaussian and max-stable processes. Finally, we showcase our methodology on a data set of global sea-surface temperature, where we estimate the parameters of a Gaussian process model in 2161 spatial regions, each containing thousands of irregularly-spaced data points, in just a few minutes with a single graphics processing unit.
- A valid MatĆ©rn class of cross-covariance functions for multivariate random fields with any number of components. Journal of the American Statistical Association, 497:180ā193.
- Spatial Point Patterns: Methodology and Applications with R. Chapman & Hall/CRC, Boca Raton, FL.
- Fast Gaussian process estimation for large-scale in situ inference using convolutional neural networks. In IEEE International Conference on Big Data, pages 3731ā3739.
- Estimating space and space-time covariance functions for large data sets: A weighted composite likelihood approach. Journal of the American Statistical Association, 107:268ā280.
- Extreme values of independent stochastic processes. Journal of Applied Probability, 14:732ā739.
- Suomi NPP VIIRS sensor data record verification, validation, and long-term performance monitoring. Journal of Geophysical Research: Atmospheres, 118:11ā664.
- Local likelihood estimation of complex tail dependence structures, applied to U.S. precipitation extremes. Journal of the American Statistical Association, 115:1037ā1054.
- High-order composite likelihood inference for max-stable distributions and processes. Journal of Computational and Graphical Statistics, 25:1212ā1229.
- Neural approximate sufficient statistics for implicit models. In Proceedings of the 9th International Conference on Learning Representations, ICLR.
- Deep graphical regression for jointly moderate and extreme Australian wildfires. arXiv:2308.14547v1.
- Creel, M. (2017). Neural nets for indirect inference. Econometrics and Statistics, 2:36ā49.
- Cressie, N. (2023). Decisions, decisions, decisions in an uncertain environment. Environmetrics, 34:e2767.
- Basis-function models in spatial statistics. Annual Review of Statistics and its Applications, 9:373ā400.
- Statistics of extremes. Annual Review of Statistics and its Application, 2:203ā235.
- Spatial extremes. In Gelfand, A.Ā E., Fuentes, M., Hoeting, J.Ā A., and Smith, R.Ā L., editors, Handbook of Environmental and Ecological Statistics, pages 711ā744. Chapman & Hall/CRC Press, Boca Raton, FL.
- Diggle, P. (2013). Statistical Analysis of Spatial and Spatio-Temporal Point Patterns. Chapman & Hall/CRC, New York, NY, 3rd edition.
- The unreasonable effectiveness of convolutional neural networks in population genetic inference. Molecular Biology and Evolution, 36:220ā238.
- Cross-covariance functions for multivariate geostatistics. Statistical Science, 30:147ā163.
- Multivariate max-stable spatial processes. Biometrika, 102:215ā230.
- Fast covariance parameter estimation of spatial Gaussian process models using neural networks. Stat, 10:e382.
- Neural message passing for quantum chemistry. In Precup, D. and Teh, Y.Ā W., editors, Proceedings of the 34th International Conference on Machine Learning, volumeĀ 70 of Proceedings of Machine Learning Research, pages 1263ā1272. PMLR.
- MatĆ©rn cross-covariance functions for multivariate random fields. Journal of the American Statistical Association, 105:1167ā1177.
- Deep Learning. MIT Press, Cambridge, MA.
- Understanding pooling in graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, pages 1ā11.
- Guinness, J. (2018). Permutation and grouping methods for sharpening Gaussian process approximations. Technometrics, 60:415ā429.
- Haas, T.Ā C. (1990a). Kriging and automated variogram modeling within a moving window. Atmospheric Environment, 24:1759ā1769.
- Haas, T.Ā C. (1990b). Lognormal and moving window methods of estimating acid deposition. Journal of the American Statistical Association, 85:950ā963.
- Universal approximation of symmetric and anti-symmetric functions. Communications in Mathematical Sciences, 20:1397ā1408.
- Multilayer feedforward networks are universal approximators. Neural Networks, 2:359ā366.
- Huser, R. (2013). Statistical Modeling and Inference for Spatio-Temporal Extremes. PhD thesis, Swiss Federal Institute of Technology, Lausanne, Switzerland.
- Full likelihood inference for max-stable data. Stat, 8:e218.
- Advances in statistical modeling of spatial extremes. Wiley Interdisciplinary Reviews: Computational Statistics, 14:e1537.
- Statistical Analysis and Modelling of Spatial Point Patterns. Wiley, New York, NY.
- Learning summary statistic for approximate Bayesian computation via deep neural network. Statistica Sinica, 27:1595ā1618.
- Theory of Point Estimation. Springer, New York, NY, 2nd edition.
- Neural networks for parameter estimation in intractable models. Computational Statistics & Data Analysis, 185:107762.
- Lucibello, C. (2021). GraphNeuralNetworks.jl: a geometric deep learning library for the Julia programming language.
- Exponential Random Graph Models for Social Networks: Theory, Methods, and Applications. Cambridge University Press, Cambridge, England.
- Rethinking pooling in graph neural networks. In Advances in Neural Information Processing Systems.
- Statistical Inference and Simulation for Spatial Point Processes. Chapman & Hall/CRC, Boca Raton, FL.
- Universal readout for graph convolutional neural networks. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1ā7.
- Likelihood-free inference with generative neural networks via scoring rule minimization. arXiv:2205.15784.
- Likelihood-based inference for max-stable processes. Journal of the American Statistical Association, 105:263ā277.
- BayesFlow: Learning complex stochastic models with invertible neural networks. IEEE Transactions on Neural Networks and Learning Systems, 33:1452ā1466.
- Fast parameter estimation of generalized extreme value distribution using neural networks. arXiv:2305.04341v1.
- Neural Bayes estimators for censored inference with peaks-over-threshold models. arXiv:2306.15642.
- Robert, C.Ā P. (2007). The Bayesian Choice. Springer, New York, NY, 2nd edition.
- An introduction to exponential random graph (p*)superscriptš(p^{*})( italic_p start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ) models for social networks. Social Networks, 29:173ā191.
- Parameter estimation with dense and convolutional neural networks applied to the FitzHugh-Nagumo ODE. In Bruna, J., Hesthaven, J., and Zdeborova, L., editors, Proceedings of the 2nd Annual Conference on Mathematical and Scientific Machine Learning, volume 145 of Proceedings of Machine Learning Research, pages 1ā28. PMLR.
- Likelihood-free parameter estimation with neural Bayes estimators. The American Statistician, to appear, arXiv:2208.12942.
- Tapered composite likelihood for spatial max-stable models. Spatial Statistics, 8:86ā103.
- Schlather, M. (2002). Models for stationary max-stable random fields. Extremes, 5:33ā44.
- Forecasting west nile virus with graph neural networks: Harnessing spatial dependence in irregularly sampled geospatial data. arXiv:2212.11367v1.
- Universal approximation of functions on sets. Journal of Machine Learning Research, 23:1ā56.
- Neural likelihood surfaces for spatial processes with computationally intensive or intractable likelihoods. arXiv:2305.04634.
- A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32:4ā24.
- Deep sets. In Guyon, I., Luxburg, U.Ā V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volumeĀ 30. Curran Associates, Inc.
- Multi-scale process modelling and distributed computation for spatial data. Statistics and Computing, 30:1609ā1627.
- Deep integro-difference equation models for spatio-temporal forecasting. Spatial Statistics, 37:100408.
- Neural networks for geospatial data. arXiv:2304.09157v1.
- Graph convolutional networks: A comprehensive review. Computational Social Networks, 6:1ā23.
- Zhou, D. (2018). Universality of deep convolutional neural networks. Applied and Computational Harmonic Analysis, 48:787ā794.
- Graph neural networks: A review of methods and applications. AI Open, 1:57ā81.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.