Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Regression in Extreme Regions (2303.03084v2)

Published 6 Mar 2023 in stat.ML, cs.LG, math.ST, and stat.TH

Abstract: The statistical learning problem consists in building a predictive function $\hat{f}$ based on independent copies of $(X,Y)$ so that $Y$ is approximated by $\hat{f}(X)$ with minimum (squared) error. Motivated by various applications, special attention is paid here to the case of extreme (i.e. very large) observations $X$. Because of their rarity, the contributions of such observations to the (empirical) error is negligible, and the predictive performance of empirical risk minimizers can be consequently very poor in extreme regions. In this paper, we develop a general framework for regression on extremes. Under appropriate regular variation assumptions regarding the pair $(X,Y)$, we show that an asymptotic notion of risk can be tailored to summarize appropriately predictive performance in extreme regions. It is also proved that minimization of an empirical and nonasymptotic version of this 'extreme risk', based on a fraction of the largest observations solely, yields good generalization capacity. In addition, numerical results providing strong empirical evidence of the relevance of the approach proposed are displayed.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Max K-Armed Bandit: On the ExtremeHunter Algorithm and Beyond. Machine Learning and Knowledge Discovery in Databases ECML PKDD 2017, pages 389–404, 2017.
  2. Cross validation for extreme value analysis. arXiv:2202.00488, 2023.
  3. Tail inverse regression: Dimension reduction for prediction of extremes. Bernoulli, 30(1):503–533, 2024.
  4. Permutation importance: a corrected feature importance measure. Bioinformatics, 26(10):1340–1347, 2010.
  5. Statistics of extremes: theory and applications. John Wiley & Sons, 2006.
  6. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013.
  7. L. Breiman. Random forests. Machine learning, 45:5–32, 2001.
  8. Empirical risk minimization for heavy-tailed losses. The Annals of Statistics, 43(6):2507–2536, 2015.
  9. Estimation of extreme risk regions under multivariate regular variation. The Annals of Statistics, 39(3):1803–1826, 2011.
  10. Extreme bandits. In Advances in Neural Information Processing Systems 27, pages 1089–1097. Curran Associates, Inc., 2014.
  11. Extreme-quantile tracking for financial time series. Journal of Econometrics, 181(1):44–52, 2014.
  12. A multivariate extreme value theory approach to anomaly clustering and visualization. Computational Statistics, 35(2):607–628, 2020.
  13. Identifying groups of variables with the potential of being large simultaneously. Extremes, 22:193–222, 2019.
  14. Concentration bounds for the empirical angular measure with statistical learning applications. Bernoulli, 29(4):2797–2827, 2023.
  15. Decompositions of dependence for high-dimensional extremes. Biometrika, 106(3):587–604, 2019.
  16. On kernel smoothing for extremal quantile regression. Bernoulli, 19:2557–2589, 2013.
  17. Optimal weighted pooling for inference about the tail index and extreme quantiles. Bernoulli, 2023.
  18. L. De Haan and A. Ferreira. Extreme value theory: an introduction. Springer Science & Business Media, 2007.
  19. On regular variation of probability densities. Stochastic processes and their applications, 25:83–93, 1987.
  20. Principal component analysis for multivariate extremes. Electronic Journal of Statistics, 15:908–943, 2021.
  21. J. H. J. Einmahl and J. Segers. Maximum empirical likelihood estimation of the spectral measure of an extreme-value distribution. Ann. Stat., 37:2953–2989, 2009.
  22. Nonparametric estimation of the spectral measure of an extreme value distribution. Ann. Stat., 29:1401–1423, 2001.
  23. Estimation of extreme quantiles from heavy and light tailed distributions. Journal of Statistical Planning and Inference, 142(10):2735–2747, 2012.
  24. Graphical models for extremes. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82(4):871–932, 2020.
  25. On consistency of kernel density estimators for randomly censored data: rates holding uniformly over adaptive intervals. Annales de l’IHP Probabilités et statistiques, 37(4):503–522, 2001.
  26. Extremal random forests. arxiv:201.12865, 2022.
  27. Learning the dependence structure of rare events: a non-asymptotic study. Conference on Learning Theory, pages 843–860, 2015.
  28. Sparse representation of multivariate extremes with applications to anomaly ranking. Artificial Intelligence and Statistics, pages 75–83, 2016.
  29. Sparse representation of multivariate extremes with applications to anomaly detection. Journal of Multivariate Analysis, 161:12–31, 2017.
  30. Ulrike Grömping. Variable importance in regression models. Wiley interdisciplinary reviews: Computational statistics, 7(2):137–152, 2015.
  31. A Distribution-Free Theory of Nonparametric Regression. Springer, 2002.
  32. One-component regular variation and graphical modeling of extremes. Journal of Applied Probability, 53(3):733–746, 2016.
  33. Regular variation for measures on metric spaces. Publications de l’Institut Mathématique, 80(94):121–140, 2006.
  34. On binary classification in extreme regions. In Advances in Neural Information Processing Systems, pages 3092–3100, 2018.
  35. k-means clustering of extremes. Electronic Journal of Statistics, 14(1):1211–1233, 2020.
  36. Learning subgaussian classes : Upper and minimax bounds. arXiv:1305.4825, 2013.
  37. Uniform concentration bounds for frequencies of rare events. Statistics & Probability Letters, 189:109610, 2022.
  38. Regularly varying measures on metric spaces: Hidden regular variation and hidden jumps. Probability Surveys, pages 270–314, 2014.
  39. Risk minimization by median-of-means tournaments. Journal of the European Mathematical Society, 22(3):925–965, 2019.
  40. Pascal Massart. Concentration Inequalities and Model Selection. Springer-Verlag, 2007.
  41. Colin McDiarmid. Concentration. Probabilistic methods for algorithmic discrete mathematics, pages 195–248, 1998.
  42. Shahar Mendelson. On aggregation for heavy-tailed classes. Probability Theory and Related Fields, 168(3-4):641–674, 2017.
  43. Sparse regular variation. Advances in Applied Probability, 53(4):1115–1148, 2021.
  44. Multivariate sparse clustering for extremes. Journal of the American Statistical Association, 0(just-accepted):1–23, 2023.
  45. The revival of the gini importance? Bioinformatics, 34(21):3711–3718, 2018.
  46. Rare probability estimation under regularly varying heavy tails. Conference on Learning Theory, pages 21.1–21.24, 2012.
  47. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  48. Second-order regular variation and rates of convergence in extreme-value theory. The Annals of Probability, 24(1):97 – 124, 1996.
  49. Sidney I. Resnick. Extreme values, regular variation and point processes. Springer, 2013.
  50. Determining the dependence structure of multivariate extremes. Biometrika, 107(3):513–532, 2020.
  51. Alec Stephenson. Simulating multivariate extreme value distributions of logistic type. Extremes, 6(1):49–59, 2003.
  52. A. W. van der Vaart and Jon Wellner. Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. Springer-Verlag, New York, 1996.
  53. Gradient boosting for extreme quantile regression. Extremes, 0(0):1–29, 2023.
  54. Extreme value theory for anomaly detection–the gpd classifier. Extremes, 23(4):501–520, 2020.
  55. Clustering bivariate dependencies of compound precipitation and wind extremes over great britain and ireland. Weather and climate extremes, 32:100318, 2021.
  56. Generalizing from a few examples: A survey on few-shot learning. ACM Comput. Surv., 53(3), jun 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Nathan Huet (3 papers)
  2. Anne Sabourin (23 papers)
  3. Stephan Clémençon (32 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets