Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Finding Subgroups with Significant Treatment Effects (2103.07066v2)

Published 12 Mar 2021 in econ.EM, cs.LG, stat.ME, and stat.ML

Abstract: Researchers often run resource-intensive randomized controlled trials (RCTs) to estimate the causal effects of interventions on outcomes of interest. Yet these outcomes are often noisy, and estimated overall effects can be small or imprecise. Nevertheless, we may still be able to produce reliable evidence of the efficacy of an intervention by finding subgroups with significant effects. In this paper, we propose a machine-learning method that is specifically optimized for finding such subgroups in noisy data. Unlike available methods for personalized treatment assignment, our tool is fundamentally designed to take significance testing into account: it produces a subgroup that is chosen to maximize the probability of obtaining a statistically significant positive treatment effect. We provide a computationally efficient implementation using decision trees and demonstrate its gain over selecting subgroups based on positive (estimated) treatment effects. Compared to standard tree-based regression and classification tools, this approach tends to yield higher power in detecting subgroups affected by the treatment.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Armstrong, Timothy B and Shu Shen (2015). Inference on optimal treatment assignments. SSRN Electronic Journal.
  2. Athey, Susan and Guido Imbens (2016). Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113(27):7353–7360.
  3. Generalized random forests. The Annals of Statistics, 47(2):1148–1178.
  4. Athey, Susan and Stefan Wager (2017). Efficient policy learning. arXiv preprint arXiv:1702.02896.
  5. Athey, Susan and Stefan Wager (2021). Policy learning with observational data. Econometrica, 89(1):133–161.
  6. Algorithms for optimizing the ratio of submodular functions. In International Conference on Machine Learning, pages 2751–2759. PMLR.
  7. Identifying Exceptional Responders in Randomized Trials: An Optimization Approach. INFORMS Journal on Optimization, 1(3):187–199.
  8. Breiman, Leo (2001). Random forests. Machine learning, 45:5–32.
  9. Generic machine learning inference on heterogeneous treatment effects in randomized experiments, with an application to immunization in india. Technical report, National Bureau of Economic Research.
  10. Orthogonal machine learning for demand estimation: High dimensional causal inference in dynamic panels. arXiv preprint arXiv:1712.09988.
  11. Plug-in regularized estimation of high-dimensional parameters in nonlinear semiparametric models. arXiv preprint arXiv:1806.04823.
  12. Double/de-biased machine learning using regularized riesz representers. arXiv preprint arXiv:1802.08667.
  13. Adversarial estimation of riesz representers. arXiv preprint arXiv:2101.00009.
  14. Automatic debiased machine learning of causal and structural effects. arXiv preprint arXiv:1809.05224, 8.
  15. Nonparametric Tests for Treatment Effect Heterogeneity. The review of economics and statistics, 90(3):389–405.
  16. Díaz, Iván and Mark J van der Laan (2013). Targeted data adaptive estimation of the causal dose–response curve. Journal of Causal Inference, 1(2):171–192.
  17. Interactive identification of individuals with positive treatment effect while controlling false discoveries. arXiv preprint arXiv:2102.10778.
  18. Doubly robust policy evaluation and learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning, pages 1097–1104. Omnipress.
  19. Hiv prevention among youth: A randomized controlled trial of voluntary counseling and testing for hiv and male condom distribution in rural kenya. PloS one, 14(7):e0219535.
  20. Stable discovery of interpretable subgroups via calibration in causal studies. International Statistical Review, 88(S1):S135–S178.
  21. Foster, Dylan J and Vasilis Syrgkanis (2019). Orthogonal statistical learning. arXiv preprint arXiv:1901.09036.
  22. Local linear forests. arXiv preprint arXiv:1807.11408.
  23. Kallus, Nathan (2021). More efficient policy learning via optimal retargeting. Journal of the American Statistical Association, 116(534):646–658.
  24. Kallus, Nathan and Angela Zhou (2018). Policy evaluation and optimization with continuous treatments. In International Conference on Artificial Intelligence and Statistics, pages 1243–1251.
  25. Robust causal inference with continuous instruments using the local instrumental variable curve. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 81(1):121–143.
  26. Non-parametric methods for doubly robust estimation of continuous treatment effects. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(4):1229–1245.
  27. Kitagawa, Toru and Aleksey Tetenov (2018). Who should be treated? empirical welfare maximization methods for treatment choice. Econometrica, 86(2):591–616.
  28. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the National Academy of Sciences, 116(10):4156–4165.
  29. Learning and Testing Sub-groups with Heterogeneous Treatment Effects:A Sequence of Two Studies.
  30. Decision trees as partitioning machines to characterize their generalization properties. In Larochelle, H., M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 18135–18145. Curran Associates, Inc.
  31. Star: A general interactive framework for fdr control under structural constraints. arXiv preprint arXiv:1710.02776.
  32. Leqi, Liu and Edward H Kennedy (2021). Median optimal treatment regimes. arXiv preprint arXiv:2103.01802.
  33. Mansour, Yishay (1997). Pessimistic decision tree pruning based on tree size. In Proceedings of the Fourteenth International Conference on Machine Learning, pages 195–201. Morgan Kaufmann.
  34. Maurer, Andreas and Massimiliano Pontil (2009). Empirical Bernstein bounds and sample variance penalization. In The 22nd Conference on Learning Theory (COLT).
  35. Efficient Discovery of Heterogeneous Treatment Effects in Randomized Experiments via Anomalous Pattern Detection.
  36. Nie, Xinkun and Stefan Wager (2017). Quasi-oracle estimation of heterogeneous treatment effects. arXiv preprint arXiv:1712.04912.
  37. Opper, Isaac M (2021). Improving average treatment effect estimates in small-scale randomized controlled trials.
  38. Orthogonal random forest for causal inference. In International Conference on Machine Learning, pages 4932–4941.
  39. Some methods for heterogeneous treatment effect estimation in high dimensions. Statistics in medicine, 37(11):1767–1787.
  40. A unified treatment of multiple testing with prior knowledge using the p-filter. Annals of Statistics, 47(5):2790–2821.
  41. Rubin, Daniel and Mark J van der Laan (2005). A general imputation methodology for nonparametric regression with censored data.
  42. Rubin, Daniel and Mark J van der Laan (2007). A doubly robust censoring unbiased transformation. The international journal of biostatistics, 3(1).
  43. Schrijver, Alexander (2000). A combinatorial algorithm minimizing submodular functions in strongly polynomial time. Journal of Combinatorial Theory, Series B, 80(2):346–355.
  44. Shalev-Shwartz, Shai and Shai Ben-David (2014). Understanding machine learning: From theory to algorithms. Cambridge university press.
  45. Spiess, Jann (2018). Optimal Estimation when Researcher and Social Preferences are Misaligned.
  46. Stobbe, Peter and Andreas Krause (2010). Efficient minimization of decomposable submodular functions. In Lafferty, J., C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc.
  47. Swaminathan, Adith and Thorsten Joachims (2015). Counterfactual risk minimization: Learning from logged bandit feedback. In International Conference on Machine Learning, pages 814–823.
  48. Syrgkanis, Vasilis and Manolis Zampetakis (2020). Estimation and inference with trees and forests in high dimensions. In Conference on Learning Theory, pages 3453–3454. PMLR.
  49. Talisa, Victor B and Chung-Chou H Chang (2021). Learning and confirming a class of treatment responders in clinical trials. Statistics in medicine.
  50. van der Laan, Mark J and Alexander R Luedtke (2014). Targeted learning of an optimal dynamic treatment, and statistical inference for its mean outcome.
  51. Wu, Edward and Johann A Gagnon-Bartsch (2018). The LOOP Estimator: Adjusting for Covariates in Randomized Experiments. Evaluation Review, 42(4):458–488.
  52. Offline multi-action policy learning: Generalization and optimization. arXiv preprint arXiv:arXiv:1810.04778.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Jann Spiess (17 papers)
  2. Vasilis Syrgkanis (106 papers)
  3. Victor Yaneng Wang (1 paper)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now