Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Data-Driven Preference Sampling for Pareto Front Learning (2404.08397v1)

Published 12 Apr 2024 in cs.LG

Abstract: Pareto front learning is a technique that introduces preference vectors in a neural network to approximate the Pareto front. Previous Pareto front learning methods have demonstrated high performance in approximating simple Pareto fronts. These methods often sample preference vectors from a fixed Dirichlet distribution. However, no fixed sampling distribution can be adapted to diverse Pareto fronts. Efficiently sampling preference vectors and accurately estimating the Pareto front is a challenge. To address this challenge, we propose a data-driven preference vector sampling framework for Pareto front learning. We utilize the posterior information of the objective functions to adjust the parameters of the sampling distribution flexibly. In this manner, the proposed method can sample preference vectors from the location of the Pareto front with a high probability. Moreover, we design the distribution of the preference vector as a mixture of Dirichlet distributions to improve the performance of the model in disconnected Pareto fronts. Extensive experiments validate the superiority of the proposed method compared with state-of-the-art algorithms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. S. Zhou, B. Xu, L. Lu, W. Jin, and Z. Mao, “Multi-objective optimization of rear guide vane of diagonal flow fan based on robustness design theory,” Applied Sciences, vol. 12, no. 19, p. 9858, 2022.
  2. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: NSGA-II,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182–197, 2002.
  3. Q. Zhang and H. Li, “MOEA/D: A multiobjective evolutionary algorithm based on decomposition,” IEEE Transactions on Evolutionary Computation, vol. 11, no. 6, pp. 712–731, 2007.
  4. R. Eberhart and J. Kennedy, “Particle swarm optimization,” in Proceedings of the IEEE International Conference on Neural Networks, vol. 4.   Citeseer, 1995, pp. 1942–1948.
  5. J.-A. Désidéri, “Multiple-gradient descent algorithm (MGDA) for multiobjective optimization,” Comptes Rendus Mathematique, vol. 350, no. 5-6, pp. 313–318, 2012.
  6. A. Dosovitskiy and J. Djolonga, “You only train once: Loss-conditional training of deep networks,” in International Conference on Learning Representations, 2020.
  7. X. Lin, Z. Yang, Q. Zhang, and S. Kwong, “Controllable Pareto multi-task learning,” arXiv preprint arXiv:2010.06313, 2020.
  8. O. Sener and V. Koltun, “Multi-task learning as multi-objective optimization,” Advances in Neural Information Processing Systems, vol. 31, 2018.
  9. C. Rich, “Multitask learning,” Machine learning, vol. 28, no. 1, pp. 41–75, 1997.
  10. A. Navon, A. Shamsian, G. Chechik, and E. Fetaya, “Learning the Pareto front with hypernetworks,” arXiv preprint arXiv:2010.04104, 2020.
  11. W. Chen and J. Kwok, “Multi-objective deep learning with adaptive reference vectors,” Advances in Neural Information Processing Systems, vol. 35, pp. 32 723–32 735, 2022.
  12. M. Ruchte and J. Grabocka, “Scalable Pareto front approximation for deep multi-objective learning,” in 2021 IEEE International Conference on Data Mining (ICDM).   IEEE, 2021, pp. 1306–1311.
  13. X. Liu, X. Tong, and Q. Liu, “Profiling Pareto front with multi-objective stein variational gradient descent,” Advances in Neural Information Processing Systems, vol. 34, pp. 14 721–14 733, 2021.
  14. P. Ma, T. Du, and W. Matusik, “Efficient continuous Pareto exploration in multi-task learning,” in International Conference on Machine Learning.   PMLR, 2020, pp. 6522–6531.
  15. L. P. Hoang, D. D. Le, T. A. Tuan, and T. N. Thang, “Improving pareto front learning via multi-sample hypernetworks,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 7, 2023, pp. 7875–7883.
  16. S. Chib and E. Greenberg, “Understanding the metropolis-hastings algorithm,” The American Statistician, vol. 49, no. 4, pp. 327–335, 1995.
  17. E. Zitzler, K. Deb, and L. Thiele, “Comparison of multiobjective evolutionary algorithms: Empirical results,” Evolutionary Computation, vol. 8, no. 2, pp. 173–195, 2000.
  18. X. Lin, H.-L. Zhen, Z. Li, Q.-F. Zhang, and S. Kwong, “Pareto multi-task learning,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  19. K. Deb, L. Thiele, M. Laumanns, and E. Zitzler, “Scalable test problems for evolutionary multiobjective optimization,” in Evolutionary Multiobjective Optimization: Theoretical Advances and Applications.   Springer, 2005, pp. 105–145.
  20. H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.
  21. C. A. Coello Coello and M. Reyes Sierra, “A study of the parallelization of a coevolutionary multi-objective evolutionary algorithm,” in MICAI 2004: Advances in Artificial Intelligence: Third Mexican International Conference on Artificial Intelligence.   Springer, 2004, pp. 688–697.
  22. H. Mausser, “Normalization and other topics in multi-objective optimization,” in Fields-MITACS Industrial Problems Workshop.   Citeseer, 2006, p. 89.
  23. H. Li, J. Sun, Q. Zhang, and Y. Shui, “Adjustment of weight vectors of penalty-based boundary intersection method in moea/d,” in Evolutionary Multi-Criterion Optimization.   Springer, 2019, pp. 91–100.
  24. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  25. S. Ruder, “An overview of multi-task learning in deep neural networks,” arXiv preprint arXiv:1706.05098, 2017.
  26. Y. Zhang and Q. Yang, “A survey on multi-task learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 12, pp. 5586–5609, 2021.
  27. T. Standley, A. Zamir, D. Chen, L. Guibas, J. Malik, and S. Savarese, “Which tasks should be learned together in multi-task learning?” in International Conference on Machine Learning.   PMLR, 2020, pp. 9120–9132.
  28. Z. Chen, V. Badrinarayanan, C.-Y. Lee, and A. Rabinovich, “Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks,” in International Conference on Machine Learning.   PMLR, 2018, pp. 794–803.
  29. A. Kendall, Y. Gal, and R. Cipolla, “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7482–7491.
  30. S. Liu, E. Johns, and A. J. Davison, “End-to-end multi-task learning with attention,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1871–1880.
  31. A. Navon, I. Achituve, H. Maron, G. Chechik, and E. Fetaya, “Auxiliary learning by implicit differentiation,” arXiv preprint arXiv:2007.02693, 2020.
  32. K. Van Moffaert and A. Nowé, “Multi-objective reinforcement learning using sets of Pareto dominating policies,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 3483–3512, 2014.
  33. M. Pirotta and M. Restelli, “Inverse reinforcement learning through policy gradient minimization,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, 2016.
  34. S. Parisi, M. Pirotta, and M. Restelli, “Multi-objective reinforcement learning through continuous Pareto manifold approximation,” Journal of Artificial Intelligence Research, vol. 57, pp. 187–227, 2016.
  35. S. Parisi, M. Pirotta, N. Smacchia, L. Bascetta, and M. Restelli, “Policy gradient approaches for multi-objective sequential decision making,” in 2014 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2014, pp. 2323–2330.
  36. D. Mahapatra and V. Rajan, “Multi-task learning with user preferences: Gradient descent with controlled ascent in Pareto optimization,” in International Conference on Machine Learning.   PMLR, 2020, pp. 6597–6607.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets