Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
11 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Ant Colony Sampling with GFlowNets for Combinatorial Optimization (2403.07041v4)

Published 11 Mar 2024 in cs.LG and cs.NE

Abstract: We present the Generative Flow Ant Colony Sampler (GFACS), a novel meta-heuristic method that hierarchically combines amortized inference and parallel stochastic search. Our method first leverages Generative Flow Networks (GFlowNets) to amortize a \emph{multi-modal} prior distribution over combinatorial solution space that encompasses both high-reward and diversified solutions. This prior is iteratively updated via parallel stochastic search in the spirit of Ant Colony Optimization (ACO), leading to the posterior distribution that generates near-optimal solutions. Extensive experiments across seven combinatorial optimization problems demonstrate GFACS's promising performances.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. Learning what to defer for maximum independent sets. In International Conference on Machine Learning, pp.  134–144. PMLR, 2020.
  2. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940, 2016.
  3. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381–27394, 2021.
  4. Gflownet foundations. Journal of Machine Learning Research, 24(210):1–55, 2023.
  5. Select and optimize: Learning to aolve large-scale tsp instances. In International Conference on Artificial Intelligence and Statistics, pp.  1219–1231. PMLR, 2023a.
  6. Select and optimize: Learning to aolve large-scale tsp instances. In Ruiz, F., Dy, J., and van de Meent, J.-W. (eds.), Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp.  1219–1231. PMLR, 25–27 Apr 2023b. URL https://proceedings.mlr.press/v206/cheng23a.html.
  7. Simulation-guided beam search for neural combinatorial optimization. Advances in Neural Information Processing Systems, 35:8760–8772, 2022.
  8. Croes, A. A method for solving traveling salesman problems. Operations Research, 5:791––812, 1958.
  9. Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning. In Asian conference on machine learning, pp.  465–480. PMLR, 2020.
  10. Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 26(1):29–41, 1996. doi: 10.1109/3477.484436.
  11. Ant colony optimization. IEEE Computational Intelligence Magazine, 1(4):28–39, 2006. doi: 10.1109/MCI.2006.329691.
  12. Bq-nco: Bisimulation quotienting for generalizable neural combinatorial optimization. arXiv preprint arXiv:2301.03313, 2023.
  13. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3–11, 2018.
  14. Solving the orienteering problem through branch-and-cut. INFORMS Journal on Computing, 10(2):133–148, 1998.
  15. Generalize a small pre-trained model to arbitrarily large tsp instances. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp.  7474–7482, 2021.
  16. Population-based reinforcement learning for combinatorial optimization. arXiv preprint arXiv:2210.03475, 2022.
  17. Helsgaun, K. An extension of the lin-kernighan-helsgaun tsp solver for constrained traveling salesman and vehicle routing problems. 12 2017. doi: 10.13140/RG.2.2.25569.40807.
  18. Neural large neighborhood search for the capacitated vehicle routing problem. CoRR, abs/1911.09539, 2019. URL http://arxiv.org/abs/1911.09539.
  19. Efficient active search for combinatorial optimization problems. In International Conference on Learning Representations, 2021.
  20. Generalize learned heuristics to solve large-scale vehicle routing problems in real-time. In The Eleventh International Conference on Learning Representations, 2022.
  21. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp.  448–456. pmlr, 2015.
  22. Learning energy decompositions for partial inference of GFlowNets. International Conference on Learning Representations (ICLR), 2024.
  23. Pointerformer: Deep reinforced multi-pointer transformer for the traveling salesman problem. arXiv preprint arXiv:2304.09407, 2023.
  24. Learning tsp requires rethinking generalization. In 27th International Conference on Principles and Practice of Constraint Programming (CP 2021). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2021.
  25. Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems, 30, 2017.
  26. Enhancing sample efficiency in black-box combinatorial optimization via symmetric replay training, 2023a.
  27. Devformer: A symmetric transformer for context-aware device placement. In International Conference on Machine Learning, pp.  16541–16566. PMLR, 2023b.
  28. Reinforcement learning-based auto-router considering signal integrity. In 2020 IEEE 29th Conference on Electrical Performance of Electronic Packaging and Systems (EPEPS), pp.  1–3, 2020. doi: 10.1109/EPEPS48591.2020.9231473.
  29. Imitation learning for simultaneous escape routing. In 2021 IEEE 30th Conference on Electrical Performance of Electronic Packaging and Systems (EPEPS), pp.  1–3, 2021a. doi: 10.1109/EPEPS51341.2021.9609145.
  30. Learning collaborative policies to solve np-hard routing problems. Advances in Neural Information Processing Systems, 34:10418–10430, 2021b.
  31. Neuro cross exchange: Learning to cross exchange to solve realistic vehicle routing problems. arXiv preprint arXiv:2206.02771, 2022a.
  32. Sym-nco: Leveraging symmetricity for neural combinatorial optimization. Advances in Neural Information Processing Systems, 35:1936–1949, 2022b.
  33. Learning to scale logits for temperature-conditional GFlowNets. arXiv preprint arXiv:2310.02823, 2023c.
  34. Local search gflownets. arXiv preprint arXiv:2310.02710, 2023d.
  35. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  36. Attention, learn to solve routing problems! In International Conference on Learning Representations, 2018.
  37. Deep policy dynamic programming for vehicle routing problems. In International conference on integration of constraint programming, artificial intelligence, and operations research, pp.  190–213. Springer, 2022.
  38. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems, 33:21188–21198, 2020.
  39. An effective heuristic algorithm for the traveling-salesman problem. Operations research, 21(2):498–516, 1973.
  40. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.
  41. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  42. Neural combinatorial optimization with heavy decoder: Toward large scale generalization. arXiv preprint arXiv:2310.07985, 2023.
  43. Learning to iteratively solve routing problems with dual-aspect collaborative transformer. Advances in Neural Information Processing Systems, 34:11096–11107, 2021.
  44. Efficient neural neighborhood search for pickup and delivery problems. arXiv preprint arXiv:2204.11399, 2022.
  45. Learning to search feasible and infeasible regions of routing problems with flexible neural k-opt. arXiv preprint arXiv:2310.18264, 2023.
  46. Learning GFlowNets from partial episodes for improved convergence and stability. International Conference on Machine Learning (ICML), 2022.
  47. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955–5967, 2022.
  48. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
  49. Reinforcement learning for solving the vehicle routing problem. Advances in neural information processing systems, 31, 2018.
  50. Generative augmented flow networks. In The Eleventh International Conference on Learning Representations, 2022.
  51. Better training of gflownets with local credit and incomplete trajectories. arXiv preprint arXiv:2302.01687, 2023a.
  52. H-tsp: Hierarchically solving the large-scale travelling salesman problem. arXiv preprint arXiv:2304.09395, 2023b.
  53. Transformer network-based reinforcement learning method for power distribution network (pdn) optimization of high bandwidth memory (hbm). IEEE Transactions on Microwave Theory and Techniques, 70(11):4772–4786, 2022. doi: 10.1109/TMTT.2022.3202221.
  54. Dimes: A differentiable meta solver for combinatorial optimization problems. Advances in Neural Information Processing Systems, 35:25531–25546, 2022.
  55. Thompson sampling for improved exploration in GFlowNets. arXiv preprint arXiv:2306.17693, 2023.
  56. Solving np-hard min-max routing problems as sequential generation with equity context. arXiv preprint arXiv:2306.02689, 2023a.
  57. Meta-sage: scale meta-learning scheduled adaptation with guided exploration for mitigating scale shift on combinatorial optimization. In International Conference on Machine Learning, pp.  32194–32210. PMLR, 2023b.
  58. Max–min ant system. Future Generation Computer Systems, 16(8):889–914, 2000. ISSN 0167-739X. doi: https://doi.org/10.1016/S0167-739X(00)00043-1. URL https://www.sciencedirect.com/science/article/pii/S0167739X00000431.
  59. Revisiting sampling for combinatorial optimization. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp.  32859–32874. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/sun23c.html.
  60. Difusco: Graph-based diffusion solvers for combinatorial optimization. arXiv preprint arXiv:2302.08224, 2023.
  61. Graph attention networks. In International Conference on Learning Representations, 2018.
  62. Pointer networks. Advances in neural information processing systems, 28, 2015.
  63. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229–256, 1992.
  64. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  12042–12049, 2021.
  65. Deepaco: Neural-enhanced ant systems for combinatorial optimization. arXiv preprint arXiv:2309.14032, 2023.
  66. Glop: Learning global partition and local construction for solving large-scale routing problems in real-time. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
  67. Learning to dispatch for job shop scheduling via deep reinforcement learning. Advances in Neural Information Processing Systems, 33:1621–1632, 2020.
  68. Generative flow networks for discrete probabilistic modeling. In International Conference on Machine Learning, pp.  26412–26428. PMLR, 2022a.
  69. Let the flows tell: Solving graph combinatorial optimization problems with gflownets. arXiv preprint arXiv:2305.17010, 2023a.
  70. Distributional gflownets with quantile flows. Transactions on Machine Learning Research, abs/2302.05793, 2023b.
  71. Robust scheduling with gflownets. In The Eleventh International Conference on Learning Representations, 2022b.
Citations (16)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets