Published Feb 9, 2024
in
math.OC,cs.LG,
and
stat.ML
Abstract
Bandit convex optimisation is a fundamental framework for studying zeroth-order convex optimisation. These notes cover the many tools used for this problem, including cutting plane methods, interior point methods, continuous exponential weights, gradient descent and online Newton step. The nuances between the many assumptions and setups are explained. Although there is not much truly new here, some existing tools are applied in novel ways to obtain new algorithms. A few bounds are improved in minor ways.
Smoothed surrogates show better approximation for $-\log(x)$ due to its smoothness compared to $|x|$.
Overview
The paper discusses computational challenges in convex bandit and zeroth-order optimization, focusing on the manipulation of convex sets.
Various representations of convex sets and their computational implications are analyzed, including polytope, convex hull, separation oracle, and linear optimization oracle.
Methods for improving computational efficiency, such as projection algorithms, finding self-concordant barriers, and approximating MVEEs, are proposed.
Open problems in the field are identified, emphasizing the need for algorithms that balance efficiency and accuracy, adapt to different constraints, and scale with problem complexity.
Computation Challenges and Solutions in Convex Bandit and Zeroth-Order Optimization
Overview
Computational efficiency in convex bandit and zeroth-order optimization problems heavily depends on the representation and manipulation of convex sets. This paper discusses various computational challenges encountered when dealing with convex sets in optimization problems and proposes several methods to overcome these hurdles. The core of the discussion revolves around efficient ways to perform operations such as projections, finding self-concordant barriers, and approximating minimum volume enclosing ellipsoids (MVEEs). These operations are crucial in implementing algorithms for both convex bandit optimization and zeroth-order optimization.
Representation of Convex Sets
Convex sets can be represented in several ways, each offering different computational advantages and challenges:
Polytope representation allows defining a convex set as a collection of linear inequalities. Operations such as projection and finding a separation oracle are computationally feasible but can be complex depending on the size of the inequality system.
Convex hull representation involves defining a convex set through a set of points. While this representation simplifies certain computations, it becomes challenging when the point set is large.
Separation oracle provides a mechanism to determine if a point lies inside a convex set and, if not, to find a hyperplane that separates the point from the set. This approach is versatile and can be applied to various convex set representations but may require iterative procedures to find an accurate separation.
Linear optimization oracle offers a direct way to perform optimization over convex sets by solving a linear program defined by the convex set constraints. This method is efficient but lacks flexibility in handling non-linear constraints.
Computational Methods
Projection onto Convex Sets: Efficient projection algorithms are key in many optimization procedures. For polytope and convex hull representations, various Newton method-based and Frank-Wolfe algorithm-based techniques provide efficient solutions. When only separation oracles are available, the ellipsoid method serves as an effective though computationally intensive option.
Finding Self-Concordant Barriers: Identifying appropriate self-concordant barriers is crucial for interior-point methods in optimization. While logarithmic and volumetric barriers provide practical solutions for polytopes, identifying efficient barriers for more complex convex sets remains challenging.
Approximating Minimum Volume Enclosing Ellipsoids (MVEEs): Approximating MVEEs is vital for many optimization algorithms. The paper discusses procedures applicable to both polytope representations and separation oracles, ensuring computational efficiency while maintaining a balance between approximation fidelity and operational complexity.
Challenges and Open Problems
The computation in convex optimization, particularly in bandit and zeroth-order settings, presents various open problems:
Balancing Efficiency and Accuracy: Finding the most computationally efficient algorithms that do not compromise the accuracy of optimization solutions, especially in high-dimensional spaces.
Adaptive Algorithms: Developing algorithms that can adapt to different convex set representations and optimization constraints without significant manual tuning or prior knowledge.
Scalability: Ensuring that computational methods remain efficient as the dimensionality of the problem space and the complexity of the convex sets increase.
Conclusion
Effective computation in convex bandit and zeroth-order optimization is contingent upon the precise representation of convex sets and the efficiency of operations such as projection, barrier identification, and MVEE approximation. Future research directions include improving algorithmic adaptability, enhancing computational efficiency, and solving open problems related to convex set manipulation.