Create and explore imaginative 3D worlds with our new product Dreamfield (opens in a new tab)
Papers
Topics
Authors
Recent
Search
2000 character limit reached

Fast Generating A Large Number of Gumbel-Max Variables

Published 2 Feb 2020 in stat.CO, cs.LG, and stat.ML | (2002.00413v1)

Abstract: The well-known Gumbel-Max Trick for sampling elements from a categorical distribution (or more generally a nonnegative vector) and its variants have been widely used in areas such as machine learning and information retrieval. To sample a random element ii (or a Gumbel-Max variable ii) in proportion to its positive weight viv_i, the Gumbel-Max Trick first computes a Gumbel random variable gig_i for each positive weight element ii, and then samples the element ii with the largest value of gi+ln⁡vig_i+\ln v_i. Recently, applications including similarity estimation and graph embedding require to generate kk independent Gumbel-Max variables from high dimensional vectors. However, it is computationally expensive for a large kk (e.g., hundreds or even thousands) when using the traditional Gumbel-Max Trick. To solve this problem, we propose a novel algorithm, \emph{FastGM}, that reduces the time complexity from O(kn+)O(kn^+) to O(kln⁡k+n+)O(k \ln k + n^+), where n+n^+ is the number of positive elements in the vector of interest. Instead of computing kk independent Gumbel random variables directly, we find that there exists a technique to generate these variables in descending order. Using this technique, our method FastGM computes variables gi+ln⁡vig_i+\ln v_i for all positive elements ii in descending order. As a result, FastGM significantly reduces the computation time because we can stop the procedure of Gumbel random variables computing for many elements especially for those with small weights. Experiments on a variety of real-world datasets show that FastGM is orders of magnitude faster than state-of-the-art methods without sacrificing accuracy and incurring additional expenses.

Citations (4)

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.