On the Complexity of Best-Arm Identification in Non-Stationary Linear Bandits

Abstract¶

We study the fixed-budget best-arm identification (BAI) problem in non-stationary linear bandits. Concretely, given a fixed time budget Tin N, finite arm set X subset R^(d), and a potentially adversarial sequence of unknown parameters lbrace theta_trbrace_t = 1^(T) (hence non-stationary), a learner aims to identify the arm with the largest cumulative reward x_* = argmax_x in X x^(top)sum_t = 1^(T) theta_t with high probability. In this setting, it is well-known that uniformly sampling arms from the G-optimal design yields a minimax-optimal error probability of exp(-Theta(T / H_G)), where H_G scales proportionally with the dimension d. However, this notion of complexity is overly pessimistic, as it is derived from a lower bound in which the arm set consists only of the standard basis vectors, thus masking any potential advantages arising from arm sets with richer geometric structure. To address this, we establish an arm-set-dependent lower bound that, in contrast, holds for any arm set. Motivated by the ideas underlying our lower bound, we propose the Adjacent-optimal design, a specialization of the well-known XY-optimal design, and develop the textsfAdjacent-BAI algorithm. We prove that the error probability of textsfAdjacent-BAI matches our lower bound up to constants, verifying the tightness of our lower bound, and establishing the arm-set-dependent complexity of this setting.

Links¶

Similar Papers¶