Abstract:This study explores a method to adjust the dynamic upper confidence bound (UCB) coefficient c_i(t) based on reward changes, aiming to improve the performance of the UCB algorithm in dynamic environments. The conventional UCB algorithm balances exploration and exploitation through a fixed upper confidence bound coefficient. However, in a changing environment, a fixed coefficient may lead to excessive exploration or exploitation, affecting the adaptability of the algorithm. To address this issue, a decay mechanism is proposed based on the reward change rate, which dynamically adjusts the upper confidence bound coefficient for each arm. This enables the algorithm to flexibly respond to environmental changes and more effectively balance exploration and exploitation. Experimental results show that the UCB algorithm based on the reward decay mechanism outperforms the standard UCB algorithm in dynamic environments with significant reward fluctuations. By dynamically adjusting the upper confidence bound coefficient according to the historical performance of each arm, the algorithm reduces exploration of uncertain arms and concentrates resources on exploiting better-performing arms more quickly, thereby increasing cumulative rewards and accelerating convergence. This optimization method has significant advantages in dynamic and complex environments and can effectively enhance the long-term performance of the UCB algorithm.