This paper presents a biologically inspired basal ganglia reinforcement-learning model that uses dopamine and noradrenaline regulation plus an Ising-model STN network to generate structured exploration, demonstrating improved adaptation in tasks and impaired performance when DA/NA are clamped.

Although not directly therapeutic, the model links DA/NA and STN population dynamics to decision-making deficits in Parkinson's disease, offering a mechanistic computational platform to generate testable hypotheses for neuromodulation strategies or noradrenergic-focused interventions.

As a core brain region for motor control and cognitive decision-making, the basal ganglia relies on the dynamic balance among direct, indirect, and hyperdirect pathways, which is maintained by the synergistic action of multiple neurotransmitters. To address the critical limitations of traditional reinforcement learning algorithms, including low exploration efficiency and poor environmental adaptability, this study proposes a multi-neurotransmitter-regulated basal ganglia reinforcement learning (MNBG-RL) model inspired by the neurocomputational principles of the cortex-basal ganglia-thalamus loop. The model features three core innovations: First, a dual neurotransmitter regulation system of dopamine (DA) and noradrenaline (NA) is constructed, where DA encodes reward prediction errors and dynamically modulates pathway weights through D1/D2 receptors, while NA, under the regulation of DA, enables precise generation of exploration intensity by modulating subthalamic nucleus (STN) states. Second, a two-dimensional Ising network is embedded in the indirect pathway to simulate the population activity of STN neurons, generating adaptive structured exploration signals (replacing traditional blind Gaussian noise) that support smooth state transitions among ordered, critical, and disordered regimes. Third, the model performance is validated on three typical tasks: (1) the model flexibly adapts to environmental reversals and converges to optimal policies in complex decision-making tasks; (2) clamping DA/NA levels (simulating impaired neurotransmitter systems) significantly reduces decision flexibility and even leads to task failure; (3) compared with traditional exploration methods, the MNBG-RL model achieves substantially improved convergence rates and greatly reduced policy oscillation. This study not only supports the development of biologically plausible basal ganglia-inspired reinforcement learning frameworks but also provides a computational platform for modeling decision-making deficits in neurological disorders such as Parkinson's disease.