Learning Residual Kinematic Corrections for Continuous Neural Decoding via Reinforcement Learning

Decoding continuous 3D motor imagery from non-invasive EEG signals is often hindered by signal non-stationarity and systematic errors in neural decoders. The authors investigate whether a secondary, offline reinforcement learning (RL) layer can correct these residual kinematic errors without the need for further neural data collection or complex online model updates.

The researchers propose a two-stage 'CNN-LSTM-RL' architecture. In the first stage, a standard CNN-LSTM model is trained to decode 3D limb velocity from EEG signals. In the second stage, an RL agent—specifically using the Soft Actor-Critic (SAC) algorithm—is trained offline to learn a residual correction policy. This agent takes the decoder's output, time, and previous velocity as input and outputs a correction term that is added to the initial prediction. By training the RL agent offline on pre-recorded kinematic data, the system avoids the instability of using noisy EEG signals as a direct reward signal during the correction phase.

The proposed framework demonstrated substantial improvements in decoding accuracy across both 2D screen and immersive VR environments. In 2D tasks, the mean Pearson correlation coefficient increased from 0.5076 to 0.7181, while RMSE was reduced by approximately 40%. Similar performance gains were observed in VR, with correlation increasing from 0.6420 to 0.7780. These results suggest that residual RL is a highly effective, scalable method for refining BCI outputs post-decoding.

This study provides a robust, modular solution for enhancing BCI performance. By decoupling the neural decoding (CNN-LSTM) from the kinematic correction (RL), the framework allows for performance optimization that is isolated from the inherent noise and variability of EEG signals. This approach is particularly promising for neurorehabilitation and prosthetic control, where precise, continuous movement tracking is essential for user success.