Black-Mamba: Biologically-Inspired Leaky Accumulation for Conceptual Knowledge under Distribution Drift

Traditional test-time adaptive models often update internal states at every time step based on instantaneous prediction errors. This approach struggles to distinguish between persistent distribution drift and transient noise, leading to unnecessary updates, error accumulation, or catastrophic forgetting. This paper investigates whether a more selective, event-driven adaptation policy can maintain predictive accuracy while reducing the computational and statistical costs of continuous updates.

Black-Mamba introduces a three-stage architecture: a backbone predictor, a surprisal encoder, and a leaky evidence accumulator. Instead of updating parameters immediately upon observing a high loss, the model filters the surprisal signal through a leaky integrator. This accumulator acts as a low-pass filter, suppressing zero-mean innovation noise while allowing persistent drift to accumulate. A memory update—implemented here via a low-rank adapter (LoRA)—is triggered only when the accumulated evidence crosses a predefined threshold. This design draws inspiration from neuroscientific principles of long-term potentiation and synaptic tagging, where durable memory traces require coordinated reinforcement rather than isolated activation.

Theoretical analysis demonstrates that this leaky accumulation effectively bounds the variance of innovation noise while allowing structured drift to accumulate, providing a principled method for false-trigger control. Empirically, Black-Mamba achieves predictive performance competitive with continuous-update baselines across standard forecasting benchmarks. In synthetic drift scenarios, the model captures approximately 99% of the accuracy gains seen in continuous-update methods while reducing the total number of memory writes by nearly 50%. These results suggest that selective, evidence-gated adaptation is a robust alternative to continuous learning in non-stationary environments.