Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

Sleep health notes

Sleep health notes will be added by the Sleepatch editorial team.

中文解读

中文解读待补充：本站会优先为失眠研究、睡眠质量改善、昼夜节律等高价值睡眠研究添加中文说明。

Original abstract

One of the critical limitations of artificial neural networks is their lack of ability to continually learn: training on new tasks often leads to interference and forgetting of the previous ones. While several algorithms have been proposed to protect old memories from interference, they are typically applied during or immediately after each new episode of training. In contrast, humans and animals can learn continuously, acquiring multiple new memories during active learning before consolidating all of them into long-term storage. Here we show that multiple new tasks can be trained sequentially before an unsupervised sleep-like replay phase is applied to partially restore performance across all previously learned tasks. Our study further suggests that task-specific information remains resilient to new training but decays gradually as network is trained on new tasks. These findings point to novel principles for developing a broad range of continual learning AI solutions.

Links and sources

• Official / arXiv page
• PDF from original source

Comments