Understanding the brain with attention: A survey of transformers in brain sciences

TL;DR

A survey of Transformer applications in brain sciences, covering disease diagnosis, brain age prediction, anomaly detection, segmentation, multimodal registration, fMRI modeling, EEG processing and multi-task learning, with model details and open-source code organized for replication.

中文简介：一篇关于 Transformer 在脑科学中应用的综述，涵盖脑疾病诊断、脑龄预测、异常检测、语义分割、多模态配准、fMRI 建模、脑电处理与多任务协同，并整理了模型细节与开源代码。

Key points

• Introduces the core Transformer architecture for readers in brain sciences.
• Reviews applications: brain disease diagnosis, brain age prediction, anomaly detection, semantic segmentation, multimodal registration, fMRI modeling, EEG processing and multi-task collaboration.
• Organizes model details and open-source code for reference and replication.
• Discusses quantitative assessment, model complexity and optimization, and future challenges.

Abstract

Owing to their superior capabilities and advanced achievements, Transformers have gradually attracted attention with regard to understanding complex brain processing mechanisms. This study aims to comprehensively review and discuss the applications of Transformers in brain sciences. First, we present a brief introduction of the critical architecture of Transformers. Then, we overview and analyze their most relevant applications in brain sciences, including brain disease diagnosis, brain age prediction, brain anomaly detection, semantic segmentation, multi-modal registration, functional Magnetic Resonance Imaging (fMRI) modeling, Electroencephalogram (EEG) processing, and multi-task collaboration. We organize the model details and open sources for reference and replication. In addition, we discuss the quantitative assessments, model complexity, and optimization of Transformers, which are topics of great concern in the field. Finally, we explore possible future challenges and opportunities, exploiting some concrete and recent cases to provoke discussion and innovation. We hope that this review will stimulate interest in further research on Transformers in the context of brain sciences.

License: Open access.

Citation

Cheng Chen, Huilin Wang, Yunqing Chen, Zihan Yin, Xinye Yang, Huansheng Ning, Qian Zhang, Weiguang Li, Ruoxiu Xiao, Jizong Zhao. Understanding the brain with attention: A survey of transformers in brain sciences. Brain-X, 1(3), e29 (2023). https://doi.org/10.1002/brx2.29

@article{chen2023transformers,
  title   = {Understanding the brain with attention: A survey of transformers in brain sciences},
  author  = {Chen, Cheng and Wang, Huilin and Chen, Yunqing and Yin, Zihan and Yang, Xinye and Ning, Huansheng and Zhang, Qian and Li, Weiguang and Xiao, Ruoxiu and Zhao, Jizong},
  journal = {Brain-X},
  volume  = {1},
  number  = {3},
  pages   = {e29},
  year    = {2023},
  doi     = {10.1002/brx2.29}
}

FAQ