Brain-OF:一种用于fMRI、EEG和MEG的全功能基础模型

📄 中文摘要

脑基础模型在神经科学任务中取得了显著进展。然而,大多数现有模型仅限于单一功能模态,限制了其利用互补时空动态和跨成像技术的集体数据规模的能力。为了解决这一限制,提出了Brain-OF,这是第一个联合预训练于fMRI、EEG和MEG的全功能脑基础模型,能够在统一框架内处理单模态和多模态输入。为了解决异构时空分辨率的问题,引入了Any-Resolution神经信号采样器,将不同的脑信号投影到共享的语义空间。此外,为了进一步管理语义转变,Brain-O模型采用了创新的方法来优化多模态数据的整合。

📄 English Summary

Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG

The study introduces Brain-OF, the first omnifunctional brain foundation model jointly pretrained on fMRI, EEG, and MEG, which is capable of handling both unimodal and multimodal inputs within a unified framework. Existing brain foundation models have made significant strides in various neuroscience tasks, but they are often limited to a single functional modality, hindering their ability to leverage complementary spatiotemporal dynamics and the collective data scale across different imaging techniques. To address this limitation, Brain-OF incorporates the Any-Resolution Neural Signal Sampler, which projects diverse brain signals into a shared semantic space to reconcile heterogeneous spatiotemporal resolutions. Furthermore, innovative methods are employed to manage semantic shifts, enhancing the integration of multimodal data.

Powered by Cloudflare Workers + Payload CMS + Claude 3.5

数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等