Preprint, 2025

While foundation models have achieved remarkable results across a diversity of domains, they still rely on human-generated data, such as text, as a fundamental source of knowledge. However, this data is ultimately the product of human brains, the filtered projection of a deeper neural complexity. In this paper, we explore a new strategy for artificial intelligence: moving beyond surface-level statistical regularities by training foundation models directly on human brain data. We hypothesize that neuroimaging data could open a window into elements of human cognition that are not accessible through observable actions, and argue that this additional knowledge could be used, alongside classical training data, to overcome some of the current limitations of foundation models. While previous research has demonstrated the possibility to train classical machine learning or deep learning models on neural patterns, this path remains largely unexplored for high-level cognitive functions. Here, we classify the current limitations of foundation models, as well as the promising brain regions and cognitive processes that could be leveraged to address them, along four levels: perception, valuation, execution, and integration. Then, we propose two methods that could be implemented to prioritize the use of limited neuroimaging data for strategically chosen, high-value steps in foundation model training: reinforcement learning from human brain (RLHB) and chain of thought from human brain (CoTHB). We also discuss the potential implications for agents, artificial general intelligence, and artificial superintelligence, as well as the ethical, social, and technical challenges and opportunities. We argue that brain-trained foundation models could represent a realistic and effective middle ground between continuing to scale current architectures and exploring alternative, neuroscience-inspired solutions. We also note that future discoveries in cognitive and computational neuroscience could make this strategy increasingly relevant over time, as new neural signals of interest are retroactively unlocked in present neuroimaging datasets.

Frontiers in Systems Biology, 2025

Electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) are two widely used neuroimaging techniques, with complementary strengths and weaknesses. Predicting fMRI activity from EEG activity could give us the best of both worlds, and open new horizons for neuroscience research and neurotechnology applications. Here, we formulate this prediction objective both as a classification task (predicting whether the fMRI signal increases or decreases) and a regression task (predicting the value of this signal). We follow two distinct strategies: training classical machine learning and deep learning models (including MLP, CNN, RNN, and transformer) on an EEG-fMRI dataset, or leveraging the capabilities of pre-trained large language models (LLMs) and large multimodal models. We show that predicting fMRI activity from EEG activity is possible for the brain regions defined by the Harvard-Oxford cortical atlas, in the context of subjects performing a neurofeedback task. Interestingly, both strategies yield promising results, possibly highlighting two complementary paths for our prediction objective. Furthermore, a Chain-of-Thought approach demonstrates that LLMs can infer the cognitive functions associated with EEG data, and subsequently predict the fMRI data from these cognitive functions. The natural combination of the two strategies, i.e., fine-tuning an LLM on an EEG-fMRI dataset, is not straightforward and would certainly require further study. These findings could provide important insights for enhancing neural interfaces and advancing toward a multimodal foundation model for neuroscience, integrating EEG, fMRI, and possibly other neuroimaging modalities.

2025

In this first extension (X1) of Neuropolis, we continue the development of an artificial intelligence system for human brain activity prediction. Our objective is to predict fMRI activity from EEG activity, an endeavor that can be formulated as either a classification task, i.e., predicting whether the fMRI signal increases or decreases, or a regression task, i.e., predicting the value of this signal. To achieve this objective, we follow two distinct strategies: training machine learning and deep learning models (including feedforward neural networks, CNNs, RNNs, and transformers) on an EEG-fMRI dataset, or relying on the capabilities of pre-trained large language models (Gemma, Llama) and large multimodal models (PaliGemma). Our results show that predicting fMRI activity from EEG activity is possible for the brain regions defined by the Harvard-Oxford cortical atlas, even in the challenging context of subjects performing a cognitive task such as neurofeedback. Interestingly, both strategies yield promising results, possibly highlighting two complementary paths for predicting fMRI signals based on EEG signals. Furthermore, a Chain-of-Thought approach demonstrates that large language models can infer the cognitive functions associated with EEG data, and subsequently the fMRI data associated with these cognitive functions. The natural combination of both strategies, i.e., fine-tuning a large language model on an EEG-fMRI dataset, is not straightforward and would certainly require further study. Overall, the methods developed in this project could represent an important step for improving neural interfaces and advancing toward a multimodal foundation model for neuroscience.

2024

This project builds an API (Django, Docker) around Neuropolis. Our objective is to compute neurofeedback (NF) scores, whether directly on the Electroencephalography (EEG) activity, or indirectly on the predicted functional Magnetic Resonance Imaging (fMRI) activity. For this task, we use a series of machine learning models trained in the Neuropolis project, including linear regression, k-nearest neighbors, decision trees, random forests, and support vector machines.

2024

This project uses data science (NumPy, Pandas, Matplotlib), machine learning (Scikit-Learn), and deep learning (TensorFlow, Keras) tools on brain data obtained with Electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI). Our objective is to build an artificial intelligence system for human brain activity prediction, using EEG data to predict fMRI data, whether across experimental conditions or across subjects. For this regression task, we use a variety of machine learning and deep learning models, including linear regression, k-nearest neighbors, decision trees, random forests, support vector machines, fully connected neural networks, convolutional neural networks, recurrent neural networks, and transformers. Our results show that using EEG data to predict fMRI data is possible to a certain extent. Nevertheless, for our voxels of interest, the regression task seems too challenging, at least for relatively simple machine learning and deep learning models, even if the deep learning models perform well on a classification task. The methods developed in this project could provide useful insights for advancing toward a multimodal foundation model for neuroscience, and could help to improve promising brain technologies, such as neurofeedback systems and brain-computer interfaces.

2022

This project uses data science (NumPy, Pandas, Matplotlib), machine learning (Scikit-Learn) and deep learning (TensorFlow) tools on bandpowers obtained with Electroencephalography (EEG), brain images obtained with functional Magnetic Resonance Imaging (fMRI), and Neurofeedback (NF) scores computed using both techniques. Our objective is to explore several ways to apply machine learning models to EEG-fMRI NF data, using a dataset from the open data repository OpenNeuro. Specifically, we use machine learning models to classify brain images, to predict the value of the fMRI BOLD signal and fMRI NF scores, and to predict the value of the EEG bandpowers and EEG NF scores. Our results show that some of these tasks are possible, suggesting that machine learning can be used to extract subtle patterns from EEG and fMRI data in the context of NF training.

2022

This project uses data science (NumPy, Pandas, Matplotlib) and machine learning (Scikit-Learn) tools on bandpowers obtained with Electroencephalography (EEG) and brain images obtained with functional Magnetic Resonance Imaging (fMRI). Our objective is to train machine learning models to predict EEG activity from fMRI activity, and vice versa, using a dataset from the open data repository OpenNeuro. Specifically, we use machine learning models with EEG predictors to predict the value of the fMRI BOLD signal, and machine learning models with fMRI predictors to predict the value of the EEG bandpowers. Our results show that both tasks are possible, suggesting that each technique provides some insight on processes that are, traditionally, in the realm of the other technique.

2021

This project uses data science (NumPy, Pandas, Matplotlib), machine learning (Scikit-Learn) and deep learning (TensorFlow) tools on statistical maps of the human brain obtained with functional Magnetic Resonance Imaging (fMRI). Our objective is to train machine learning models to recognize and predict brain activity, using a dataset from the open data repository NeuroVault. Specifically, we use machine learning models to classify brain images, and to predict the values of voxels inside these brain images. Our results show that both classification and regression are possible, suggesting that the statistical maps obtained in this experimental setup contain relevant and generalizable knowledge about the brain activity.

2017-2022

We build upon the practical knowledge acquired after founding and directing the Institut Lémanique du Cerveau. During several years, this institute created and delivered innovative biofeedback, neurofeedback, and cognitive training services, based on real-time monitoring of brain states and cognitive processes, using physiological, neurological (EEG), and psychometric measures.

mael.donoso@ouroboros-neurotechnologies.com

You are welcome to contact us by email or using this form: