The article discusses MindEye2, a model capable of reconstructing images from fMRI data using Stable Diffusion XL and unCLIP. It explores the intricate processes of aligning brain activity with visual stimuli, emphasizing image captioning and retrieval methods. Contributions from various authors highlight different aspects of the research, focusing on model evaluation and comparative analyses. Results demonstrate the efficacy of MindEye2 in both image reconstruction and caption generation, showcasing its application across multiple subjects. The article concludes with future implications for understanding brain functions through advanced image processing techniques.
PSS: project lead, drafted the initial manuscript and contributed to all parts of MindEye2 development.
MT (core contributor): ... experimented with approaches not used in the final model including training custom ControlNet and T2I adapters.
CKTV (core contributor): ... and experimented with approaches not used in the final model including trying out different pretrained model embeddings.
RK (core contributor): brain correlations, human preference experiments, recalculated metrics for 40-hou.
Collection
[
|
...
]