MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data
(2403.11207)Abstract
Reconstructions of visual perception from brain activity have improved tremendously, but the practical utility of such methods has been limited. This is because such models are trained independently per subject where each subject requires dozens of hours of expensive fMRI training data to attain high-quality results. The present work showcases high-quality reconstructions using only 1 hour of fMRI training data. We pretrain our model across 7 subjects and then fine-tune on minimal data from a new subject. Our novel functional alignment procedure linearly maps all brain data to a shared-subject latent space, followed by a shared non-linear mapping to CLIP image space. We then map from CLIP space to pixel space by fine-tuning Stable Diffusion XL to accept CLIP latents as inputs instead of text. This approach improves out-of-subject generalization with limited training data and also attains state-of-the-art image retrieval and reconstruction metrics compared to single-subject approaches. MindEye2 demonstrates how accurate reconstructions of perception are possible from a single visit to the MRI facility. All code is available on GitHub.
Overview
-
ICML 2024 introduces electronic submission and consolidates appendices with the main document to streamline the review process.
-
Strict adherence to formatting guidelines, including a 10-point Times font and embedding only Type-1 fonts, is mandatory.
-
A double-blind review policy is enforced to ensure anonymity and fairness in the submission review process.
-
The formatting and submission instructions aim to maintain the integrity of the review process and encourage high readability and uniformity across papers.
Overview of ICML 2024 Submission and Formatting Guidelines
Submission Process
The International Conference on Machine Learning (ICML) 2024 emphasizes a purely electronic submission process through a designated website, diverging from email-based submissions. This year introduces a crucial update requiring authors to merge appendices with the main document into a single PDF file, ensuring that appendices are not overlooked during the review as separate entities. A strict page limit is enforced, with the main body of the paper capped at 8 pages, exclusive of references and appendices. Authors of accepted papers have the liberty to extend the main body by one additional page for the final submission.
Formatting Specifications
For the initial submission and the camera-ready copy, adherence to the specified formatting guidelines is mandatory. Papers must be presented in a PDF format utilizing a 10-point Times font. Special attention should be paid to embedding only Type-1 fonts to avoid compatibility issues. The figures and tables should be adequately sized and placed, following the guideline specifics for captions and alignments.
Double-Blind Review and Anonymity
ICML 2024 maintains a double-blind review policy requiring submissions to anonymize author and affiliation information. This extends to ensuring that citations to the authors' previous work are formulated in a manner that does not reveal the identity of the authors. Simultaneous submissions to other venues or any overlapping technical content with previously published work disqualify a submission from consideration.
Final Submission Instructions
Accepted papers must incorporate author names and affiliations while adhering to the provided formatting guidelines. The footnote indicating the preliminary review status must be updated to reflect the paper's acceptance and the copyright transfer to ICML. Adjustments are allowed in the header for a running title if the original exceeds the permissible length.
Theoretical Implications and Practical Applications
The systematic structuring and specific criteria set forth by the submission and formatting instructions for ICML 2024 not only streamline the submission process but also encourage uniformity and high readability across papers. This approach significantly aids in maintaining the integrity of the review process and ensures that papers meeting the conference's standards are judged on their merit and content quality. Looking forward, this rigorous adherence to format may inspire refinements in submission guidelines across related conferences, potentially leading to a standardization that could simplify the submission process for authors and the review process for organizers and reviewers alike.
In conclusion, the submission and formatting instructions for ICML 2024 serve as a detailed roadmap for authors, ensuring that submissions are not only consistent in their presentation but also conducive to a fair and blind review process. As machine learning continues to evolve, the importance of clear, accessible, and uniformly presented research cannot be overstated, underpinning the advancement of the field.
Create an account to read this summary for free: