Multi-SpatialMLLM
[CVPR 2026] Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models
About
We present Multi-SpatialMLLM to equip MLLMs with robust multi-frame spatial understanding by integrating depth perception, visual correspondence, and dynamic perception. Central to our approach is the MultiSPA dataset, a novel, large-scale collection of more than 27 million samples spanning diverse 3D and 4D scenes. Alongside MultiSPA, we introduce a comprehensive benchmark that tests a wide spectrum of spatial tasks under uniform metrics. Our model achieves significant gains over baselines and proprietary systems, demonstrating scalable, generalizable multi-frame reasoning. We further observe multi-task benefits and early indications of emergent capabilities in challenging scenarios, and showcase our model can serve as a multi-frame reward annotator for robotics.
Open Source Health
- Stars
- 179
- Forks
- 9
- License
- Not stated
- Last commit
- 8 months ago
Resources & Links
Related Categories
Vendor
Facebookresearch
Publisher of Fairseq
Quick Links
Open Source
More by Facebookresearch
Related Products
RPG_KDD2025
This repository provides the code for implementing RPG described in our KDD'25 paper "Generating Long Semantic IDs in Parallel for Recommendation".
More from this vendor
Seamless Communication
Foundational Models for State-of-the-Art Speech and Text Translation
More from this vendor
OrienterNet
Source Code for Paper "OrienterNet Visual Localization in 2D Public Maps with Neural Matching"
More from this vendor
Matrix
Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data generation.
More from this vendor
Spider
A general physic-based retargeting framework.
More from this vendor