Multi-SpatialMLLM
[CVPR 2026] Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models
About
We present Multi-SpatialMLLM to equip MLLMs with robust multi-frame spatial understanding by integrating depth perception, visual correspondence, and dynamic perception. Central to our approach is the MultiSPA dataset, a novel, large-scale collection of more than 27 million samples spanning diverse 3D and 4D scenes. Alongside MultiSPA, we introduce a comprehensive benchmark that tests a wide spectrum of spatial tasks under uniform metrics. Our model achieves significant gains over baselines and proprietary systems, demonstrating scalable, generalizable multi-frame reasoning. We further observe multi-task benefits and early indications of emergent capabilities in challenging scenarios, and showcase our model can serve as a multi-frame reward annotator for robotics.
Open Source Health
- Stars
- 179
- Forks
- 9
- License
- Not stated
- Last commit
- 8 months ago
Alternatives to Multi-SpatialMLLM
- RPG_KDD2025by FacebookresearchThis repository provides the code for implementing RPG described in our KDD'25 paper "Generating Long Semantic IDs in Parallel for Recommendation".
Seamless Communicationby FacebookresearchFoundational Models for State-of-the-Art Speech and Text Translation
OrienterNetby FacebookresearchSource Code for Paper "OrienterNet Visual Localization in 2D Public Maps with Neural Matching"- Matrixby FacebookresearchMatrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data generation.
- Spiderby FacebookresearchA general physic-based retargeting framework.
Resources & Links
Related Categories
Using Multi-SpatialMLLM?
Track its cost next to the rest of your stack and get a reminder before it renews.
Add to my stackVendor
Facebookresearch
Publisher of Fairseq
