Huh et al., ICML 2024
Multimodality could be achieved with relatively little or even no (as demonstrated by downstream vec2vec) pairs of representations. Especially useful for 3D, since this points to learning that scaling the raw volume of 3D data can lead to exploitable structures we can use for general intelligence, with very sparse 3D alignment data generally available.
Microsoft made a really good model for 3D
A really good model that demonstrated that a field-free, hierarchical representation that tries to represent the entire spatial structure is superior. A lot of my work is based on the ideas in Trellis 2.
ICLR 2025
Useful for understanding the dynamics of long-context systems and multimodal systems in terms of positions. While this paper investiages the uni-directional case, I think this is still useful
Ye et al., arXiv 2026
A unified framework for 3D understanding, generation, editing, and text-grounded part generation. It combines a 3D vision-language model with a diffusion transformer and scales training with a large multimodal 3D corpus. The paper finds that both generation and understanding improve editing.
Lai et al., arXiv 2025
VoxSet anchors a compact set of latent vectors to a coarse voxel grid, giving the representation explicit spatial structure while retaining the compression benefits of set-based latents. My takeaway is that compact, continuous latent representations can support scalable spatial intelligence for object generation, with positional properties emerging within the latents themselves.
Kang, Chen, Gkioxari & Perona, arXiv 2026
A neat explanation for how VLMs reason about space: attention carries the image encoder's positional information into text-token residuals as spatial IDs. These can linearly align with words like "left" and "right", so the model can reuse its text reasoning skills. Changing the IDs changes its spatial answers too.