Track2Art

Understanding articulated objects is fundamental for robots to interact with the physical world, which requires accurate segmentation of object parts and their kinematic relations. Motion provides a strong cue for understanding articulated structure: points on the same rigid part exhibit coherent motion, while relative trajectories between parts directly expose their kinematic constraints. However, many existing approaches recover articulation indirectly by first reconstructing object geometry across a small number of articulation states and subsequently inferring motion through cross-state alignment or per-instance optimization. Such reconstruction-first pipelines can underutilize rich temporal evidence and couple articulation estimation to errors in geometry reconstruction and correspondence.

We present Track2Art, a motion-centric framework for recovering structured articulated object models from RGB-D interaction videos. Track2Art leverages a pretrained point tracker to obtain feature-rich 4D point trajectories. A motion-aware slot model groups trajectories into rigid object parts based on tracker appearance embeddings and explicit trajectory geometry. Given the recovered parts, a pairwise relation module predicts the directed kinematic graph and joint types, while a rotation-equivariant geometry module fuses motion-derived directional proposals and performs constrained axis-line refinement to recover joint geometry. Experimental results demonstrate the effectiveness of reasoning directly from persistent point motion for recovering structured articulated object models from dynamic visual observations.