An algorithm for 6D object pose estimation based on generative single-image 3D reconstruction

Reliable 6D object pose estimation is a foundational capability for robotic manipulation in Industry 4.0 and unstructured warehouse environments. Current state-of-the-art methods, such as PVN3D and SAM-6D, predominantly rely on “Model-Based” paradigms that necessitate high-fidelity CAD models of the target objects for training or template matching. This requirement severely limits their applicability in open-world scenarios where such CAD models are unavailable in real-time. To bridge this gap, a novel 6D Pose Estimation Framework without fine-tuning on the target object (Zero-Shot) is proposed to eliminate the dependency on prior CAD models. A single-view 3D generative model is leveraged to synthesize a geometrically plausible 3D mesh from a single RGB image. To address the scale ambiguity inherent in generative models, which poses a critical challenge for robotic grasping, a Template-Prior Iterative Closest Point (TP-ICP) based on the classical ICP is introduced for iterative recovery of the metric scale. This algorithm robustly recovers the metric physical scale of the neural-generated mesh using depth information, employing median-based outlier removal and template-guided ICP to handle occlusions and sensor noise. The recovered metric mesh is then seamlessly integrated into a standard 6D pose estimation pipeline, establishing a closed-loop framework comprising model self-generation, metric scale calibration, and pose estimation. A fully reproducible implementation scheme is presented. Systematic experiments on the LINEMOD dataset demonstrate that the proposed approach achieves robust 6D pose estimation suitable for robotic tasks, even in the complete absence of prior CAD models.

1. Thalhammer S., Bauer D., Hönig P. et al. Challenges for monocular 6-D object pose estimation in robotics // IEEE Trans. on Robotics. 2024. Vol. 40. P. 4065–4084.

2. Hay O. A. et al. Learning 6D Object Pose Estimation With Event Cameras Using Synthetic Data and Domain Randomization // IEEE Robotics and Automation Letters. 2025. N 11.2. P. 1690–1697.

3. Joyce E. C. et al. Consensus-Driven Uncertainty for Robotic Grasping based on RGB Perception // arXiv preprint arXiv:2506.20045. 2025.

4. Peng S. et al. PVNet: Pixel-wise voting network for 6D of pose estimation // Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019.

5. He Y., Sun W., Huang H. et al. PVN3D: A deep point-wise 3D keypoints voting network for 6dof pose estimation // Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2020.

6. Wang G. et al. GDR-Net: Geometry-guided direct regression network for monocular 6D object pose estimation // Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2021.

7. Hinterstoisser S. et al. Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes // Asian Conf. on Computer Vision. Springer Berlin Heidelberg, 2012.

8. Lin J. et al. SAM-6D: Segment anything model meets zero-shot 6D object pose estimation // Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2024.

9. Chen X. et al. SAM-3D: 3Dfy anything in images // arXiv preprint arXiv:2511.16624. 2025.

10. Xiang Y., Schmidt T., Narayanan V. et al. PoseCNN: A convolutional neural network for 6D object pose estimation in cluttered scenes // arXiv preprint arXiv:1711.00199. 2017.

11. Wang C., Xu D., Zhu Y. et al. DenseFusion: 6D object pose estimation by iterative dense fusion // Proc. of the IEEE/ CVF Conf. on Computer Vision and Pattern Recognition. 2019.

12. Nguyen V. N., Groueix T., Ponimatkin G. et al. CNOS: A strong baseline for cad-based novel object segmentation // Proc. of the IEEE/CVF Intern. Conf. on Computer Vision. 2023.

13. Kirillov A., Mintun E., Ravi N. et al. Segment anything // Proc. of the IEEE/CVF Intern. Conference on Computer Vision. 2023.

14. Mildenhall B., Srinivasan P. P., Tancik M. et al. NeRF: Representing scenes as neural radiance fields for view synthesis // Communications of the ACM. 2021. N 65(1). P. 99–106.

15. Kerbl B., Kopanas G., Leimkühler T. et al. 3D Gaussian splatting for real-time radiance field rendering // ACM Trans. Graph. 2023. N 42(4). P. 139:1–139:14.

16. Carion N., Gustafson L., Hu Y. T. et al. SAM 3: Segment anything with concepts // arXiv preprint arXiv:2511.16719.2025.

17. Besl P. J., McKay N. D. Method for registration of 3-D shapes // Sensor Fusion IV: Control Paradigms and Data Structures: Proc of Spie. 1992. Vol. 1611. P. 586–606.