Skip to main navigation Skip to search Skip to main content

Authenticating Matryoshka Nesting Dolls via an Auditable 2D–3D–Text Evidence Framework with BMA Compression and Zero-Shot 3D Completion

  • Princeton University

Research output: Contribution to journalArticlepeer-review

Abstract

Authenticating cultural heritage artifacts such as Matryoshka Nesting Dolls (MNDs) is increasingly complicated by high-fidelity replicas that successfully mimic surface textures and palettes, leading traditional 2D computer vision models to exhibit dangerous overconfidence in false-positive classifications. To address this, we propose an auditable multimodal framework that transitions from appearance-only detection to a robust verification system based on the following three technical pillars: (1) a 2D visual stream utilizing a ConvNeXt-Tiny backbone for fine-grained style recognition; (2) a 3D geometric stream employing a custom 2D-to-3D reconstruction pipeline based on the Blum Medial Axis (BMA) and surfaces of revolution to capture axisymmetric structural fidelity; and (3) a semantic stream leveraging the Qwen3-VL vision-language model to generate human-interpretable evidence cards. To support this framework, we introduce a novel multimodal dataset comprising 168 unique physical MND sets and 27,387 labeled frames, archived for reproducibility. Our experimental results demonstrate that while 2D-only baselines achieve 77.9% authenticity accuracy, they suffer from a high Expected Calibration Error (ECE) of 0.121. The integrated multimodal framework achieves a superior authenticity accuracy of 96.7% and reduces the ECE to 0.041, representing a 66% improvement in calibration reliability. Crucially, the system shifts the mean confidence for incorrect replica classifications from a high-risk 0.82 to a safe 0.45.

Original languageEnglish
Article number992
JournalElectronics (Switzerland)
Volume15
Issue number5
DOIs
StatePublished - Mar 2026

Keywords

  • Blum Medial Axis
  • Matryoshka nesting dolls
  • cultural heritage authentication
  • digital twins
  • multimodal fusion
  • vision-language models
  • zero-shot 3D reconstruction

Fingerprint

Dive into the research topics of 'Authenticating Matryoshka Nesting Dolls via an Auditable 2D–3D–Text Evidence Framework with BMA Compression and Zero-Shot 3D Completion'. Together they form a unique fingerprint.

Cite this