FUSION3D: Multimodal data fusion for 3D shape reconstruction: A soft-sensing approach

Michael Biehler, Jing Li and Jianjun Shi

IISE Transactions, 2025, vol. 57, issue 9, 1041-1055

Abstract: Traditional high-fidelity imaging techniques, such as X-ray computer tomography (CT), excel in capturing intricate shape details through high-resolution two-dimensional (2D) images. However, the extensive cost or impracticality of obtaining X-ray images poses a significant challenge in various applications (e.g., dense metal components in manufacturing). To overcome this limitation, we propose a soft sensing approach, named FUSION3D, aimed at reconstructing three-dimensional (3D) shapes from 2D sliced images and heterogenous process inputs using a novel model based on multi-modal process knowledge. The developed FUSION3D method is built upon a deep learning framework that utilizes continuous normalizing flow to model the complex shape distribution of 3D objects. We propose a principled probabilistic framework to regularize 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes, and the second level is the distribution of points given a shape. Our generative regularization model learns each level of the distribution with a continuous normalizing flow. The invertibility of normalizing flows enables the computation of the likelihood during training and allows us to train our model in the variational inference framework. Our methodology is validated through a case study in additive manufacturing, demonstrating its effectiveness as a soft sensing approach for accurate 3D shape reconstruction without relying on high-fidelity imaging techniques like CT during model deployment.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/24725854.2024.2376650 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:uiiexx:v:57:y:2025:i:9:p:1041-1055

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/uiie20

DOI: 10.1080/24725854.2024.2376650

Access Statistics for this article

IISE Transactions is currently edited by Jianjun Shi

More articles in IISE Transactions from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().