Multimodal Deep Learning

Amirreza Shaban () and Safoora Yousefi ()
Additional contact information
Amirreza Shaban: Cruise LLC
Safoora Yousefi: Microsoft

A chapter in Multimodal and Tensor Data Analytics for Industrial Systems Improvement, 2024, pp 209-219 from Springer

Abstract: Abstract Multimodal deep learning has gained significant attention and shown great promise in various domains, including medical, manufacturing, Internet of Things (IoT), remote sensing, and urban big data. This chapter provides an overview of neural network-based fusion techniques in multimodal deep learning. The advantages of deep learning over conventional shallow learning methods are discussed, highlighting its ability to learn both inter- and intra-modality representations with minimal preprocessing and implicit dimensionality reduction. The chapter explores different fusion methods, including early fusion, late fusion, and intermediate fusion, and discusses their capabilities and limitations. It also examines various objectives used in late fusion, such as reconstruction error, correlation-based objectives, and semantic alignment. The challenge of avoiding negative transfer in multimodal learning is addressed, and regularization objectives and training approaches are explored. Overall, this chapter serves as a comprehensive guide to multimodal deep learning and its fusion techniques, offering insights into their applications and potential for future research.

Keywords: Deep multimodal data fusion; Precision medicine; Autonomous riving (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:spochp:978-3-031-53092-0_10

Ordering information: This item can be ordered from
http://www.springer.com/9783031530920

DOI: 10.1007/978-3-031-53092-0_10

Access Statistics for this chapter

More chapters in Springer Optimization and Its Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().