Layout Aware Semantic Element Extraction for Sustainable Science & Technology Decision Support

Hyuntae Kim, Jongyun Choi, Soyoung Park and Yuchul Jung
Additional contact information
Hyuntae Kim: Department of Computer Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea
Jongyun Choi: Department of Computer Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea
Soyoung Park: Department of Computer Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea
Yuchul Jung: Department of Computer Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea

Sustainability, 2022, vol. 14, issue 5, 1-18

Abstract: New scientific and technological (S&T) knowledge is being introduced rapidly, and hence, analysis efforts to understand and analyze new published S&T documents are increasing daily. Automated text mining and vision recognition techniques alleviate the burden somewhat, but the various document layout formats and knowledge content granularities across the S&T field make it challenging. Therefore, this paper proposes LA-SEE (LAME and Vi-SEE), a knowledge graph construction framework that simultaneously extracts meta-information and useful image objects from S&T documents in various layout formats. We adopt Layout-aware Metadata Extraction (LAME), which can accurately extract metadata from various layout formats, and implement a transformer-based instance segmentation (i.e., Vision based Semantic Elements Extraction (Vi-SEE)) to maximize the vision-based semantic element recognition. Moreover, to constructing a scientific knowledge graph consisting of multiple S&T documents, we newly defined an extensible Semantic Elements Knowledge Graph (SEKG) structure. For now, we succeeded in extracting about 6 million semantic elements from 49,649 PDFs. In addition, to illustrate the potential power of our SEKG, we provide two promising application scenarios, such as a scientific knowledge guide across multiple S&T documents and questions and answering over scientific tables.

Keywords: multi-modal; document layout analysis; metadata; document structure; document object; semantic elements; knowledge graph; transformer; decision support (search for similar items in EconPapers)
JEL-codes: O13 Q Q0 Q2 Q3 Q5 Q56 (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2071-1050/14/5/2802/pdf (application/pdf)
https://www.mdpi.com/2071-1050/14/5/2802/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jsusta:v:14:y:2022:i:5:p:2802-:d:760423

Access Statistics for this article

Sustainability is currently edited by Ms. Alexandra Wu

More articles in Sustainability from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().