Transformer-Based Approaches for Legal Text Processing

Ha-Thanh Nguyen (), Minh-Phuong Nguyen (), Thi-Hai-Yen Vuong (), Minh-Quan Bui (), Minh-Chau Nguyen (), Tran-Binh Dang (), Vu Tran (), Le-Minh Nguyen () and Ken Satoh ()
Additional contact information
Ha-Thanh Nguyen: Japan Advanced Institute of Science and Technology
Minh-Phuong Nguyen: Japan Advanced Institute of Science and Technology
Thi-Hai-Yen Vuong: University of Engineering and Technology, VNU
Minh-Quan Bui: Japan Advanced Institute of Science and Technology
Minh-Chau Nguyen: Japan Advanced Institute of Science and Technology
Tran-Binh Dang: Japan Advanced Institute of Science and Technology
Vu Tran: Institute of Statistical Mathematics
Le-Minh Nguyen: Japan Advanced Institute of Science and Technology
Ken Satoh: National Institute of Informatics

The Review of Socionetwork Strategies, 2022, vol. 16, issue 1, 135-155

Abstract: Abstract In this paper, we introduce our approaches using Transformer-based models for different problems of the COLIEE 2021 automatic legal text processing competition. Automated processing of legal documents is a challenging task because of the characteristics of legal documents as well as the limitation of the amount of data. With our detailed experiments, we found that Transformer-based pretrained language models can perform well with automated legal text-processing problems with appropriate approaches. We describe in detail the processing steps for each task such as problem formulation, data processing and augmentation, pretraining, finetuning. In addition, we introduce to the community two pretrained models that take advantage of parallel translations in legal domain, NFSP and NMSP. In which, NFSP achieves the state-of-the-art result in Task 5 of the competition. Although the paper focuses on technical reporting, the novelty of its approaches can also be an useful reference in automated legal document processing using Transformer-based models.

Keywords: Transformer; Legal Text Processing; COLIEE; JNLP (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s12626-022-00102-2 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:trosos:v:16:y:2022:i:1:d:10.1007_s12626-022-00102-2

Ordering information: This journal article can be ordered from
https://www.springer ... ystems/journal/12626

DOI: 10.1007/s12626-022-00102-2

Access Statistics for this article

The Review of Socionetwork Strategies is currently edited by Katsutoshi Yada, Yasuharu Ukai and Marshall Van Alstyne

More articles in The Review of Socionetwork Strategies from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().