Contribution Analysis of Large Language Models and Data Augmentations for Person Names in Solving Legal Bar Examination at COLIEE 2023

Takaaki Onaga (), Masaki Fujita () and Yoshinobu Kano ()
Additional contact information
Takaaki Onaga: Shizuoka University
Masaki Fujita: Shizuoka University
Yoshinobu Kano: Shizuoka University

The Review of Socionetwork Strategies, 2024, vol. 18, issue 1, 123-143

Abstract: Abstract This paper describes our system for COLIEE 2023 Task 4, which automatically answers Japanese legal bar exam problems. We propose an extension to our previous system in COLIEE 2022, which achieved the highest accuracy among all submissions using data augmentation. We focus on problems that include mentions of person names. In this paper, we present two main contributions. First, we incorporate LUKE as our deep learning component, which is a named entity recognition model trained on RoBERTa. Second, we fine-tune the pretrained LUKE model in multiple ways, comparing fine-tuning on training datasets that include alphabetical person names and ensembling different fine-tuning models. We confirmed that LUKE and its fine-tuned model on person type problems improve their accuracies. Our formal run results show that LUKE and our fine-tuning approach using alphabetical person names were effective, achieving an accuracy of 0.69 in the COLIEE 2023 Task 4 formal run.

Keywords: COLIEE 2023; Question answering; Legal information entailment; Predicate argument structure analysis (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s12626-024-00155-5 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:trosos:v:18:y:2024:i:1:d:10.1007_s12626-024-00155-5

Ordering information: This journal article can be ordered from
https://www.springer ... ystems/journal/12626

DOI: 10.1007/s12626-024-00155-5

Access Statistics for this article

The Review of Socionetwork Strategies is currently edited by Katsutoshi Yada, Yasuharu Ukai and Marshall Van Alstyne

More articles in The Review of Socionetwork Strategies from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().