CPEL: A Causality-Aware, Parameter-Efficient Learning Framework for Adaptation of Large Language Models with Case Studies in Geriatric Care and Beyond

Jinzhong Xu, Junyi Gao, Xiaoming Liu (), Guan Yang, Jie Liu, Yang Long, Ziyue Huang and Kai Yang
Additional contact information
Jinzhong Xu: School of Artificial Intelligence, Zhongyuan University of Technology, Zhengzhou 450007, China
Junyi Gao: School of Artificial Intelligence, Zhongyuan University of Technology, Zhengzhou 450007, China
Xiaoming Liu: School of Artificial Intelligence, Zhongyuan University of Technology, Zhengzhou 450007, China
Guan Yang: School of Artificial Intelligence, Zhongyuan University of Technology, Zhengzhou 450007, China
Jie Liu: School of Information Science, North China University of Technology, Beijing 100144, China
Yang Long: Department of Computer Science, Durham University, Durham DH1 3LE, UK
Ziyue Huang: School of Artificial Intelligence, Zhongyuan University of Technology, Zhengzhou 450007, China
Kai Yang: School of Artificial Intelligence, Zhongyuan University of Technology, Zhengzhou 450007, China

Mathematics, 2025, vol. 13, issue 15, 1-24

Abstract: Adapting Large Language Models (LLMs) to specialized domains like geriatric care remains a significant challenge due to the limited availability of domain-specific data and the difficulty of achieving efficient yet effective fine-tuning. Current methods often fail to effectively harness domain-specific causal insights, which are crucial for understanding and solving complex problems in low-resource domains.To address these challenges, we propose Causality-Aware, Parameter-Efficient Learning (CPEL), a novel framework that leverages domain-specific causal relationships to guide a multi-layer, parameter-efficient fine-tuning process for more effective domain adaptation. By embedding causal reasoning into the model’s adaptation pipeline, CPEL enables efficient specialization in the target domain while maintaining strong task-specific performance. Specifically, the Causal Prompt Generator of CPEL extracts and applies domain-specific causal structures, generating adaptive prompts that effectively guide the model’s learning process. Complementing this, the MPEFT module employs a dual-adapter mechanism to balance domain-level adaptation with downstream task optimization. This cohesive design ensures that CPEL achieves resource efficiency while capturing domain knowledge in a structured and interpretable manner. Based on this framework, we delved into its application in the field of geriatric care and trained a specialized large language model (Geriatric Care LLaMA) tailored for the aged-care domain, leveraging its capacity to efficiently integrate domain expertise. Experimental results from question-answering tasks demonstrate that CPEL improves ROUGE scores by 9–14% compared to mainstream LLMs and outperforms frontier models by 1–2 points in auto-scoring tasks. In summary, CPEL demonstrates robust generalization and cross-domain adaptability, highlighting its scalability and effectiveness as a transformative solution for domain adaptation in specialized, resource-constrained fields.

Keywords: causality-aware; parameter-efficient learning; domain adaptation; geriatric care (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/15/2460/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/15/2460/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:15:p:2460-:d:1713534

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().