A Comparative Study of PEGASUS, BART, and T5 for Text Summarization Across Diverse Datasets

Eman Daraghmi (), Lour Atwe and Areej Jaber
Additional contact information
Eman Daraghmi: Department of Computer Science, Palestine Technical University Kadoorie, Jaffa Street, Tulkarm 9993400, Palestine
Lour Atwe: Department of Computer Science, Palestine Technical University Kadoorie, Jaffa Street, Tulkarm 9993400, Palestine
Areej Jaber: Department of Computer Science, Palestine Technical University Kadoorie, Jaffa Street, Tulkarm 9993400, Palestine

Future Internet, 2025, vol. 17, issue 9, 1-33

Abstract: This study aims to conduct a comprehensive comparative evaluation of three transformer-based models, PEGASUS, BART, and T5 variants (SMALL and BASE), for the task of abstractive text summarization. The evaluation spans across three benchmark datasets: CNN/DailyMail (long-form news articles), Xsum (extreme single-sentence summaries of BBC articles), and Samsum (conversational dialogues). Each dataset presents unique challenges in terms of length, style, and domain, enabling a robust assessment of the models’ capabilities. All models were fine-tuned under controlled experimental settings using filtered and preprocessed subsets, with token length limits applied to maintain consistency and prevent truncation. The evaluation leveraged ROUGE-1, ROUGE-2, and ROUGE-L scores to measure summary quality, while efficiency metrics such as training time were also considered. An additional qualitative assessment was conducted through expert human evaluation of fluency, relevance, and conciseness. Results indicate that PEGASUS achieved the highest ROUGE scores on CNN/DailyMail, BART excelled in Xsum and Samsum, while T5 models, particularly T5-Base, narrowed the performance gap with larger models while still offering efficiency advantages compared to PEGASUS and BART. These findings highlight the trade-offs between model performance and computational efficiency, offering practical insights into model scaling—where T5-Small favors lightweight efficiency and T5-Base provides stronger accuracy without excessive resource demands.

Keywords: text summarization; transformer-based models; CNN/DailyMail; Xsum; Samsum; BART; T5; PEGASUS; ROUGE (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/17/9/389/pdf (application/pdf)
https://www.mdpi.com/1999-5903/17/9/389/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:17:y:2025:i:9:p:389-:d:1736648

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().