Mask-Prior Knowledge Distillation for Efficient Binary Breast Ultrasound Classification

Windra Swastika and Paulus Lucky Tirma Irawan

Acta Informatica Pragensia, vol. preprint, No 1580

Abstract: Background: Binary classification of breast ultrasound (BUS) images into benign and malignant categories is a clinically critical task. Standard knowledge distillation (KD) compresses large teacher models into compact student models but transfers knowledge in a spatially blind manner, leaving ground-truth segmentation masks, routinely available in BUS datasets, entirely unexploited.Objective: This study proposes mask-prior knowledge distillation (MP-KD), a novel distillation framework that integrates ground-truth lesion segmentation masks as spatial priors to improve both classification performance and attention interpretability in a lightweight student model.Methods: A multi-teacher ensemble of EfficientNet-B4 and ResNet50, pre-trained and fine-tuned on the Breast Ultrasound Image (BUSI) dataset (697 images; 487 benign, 210 malignant), generates soft targets and attention guidance distilled into an EfficientNet-B0 student (5.3M parameters). The proposed mask-weighted attention transfer (MWAT) loss combines two components: MWAT-AT, which applies mask-derived spatial weights to the attention transfer loss to amplify distillation signals within lesion regions; and MWAT-MA, an auxiliary mask alignment objective that directly supervises student attention towards the ground-truth mask. Loss weights were selected via grid search (18 combinations) on the validation set. All the experiments were repeated with three random seeds (42, 123, 456) on a stratified 70/10/20 split; performance is reported as mean ± standard deviation on the held-out test set.Results: MP-KD achieves the highest AUC of 0.954 ± 0.004 among all the student configurations, outperforming standard KD (0.944 ± 0.002) and the no-KD baseline (0.949 ± 0.004). MP-KD also yields the highest mask-attention alignment score among all the distillation-based configurations (MAAS = 0.608 ± 0.006), substantially recovering the attention alignment lost during standard KD (0.577) while maintaining superior classification performance. Ablation studies show that MWAT-AT primarily drives accuracy (0.908) and specificity (0.921) improvements, while MWAT-MA yields the highest single-run sensitivity (0.952). The combined MP-KD model provides the most robust performance across all the seeds (MCC = 0.732 ± 0.007, AUC std = 0.004, smallest among all the configurations). Compared to the distilled ViT approach of Zhou and Mosadegh (AUC = 0.972, ~86M parameters), MP-KD achieves competitive performance with 16× fewer parameters.Conclusion: MP-KD demonstrates that ground-truth segmentation masks can be effectively integrated as spatial priors within the knowledge distillation pipeline, improving both the AUC and attention alignment of a compact student model compared to standard KD. The proposed MWAT mechanism offers a practical and interpretable solution for deploying breast cancer classification in resource-constrained clinical environments and is generalizable to other medical imaging tasks where segmentation annotations are available.

Keywords: Breast ultrasound; Knowledge distillation; Attention transfer; Binary classification; EfficientNet; Segmentation mask; Deep learning; Medical imaging (search for similar items in EconPapers)
References: Add references at CitEc
Citations:

Downloads: (external link)
http://aip.vse.cz/doi/10.18267/j.aip.325.html (text/html)
free of charge

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:prg:jnlaip:v:preprint:id:325

Ordering information: This journal article can be ordered from
Redakce Acta Informatica Pragensia, Katedra systémové analýzy, Vysoká škola ekonomická v Praze, nám. W. Churchilla 4, 130 67 Praha 3
http://aip.vse.cz

DOI: 10.18267/j.aip.325

Access Statistics for this article

Acta Informatica Pragensia is currently edited by Editorial Office

More articles in Acta Informatica Pragensia from Prague University of Economics and Business Contact information at EDIRC.
Bibliographic data for series maintained by Stanislav Vojir ().