Transformer-Based Clinical Annotation of Lung Cancer Reports: A Benchmark and Fine-Tuning Study on a Novel Tunisian Corpus

Abstract

Background: Lung cancer causes more deaths than any other malignancy worldwide, accounting for 2.2 million new cases and 1.8 million deaths in 2020. Extracting structured clinical knowledge from unstructured French-language oncology records remains methodologically unresolved in Tunisian and Francophone healthcare systems, where validated natural language processing tools do not yet exist. This study examined the effectiveness of transformer-based named-entity recognition for automated clinical annotation of Tunisian lung cancer reports. Aim: The study aimed to (i) establish performance baselines for four transformer-based models on a publicly available thoracic radiology dataset, (ii) evaluate five models, including a French biomedical specialist, on a newly constructed Tunisian clinical corpus, and (iii) demonstrate prototype deployment feasibility for structured clinical decision support. Methods: An initial comparative study evaluated BERT, RoBERTa, BioClinicalBERT, and CamemBERT using the official RadGraph dataset partitions, which natively comprise a total of 600 annotated thoracic radiology reports distributed across a standardized 80/10/10 split. Subsequently, five models were evaluated on 200 manually annotated diagnostic reports from Mami Pneumo-Phthisiology Hospital, Tunis. For the Tunisian corpus, a five-fold cross-validation approach was implemented to ensure robust performance estimation, followed by final evaluation on a dedicated hold-out test set. All models were trained for a maximum of 10 epochs, with a learning rate of 5 × 10−5 and a batch size of 16. Results: Based on the initial comparative study on the RadGraph dataset, where RoBERTa was the top performer and achieved the highest F1-score of 0.873 (precision: 0.869, recall: 0.877), we evaluated its specialized biomedical variant, DR-BERT, on our Tunisian clinical dataset. DR-BERT demonstrated strong generalization on the hold-out test set with an F1-score of 0.824, outperforming the baseline RoBERTa (test F1: 0.791) and showing competitive performance relative to multilingual BERT (0.843 ± 0.005 in five-fold cross-validation). A prototype interface generated structured clinical summaries encompassing prior conditions, imaging modalities, and TNM staging. Conclusion: Language- and domain-adapted transformer models effectively extract structured clinical entities from French-language Tunisian lung cancer reports. DR-BERT’s superior generalization on unseen data confirms that biomedical pretraining in the target language is a key driver of robust performance in specialized French oncology text. This work establishes foundational infrastructure for NLP-driven oncology data management in Tunisia and comparable Francophone settings.