Pre-trained multilingual language models (PMLMs), including mBERT and XLM-R, have demonstrated strong capabilities in cross-lingual transfer. Nonetheless, these models are not explicitly optimized to capture cross-lingual indicators related to sentiment-bearing terms. This limitation disproportionately affects low-resource languages (LRLs), which suffer from under-representation within these architectures. To optimize the fine-tuning of these models for sentiment analysis in LRLs, this paper presents a novel intermediate task fine-tuning (ITFT) approach centered on a sentiment lexicon from a high-resource language (HRL). The authors evaluate this methodology using a 3-class sentiment classification framework across three LRLs: Sinhala, Tamil, and Bengali. The findings indicate that the proposed strategy surpasses standard vanilla fine-tuning of the PMLM, while also performing on par with or exceeding conventional ITFT techniques that utilize HRL sentiment classification datasets.
Transparent Intelligence Publishing