Morphology-Aware Block Sparsification for Low-Resource Translation

Abstract

The rapid advancement of neural machine translation has historically relied on the availability of massive parallel corpora and increasingly large over-parameterized models. However, scaling these architectures poses significant challenges for low-resource languages, which often feature rich morphology and limited training data. In such resource-constrained scenarios, massive models tend to overfit, while deploying them on edge devices remains computationally prohibitive. Model compression techniques, particularly network sparsification, offer a promising solution to reduce memory footprint and improve inference speed. Yet, conventional magnitude-based and unstructured sparsification methods are language-agnostic and frequently disrupt the intricate morphological representations essential for translating morphologically rich languages. This paper introduces a novel morphology-aware block sparsification strategy designed specifically for low-resource translation. By integrating linguistic priors derived from morphological segmentation into the block sparsity scoring mechanism, the proposed framework strategically preserves parameter clusters responsible for morphological inflection, derivation, and syntactic agreement. We formalize this approach through a dynamically weighted masking function and evaluate it on several low-resource language pairs. Extensive quantitative and qualitative analyses demonstrate that the proposed method significantly outperforms traditional sparsification baselines in translation quality, preserving structural linguistic integrity while achieving substantial reductions in model size and computational overhead. The findings highlight the critical importance of incorporating linguistic structures into model compression paradigms.