A Comparative Evaluation of Back Translation and Easy Data Augmentation for Indonesian Automatic Short Answer Scoring

Abstract

Automatic Short Answer Scoring (AES) is a Natural Language Processing (NLP) application designed to automatically assess short-answer responses. One of the primary challenges in developing AES systems is the limited size and diversity of available datasets, which can adversely affect a model’s generalization capability. Previous studies have demonstrated that Easy Data Augmentation (EDA) based on IndoBERT-generated synonyms can improve model performance on the UKARA dataset; however, this approach remains limited because the augmentation process is performed at the word level. This study aims to compare the effectiveness of Back Translation and IndoBERT-based Synonym EDA for Indonesian AES systems using the UKARA dataset. To ensure a fair comparison, the dataset, preprocessing procedures, FastText-based text representation, BiLSTM architecture, and evaluation methods were kept consistent across experiments, allowing performance differences to be attributed solely to the augmentation techniques. The experiments were conducted using both Non-K-Fold Evaluation and 3-Fold Cross-Validation scenarios. The results indicate that Back Translation outperformed IndoBERT-based Synonym EDA in most experimental settings, achieving the highest accuracy of 89.00% on Dataset A. Furthermore, the findings suggest that the quality and semantic diversity of the generated data have a greater impact on model performance than merely increasing the amount of training data. Therefore, Back Translation can serve as an effective alternative for enhancing dataset quality and improving the performance of Indonesian AES systems.

Keywords: Automatic Short Answer Scoring, Back Translation, Easy Data Augmentation, IndoBERT, BiLSTM, FastText.