Design-Test: A Semi-Automated LLM-Based Framework for Typological Analysis with Human-in-the-Loop – A Case Study of Agreement Systems in the Bakhtiari Dialect of Izah
The rapid documentation of endangered Iranian dialects is hampered by a scalability crisis due to time-intensive manual methods. This study aimed to design and evaluate a semi-automated framework based on a Large Language Model (LLM) to accelerate typological analysis. Employing a human-in-the-loop workflow, the Llama 3.1 model and few-shot learning were used to extract agreement system patterns from a 5,400-sentence corpus of the Bakhtiari dialect of Izah. The model's performance was evaluated on a gold-standard annotated test set (450 sentences) using precision, recall, and F1-score metrics, followed by a qualitative error analysis.
The model achieved an overall accuracy of 87.4% and an F1-score of 87.4. Qualitative analysis of the 12.6% errors revealed two main limitations: 1) the model's bias towards high-frequency structures, and 2) its difficulty in processing specific morphophonological phenomena (e.g., broken plurals). In terms of efficiency, the framework yielded a 78% reduction in analysis time and enabled the processing of 4.6 times more data per unit of time. The findings demonstrate that LLMs, when embedded within a human-centric collaborative system, can dramatically enhance the speed and scale of low-resource language documentation without compromising accuracy. This framework provides a practical model for accelerating fieldwork in Iranian dialectology.
Mazraeh, S. and Alifar, N. (2026). Design-Test: A Semi-Automated LLM-Based Framework for Typological Analysis with Human-in-the-Loop – A Case Study of Agreement Systems in the Bakhtiari Dialect of Izah. Journal of Linguistics & Iranian Languages, (), -. doi: 10.22099/jlil.2026.55606.1473
MLA
Mazraeh, S. , and Alifar, N. . "Design-Test: A Semi-Automated LLM-Based Framework for Typological Analysis with Human-in-the-Loop – A Case Study of Agreement Systems in the Bakhtiari Dialect of Izah", Journal of Linguistics & Iranian Languages, , , 2026, -. doi: 10.22099/jlil.2026.55606.1473
HARVARD
Mazraeh, S., Alifar, N. (2026). 'Design-Test: A Semi-Automated LLM-Based Framework for Typological Analysis with Human-in-the-Loop – A Case Study of Agreement Systems in the Bakhtiari Dialect of Izah', Journal of Linguistics & Iranian Languages, (), pp. -. doi: 10.22099/jlil.2026.55606.1473
CHICAGO
S. Mazraeh and N. Alifar, "Design-Test: A Semi-Automated LLM-Based Framework for Typological Analysis with Human-in-the-Loop – A Case Study of Agreement Systems in the Bakhtiari Dialect of Izah," Journal of Linguistics & Iranian Languages, (2026): -, doi: 10.22099/jlil.2026.55606.1473
VANCOUVER
Mazraeh, S., Alifar, N. Design-Test: A Semi-Automated LLM-Based Framework for Typological Analysis with Human-in-the-Loop – A Case Study of Agreement Systems in the Bakhtiari Dialect of Izah. Journal of Linguistics & Iranian Languages, 2026; (): -. doi: 10.22099/jlil.2026.55606.1473
)