metadata.dc.type:	Trabalho de Conclusão de Curso
Title:	Generation of test datasets using LLM: a quality assurance perspective for software localization
metadata.dc.creator:	Sousa, Jose Leandro de Melo
metadata.dc.contributor.advisor1:	Rocha, Ana Carolina Oran
metadata.dc.contributor.referee1:	Souto, Eduardo James Pereira
metadata.dc.description.resumo:	Domain relevant data and an adequate number of samples are necessary to properly evaluate the robustness of the Machine Learning (ML) models. This is the case for ML models used in the software localization translation task. In general, Neural Machine Translation (NMT) models are used in software localization by automating the translation process of textual content to consider specific linguistic aspects and culture. However, unlike general machine translation which can easily utilize translation corpus for model training and testing, domain-specific machine translation faces a major obstacle due to the scarcity of domain-specific translation data. In the absence of adequate data, this paper first presents a method to generate test samples based on a text generation Large Language Model (LLM) approach. Based on the generated samples, we run tests to assess the robustness of an NMT translation model. The evaluation indicates that human judgment is important to check if the generated text is robust and coherent under different conditions. The evaluation also demonstrates that the generated samples were crucial to show some limitations related to the model’s effectiveness in software localization translation. Basically we discuss issues in specific situations such as date, time formats, numeric representations and measurement units.
Keywords:	Dataset creation LLMs evaluation Text generation Neural machine translation Software translation
metadata.dc.subject.cnpq:	CIENCIAS EXATAS E DA TERRA: CIENCIA DA COMPUTACAO: METODOLOGIA E TECNICAS DA COMPUTACAO: ENGENHARIA DE SOFTWARE
metadata.dc.language:	eng
metadata.dc.publisher.country:	Brasil
metadata.dc.publisher.department:	ICOMP - Instituto de Computação
metadata.dc.publisher.course:	Ciência da Computação - Bacharelado - Manaus
metadata.dc.rights:	Acesso Aberto
metadata.dc.rights.uri:	https://creativecommons.org/licenses/by-nc-nd/4.0/
URI:	http://riu.ufam.edu.br/handle/prefix/9911
Appears in Collections:	Trabalho de Conclusão de Curso - Graduação - Ciências Exatas e da Terra

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.