Representação de dados heterogêneos em cenário de poucos dados aplicada a automação de teste de software através de redes siamesas

Abstract

The release of new versions of the Android operating system prompts mobile device manufacturers to introduce their own updates to ensure software compatibility and quality. However, to guarantee its quality, it is necessary to conduct rigorous tests on it, which often involves increasing expenses. Therefore, there is a clear need to automate this process as much as possible. To achieve this, companies can have a framework that includes a series of automation commands, designed to carry out simple testing tasks. In this context, an operator is tasked with reading the description of a test case and selecting the corresponding command. Faced with this problem, the objective of this work is to help operators in the search for commands, where for each step of a test case, look for the command that performs this action. Siamese networks were used, combined with MLM (Masked Language Model), to represent both steps and automation commands in the same vector space. This allows us to search for commands based on cosine similarity. We propose to use a loss function that approximates the test steps of their corresponding commands, so that it is close in the latent representation space. Furthermore, the use of BLOOM, a language model, was incorporated to generate synthetic data that assists in searching for commands when there is no corresponding pair of test steps. To represent the data, the RoBERTa model was used and finally, LambdaMART was applied to perform a ranking fusion on the classification suggestions given in the rankings, thus creating an enriched ranking. The final results were very promising in both proposed experiments, evaluated using the MRR, MAP and HitRate metrics, where there was an average of 0.58 for MRR in the first experiment and 0.31 in the second experiment, concluding that the proposal presented is effective in recommending the correct commands in the highest positions of the recommended ranking.