AnnotationUI: padrões de interface para sistemas de rotulagem de texto

Abstract

Due to the large volume of data coming from various applications, Machine Learning (ML) has been explored for use in systems from different human knowledge domains. For example, systems that use large volume of textual data, such as ChatGPT, may re- quire a previous training step known as data labeling. Commonly, users who are experts in the data domain annotate data to generate a training basis for a supervised ML model. However, the task of labeling is exhausting for users. Therefore, software designers must design system interfaces considering User Experience (UX) aspects. Some research focuses on supporting the development of data labeling systems, however there is still a gap on how to design such interfaces considering the type of data to be labeled. In this sense, this research focuses on text labeling systems, addressing the problems of Natural Language Processing (NLP), which is an ML area that addresses textual data. From the context we mentioned above, the following research question (QP) guides this work: How to support the interface design of text labeling systems? To answer this question, this research was conducted based on Action Research. From an industry demand, we conducted empirical studies and a state-of-the-practice analysis, which resulted in the production of the final artifact: the AnnotationUI patterns. After evaluation with experts, the results show that the AnnotationUI patterns support the interface design of textual type data labeling systems.