Inferência de contexto para dispositivos móveis utilizando aprendizagem por reforço

Abstract

Advances in wireless communication and computer hardware technologies have boosted the popularity of mobile devices. Increasingly, these devices gain new features of hardware (i.e., sensors and other gadgets) and software (e.g., facial, voice and gestures recognition) so that the human-computer interaction can occur more naturally. These features allowed a greater awareness of the environment and the conditions under which the users are, enabling the development of applications ever more proactive and sensitive. A context aware system can modify its behavior according to the inferred context of the environment. However, erroneous interpretations of the collected data may induce inappropriate and unwanted actions in applications. Although there is variety of inference techniques in the literature (e.g., rules, ontologies, that uses supervised and unsupervised learning), generally, they do not consider whether the inferences were indeed suitable to the user contexts. Furthermore, most of these techniques uses static inference models (i.e., they are unable to adjust themselves to changes in the environment conditions), which represents a limitation of these techniques when applied to the field of mobile applications. This work proposes a new context reasoning technique for mobile applications – called CoRe-RL – which uses reinforcement learning in order that the produced inferences could be ever more suitable to the user’s contexts. In this technique, learning occurs in an incremental manner and as the user interacts with the system, allowing the inference to be adjusted by the rewards (positive reinforcements) and punishments (negative reinforcements) associated to the inferred contexts. As the contexts are continuously being learned, the proposed technique also allows a flexible context management to the applications, which enables new contexts (labels) to be registered and learned over time. The operation of the technique is divided into two stages – classification and adaptation. The CoRe-RL uses a modified version of the K nearest neighbors in the classification stage. The learning (adaptation) stage is based on examples, but also makes adjustments on the models (features ranking) which weigh the most relevant xv features of each context in the classification stage. In order to validate and evaluate the proposed technique, it was developed, as a case study of this work, an application that implements all of the functionality and capabilities of CoRe-RL. Through this application, practical experiments for evaluating the classification and adaptation were executed in two specific scenarios: there was a single context in the first scenario; and in the second, there were three. Through the practical experiments, it was observed that, in accordance to the cutting threshold used, it is possible to obtain good performances in the classification even with a small base and with a slightly adjusted ranking. Furthermore, it was demonstrated that the CoRe-RL improves its performance, converging to the optimal performance, in accordance to the occurrence of new interactions.