Compressão de modelos de reconhecimento de atividades humanas usando destilação de conhecimento

Abstract

The use of mobile and wearable devices can enable continuous monitoring of activities performed by the user. However, this process is challenging due to the complex nature of data devices by available devices. Recently, the use of deep neural networks has pushed the limits to recognize human activities with high accuracy. However, in the wearable mobile context, hardware restriction can make the use of deep neural networks unfeasible, as computational resources are limited. To mitigate as related to the computational cost of deep neural networks, this project work of a method called KD-HAR (Knowledge Distillation for Human Activity Recognition) for deep neural network compression based on the knowledge distillation technique applied to models of human activity recognition using data from inertial sensors. The knowledge acquired by teacher models, through hyperparameter optimization techniques, are transferred to student models with less complexity. One of the advantages of the proposed method is the ability to automatically extract features, low computational cost and the approximation of precision in the classification of activities when compared to more complex networks. To evaluate the compression capacity of the proposed method, this work uses two databases (UCI-HAR and WISDM) of smartphone inertial sensors. The results obtained show that the method can maintain competitive accuracy with compression rates ranging from 18 to 42 times the number of parameters of the distilled deep neural network in relation to the trained teacher model.