Um estudo sobre abordagens para avaliação out-of-sample de modelos de classificação de animais em imagens de armadilhas fotográficas

Abstract

Camera traps are a strategy for wildlife monitoring, which consists on using cameras with motion sensors that, when triggered, start recording short sequences of images or videos of animals without disturbing their natural behavior. These cameras capture millions of images, but the information extraction is traditionally performed by humans, which is an expensive and time-consuming manual task. Deep learning techniques are the state of the art for extracting information from images and have been applied in several works to perform animal species classification in camera trap images. Since these models have high representation capacity and can easily memorize the entire training set, overlapping of very similar images in training and test sets should be avoided, in order to correctly evaluate the models generalization capacity. However, the possible high similarity between camera trap images obtained at the same place in short periods of time has not received a great deal of attention in the literature. The random data splitting is the the most widely used strategy in works dealing with animal species classification in camera trap images. Nevertheless, this strategy may generate optimistic test sets when compared to the actual conditions of use, which may result in an overestimated assessment of the trained model and may lead to wrong decisions. Therefore, we conduct in this work a study related to dataset splitting approaches for camera trap datasets, in order to reduce the optimistic bias of the test sets. Real usage scenarios were simulated and evaluated to verify whether or not the test sets are able to show the generalization capacity of the models under these conditions. As a result, a set of recommendations for dataset splitting on out-of-sample evaluation of models was specified according to the protocol used by the camera trap projects.