Machine learning algorithms and training of models
The Bee Tracker software uses a combination of three machine learning algorithms to generate the above mentioned outputs: the Faster R-CNN object detection pipeline (Ren et al., 2017), a YoloV3 (Redmon & Farhadi, 2018) object detection network and a custom Keras image classification network (Chollet, 2015). The software takes a video of a nesting unit as describe above as input and as a first step detects all marked bees and cavities in each video frame using two trained Faster R-CNN networks. Subsequently, the marker tags (unique digit-color combination; Fig. 1) are identified by a YoloV3 network on each previously detected bee. Additionally, all identified markers are further classified into the digits 1-8 by a custom Keras network. Knowing the cavity positions and bee positions alongside with the bee ID for each individual frame, a custom object tracking algorithm is applied to this data in order to link the individual frames together and obtain a movement path for each bee. By analyzing the start and end point of each detected movement path, the software is able to detect cavity entering and leaving events.
The software relies on the four previously mentioned trained machine learning models. The model for detecting the bees was trained on 1303 individual images. The cavity detection model was trained on 120 individual images of nesting units, whereas each nesting unit contained between 60 and 130 cavities. The color tag detection model was trained on 4921 individual images of bees and the digit classification model was trained with 10347 individual images of number tags. Additionally, various data augmentation techniques were applied such as rotations, random brightness adjustments, random contrast adjustments and random saturation. Further detailed information about the model trainings are provided in the software manual (see Supporting Information).