Machine learning algorithms and training of models
The Bee Tracker software uses a combination of three machine
learning algorithms to generate the above mentioned outputs: the Faster
R-CNN object detection pipeline (Ren et al., 2017), a YoloV3 (Redmon &
Farhadi, 2018) object detection network and a custom Keras image
classification network (Chollet, 2015). The software takes a video of a
nesting unit as describe above as input and as a first step detects all
marked bees and cavities in each video frame using two trained Faster
R-CNN networks. Subsequently, the marker tags (unique digit-color
combination; Fig. 1) are identified by a YoloV3 network on each
previously detected bee. Additionally, all identified markers are
further classified into the digits 1-8 by a custom Keras network.
Knowing the cavity positions and bee positions alongside with the bee ID
for each individual frame, a custom object tracking algorithm is applied
to this data in order to link the individual frames together and obtain
a movement path for each bee. By analyzing the start and end point of
each detected movement path, the software is able to detect cavity
entering and leaving events.
The software relies on the four previously mentioned trained machine
learning models. The model for detecting the bees was trained on 1303
individual images. The cavity detection model was trained on 120
individual images of nesting units, whereas each nesting unit contained
between 60 and 130 cavities. The color tag detection model was trained
on 4921 individual images of bees and the digit classification model was
trained with 10347 individual images of number tags. Additionally,
various data augmentation techniques were applied such as rotations,
random brightness adjustments, random contrast adjustments and random
saturation. Further detailed information about the model trainings are
provided in the software manual (see Supporting Information).