Automatic detection of sow estrus using a lightweight real-time detector and thermal images

Haibo Zheng, Hang Zhang, Shuang Song, Yue Wang, Tonghai Liu

Abstract


Determination of ovulation time is one of the most important tasks in sow reproduction management. Temperature variation in the vulva of the sows can be used as a predictor of ovulation time. However, the skin temperatures of sows in existing studies are obtained manually from infrared thermal images, posing an obstacle to the automatic prediction of ovulation time. In this study, an improved YOLO-V5s detector based on feature fusion and dilated convolution (FD-YOLOV5s) was proposed for the automatic extraction of the vulva temperature of sows based on infrared thermal images. For the purpose of reducing the model complexity, the depthwise separable convolution and the modified lightweight ShuffleNet-V2 module were introduced in the backbone. Meanwhile, the feature fusion network structure of the model was simplified for efficiency, and a mixed dilated convolutional module was designed to obtain global features. The experimental results show that FD-YOLOV5s outperformed the other nine methods, with a mean average precision (mAP) of 99.1%, an average frame rate of 156.25 fps, and a model size of only 3.86 MB, indicating that the method effectively simplifies the model while ensuring detection accuracy. Using a linear regression between manual extraction and the results extracted using this method in randomly selected thermal images, the coefficients of determination for maximum and average vulvar temperatures reached 99.5% and 99.3%, respectively. The continuous vulva temperature of sows was obtained by the target detection algorithm, and the sow estrus detection was performed by the temperature trend and compared with the manually detected estrus results. The results showed that the sensitivity, specificity, and error rate of the estrus detection algorithm were 89.3%, 94.5%, and 5.8%, respectively. The method achieves real-time and accurate extraction of sow vulva temperature and can be used for the automatic detection of sow estrus, which could be helpful for the automatic prediction of ovulation time.
Keywords: automatic estrus detection, thermal images, real-time detector, vulva temperature, mixed dilated convolutional
DOI: 10.25165/j.ijabe.20231603.7711

Citation: Zheng H B, Zhang H, Song S, Wang Y, Liu T H. Automatic detection of sow estrus using a lightweight real-time detector and thermal images. Int J Agric & Biol Eng, 2023; 16(): 194–207.

Keywords


automatic estrus detection, thermal images, real-time detector, vulva temperature, mixed dilated convolutional

Full Text:

PDF

References


Lee J H, Lee D H, Yun W, Oh H J, An J S, Kim Y G, et al. Quantifiable and feasible estrus detection using the ultrasonic sensor array and digital infrared thermography. Journal of Animal Science and Technology, 2019; 61(3): 163-169. doi: 10.5187/jast.2019.61.3.163.

Johnson J S, Shade K A. Characterizing body temperature and activity changes at the onset of estrus in replacement gilts. Livestock Science, 2017; 199: 22-24. doi: 10.1016/j.livsci.2017.03.004.

Weng R C. Variations in the body surface temperature of sows during the post weaning period and its relation to subsequent reproductive performance. Asian-Australas Journal of Animimal Sciences (AJAS), 2020; 33(7): 1138-1147. doi: 10.5713/ajas.19.0576.

Knox R V. Artificial insemination in pigs today. Theriogenology, 2016; 85(1): 83-93. doi: 10.1016/j.theriogenology.2015.07.009.

Sandu M, Mantea Ș, Ipate I, Kruzslicika M, Chiriţescu V. Study upon the moment of ovulation in sows to establish the optimum moment for semen inoculation. Anim Sci Biotechnol, 2012; 45: 346-348.

Cassar G, Kirkwood R N, Poljak Z, Bennett-Steward K, Friendship R M. Effect of single or double insemination on fertility of sows bred at an induced estrus and ovulation. J Swine Health Prod, 2005, 13(5): 254-258.

Kemp B, Soede N M. Consequences of variation in interval from insemination to ovulation on fertilization in pigs. Journal of Reproduction and Fertility Supplement, 1997; 52(Supp.): 79-89.

Knox R V, Esparza-Harris K C, Johnston M E, Webel S K. Effect of numbers of sperm and timing of a single, post-cervical insemination on the fertility of weaned sows treated with OvuGel®. Theriogenology, 2017; 92: 197-203. doi: 10.1016/j.theriogenology.2017.01.033.

Terqui M, Guillouet P, Maurel M C, Martinat-Botté F. Relationship between peri-oestrus progesterone levels and time of ovulation by echography in pigs and influence of the interval between ovulation and artificial insemination (AI) on litter size. Reproduction Nutrition Development, 2000; 40(4): 393-404. doi: 10.1051/rnd:2000107.

Soede N M, Langendijk P, Kemp B. Reproductive cycles in pigs. Animal Reproduction Science, 2011; 124(3-4): 251-258. doi: 10.1016/j.anireprosci.2011.02.025.

Ostersen T, Cornou C, Kristensen A R. Detecting oestrus by monitoring sows’ visits to a boar. Computers and Electronics in Agriculture, 2010; 74(1): 51-58. doi: 10.1016/j.compag.2010.06.003.

Lei K D, Zong C, Du X D, Teng G H, Feng F Q. Oestrus analysis of sows based on bionic boars and machine vision technology. Animals, 2021; 11(6): 1485. doi: 10.3390/ani11061485.

Langendijk P, Soede N M, Bouwman E G, Kemp B. Responsiveness to boar stimuli and change in vulvar reddening in relation to ovulation in weaned sows. Journal of Animal Science, 2000; 78(12): 3019-3026. doi: 10.2527/2000.78123019x.

Pedersen L J. Sexual behaviour in female pigs. Hormones and Behavior, 2007; 52(1): 64-69. doi: 10.1016/j.yhbeh.2007.03.019.

Pedersen L J, Rojkittikhun T, Einarsson S, Edqvist L E. Postweaning grouped sows: Effects of aggression on hormonal patterns and oestrous behaviour. Applied Animal Behaviour Science, 1993; 38(1): 25-39. doi: 10.1016/0168-1591(93)90039-R.

Kauffold J, Rautenberg T, Richter A, Waehner M, Sobiraj A. Ultrasonographic characterization of the ovaries and the uterus in prepubertal and pubertal gilts. Theriogenology, 2004; 61(9): 1635-1648. doi: 10.1016/j.theriogenology.2003.09.012.

Rezác P, Vasícková D, Pöschl M. Changes of electrical impedance in vaginal vestibule in cyclic sows. Animal Reproduction Science, 2003; 79(1-2): 111-119. doi: 10.1016/s0378-4320(03)00101-5.

Luño V, Gil L, Jerez R A, Malo C, González N, Grandía J, et al. Determination of ovulation time in sows based on skin temperature and genital electrical resistance changes. Veterinary Record, 2013; 172(22): 579. doi: 10.1136/vr.101221.

Soede N M, Hazeleger W, Kemp B. Follicle size and the process of ovulation in sows as studied with ultrasound. Reproduction in Domestic Animals, 1998; 33(3-4): 239-244. doi: 10.1111/j.1439-0531.1998.tb01350.x.

Cornou C. Automated oestrus detection methods in group housed sows: Review of the current methods and perspectives for development. Livestock Science, 2006; 105(1-3): 1-11. doi: 10.1016/j.livsci.2006.05.023.

Hidalgo D M, Cassar G, Manjarin R, Dominguez J C, Friendship R M, Kirkwood R N. Relationship between vaginal mucus conductivity and time of ovulation in weaned sows. Canadian Journal of Veterinary Research, 2015; 79(2): 151-154.

Sykes D J, Couvillion J S, Cromiak A, Bowers S, Schenck E, Crenshaw M, et al. The use of digital infrared thermal imaging to detect estrus in gilts. Theriogenology, 2012; 78(1): 147-152. doi: 10.1016/j.theriogenology.2012.01.030.

Scolari S C, Clark S G, Knox R V, Tamassia M A. Vulvar skin temperature changes significantly during estrus in swine as determined by digital infrared thermography. Journal of Swine Health and Production, 2011; 19(3): 151-155.

Simões V G, Lyazrhi F, Picard-Hagen N, Gayrard V, Martineau G P, Waret-Szkuta A. Variations in the vulvar temperature of sows during proestrus and estrus as determined by infrared thermography and its relation to ovulation. Theriogenology, 2014; 82(8): 1080-1085. doi: 10.1016/j.theriogenology.2014.07.017.

Weng R C, Ndwandwe S B. Application of modern estrus detection protocols in small scale Hybrid Black pig production systems. Journal of Agricultural and Crop Research, 2020; 8(6): 120-131. doi: 10.33495/jacr_v8i6.20.154.

Luño V, Gil L, Olaciregui M, Grandía J, Ansó T, De Blas I. Fertilisation rate obtained with frozen-thawed boar semen supplemented with rosmarinic acid using a single insemination timed according to vulvar skin temperature changes. Acta Veterinaria Hungarica, 2015; 63(1): 100-109. doi: 10.1556/AVet.2015.008.

Lu M, He J, Chen C, Okinda, C, Shen M, Liu L, et al. An automatic ear base temperature extraction method for top view piglet thermal image. Comput Electron Agric, 2018; 155: 339-347. doi: 10.1016/j.compag.2018.10.030.

Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017; 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031.

Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016; pp.779-788. doi: 10.1109/CVPR.2016.91.

Zhou C, Lin K, Xu D M, Liu J T, Zhang S, Sun C H, et al. Method for segmentation of overlapping fish images in aquaculture. Int J Agric & Biol Eng, 2019; 12(6): 135-142. doi: 10.25165/j.ijabe.20191206.3217.

Yang Q M, Xiao D Q, Cai J H. Pig mounting behaviour recognition based on video spatial-temporal features. Biosystems Engineering, 2021; 206: 55-66. doi: 10.1016/j.biosystemseng.2021.03.011.

Li G X, Liu X L, Ma Y F, Wang B B, Zheng L H, Wang M J. Body size measurement and live body weight estimation for pigs based on back surface point clouds. Biosysems Engineering, 2022; 218: 10-22. doi: 10.1016/j.biosystemseng.2022.03.014.

Zhang X D, Kang X, Feng N N, Liu G. Automatic recognition of dairy cow mastitis from thermal images by a deep learning detector. Computers and Electronics in Agriculture, 2020; 178: 105754. doi: 10.1016/j.compag.2020.105754.

Zhang Y, Cai J, Xiao D, Li Z, Xiong B. Real-time sow behavior detection based on deep learning. Computers and Electronics in Agriculture, 2019; 163: 104884. doi: 10.1016/j.compag.2019.104884.

Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, et al. SSD: Single Shot Multibox Detector. In: Proceedings of the European Cofference on Computer Vision (2016ECCV), Amsterdam: Springer, Cham, 2016; pp.21-37. doi: 10.1007/978-3-319-46448-0_2.

Redmon J, Farhadi A. Yolov3: An incremental improvement. arXiv preprint, 2018. arXiv: 1804.02767. doi: 10.48550/arXiv.1804.02767.

Jocher G, Stoken A, Borovec J, NanoCode012, Chaurasia A, TaoXie, et al. Ultralytics/yolov5: v5.0 - YOLOv5-P6 1280 models, AWS, Supervisely and YouTube integrations (v5.0). Zenodo, 2021. Available: https://zenodo.org/record/4679653. Accessed on [2021-6-18].

Ma N, Zhang X, Zheng H T, Sun J. Shufflenet v2: Practical guidelines for efficient CNN architecture design. In: Proceedings of the European Confference on Computer Vision (2018ECCV), Munich, Germany: Springer, 2018; pp.122-138. doi: 10.1007/978-3-030-01264-9_8.

Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint, 2017. arXiv: 1704.04861. doi: 10.48550/arXiv.1704.04861.

Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. arXiv preprint, 2015. arXiv: 1511.07122. doi: 10.48550/arXiv.1511.07122.

Siewert C, Dänicke S, Kersten S, Brosig B, Rohweder D, Beyerbach M, et al. Difference method for analysing infrared images in pigs with elevated body temperatures. Zeitschrift für Medizinische Physik, 2014; 24(1): 6-15. doi: 10.1016/j.zemedi.2013.11.001.

Liu H, Shen H, Ci W B, Qi Z, Zhou C, Yao J X, et al. Effects of the distance and test angle on the precision of infrared temperature measurement. In: IOP Conference Series: Earth and Environmental Science, Bristol: IOP Publishing, 2022; 983(1): 012025. doi: 10.1088/1755-1315/983/1/012025.

Bochkovskiy A, Wang C Y, Liao H Y M. Yolov4: Optimal speed and accuracy of object detection. arXiv: 2004.10934, 2020. doi: 10.48550/arXiv.2004.10934.

Lin T-Y, Dollar P, Girshick R, He K M, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA: IEEE, 2017; pp.936-944. doi: 10.1109/CVPR.2017.106.

Liu S, Qi L, Qin H F, Shi J P, Jia J Y. Path aggregation network for instance segmentation. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA: IEEE, 2018; pp.8759-8768. doi: 10.1109/CVPR.2018.00913.

Lu Z, Bai Y, Chen Y, Su C, Lu C, Zhan T, et al. The classification of gliomas based on a pyramid dilated convolution resnet model. Pattern Recogn Letters, 2020; 133: 173-179. doi: 10.1016/j.patrec.2020.03.007.

Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016; pp.2818-2826. doi: 10.1109/CVPR.2016.308.

Zheng Z H, Wang P, Liu W, Li J Z, Ye R G, Ren D W. Distance-IoU loss: Faster and better learning for bounding box regression. In: Proceedings of the AAAI Conference on Artificial Intelligence, Palo Alto, California, USA: AAAI Press, 2020; 34(7): 12993-13000. doi: 10.1609/aaai.v34i07.6999.

Almeida F R, Novak S, Foxcroft G R. The time of ovulation in relation to estrus duration in gilts. Theriogenology, 2000; 53(7): 1389-1396. doi: 10.1016/S0093-691X(00)00281-8.

Satopaa V, Albrecht J, Irwin D, Raghavan B. Finding a" kneedle" in a haystack: Detecting knee points in system behavior. In: 2011 31st international conference on distributed computing systems workshops, Minneapolis, MN, USA: IEEE, 2011; pp.166-171. doi: 10.1109/ICDCSW.2011.20.

Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint, 2014. arXiv: 1409.1556. doi: 10.48550/arXiv.1409.1556.

Sandler M, Howard A, Zhu M L, Zhmoginov A, Chen L C. Mobilenetv2: Inverted residuals and linear bottlenecks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA: IEEE, 2018; pp.4510-4520. doi: 10.1109/CVPR.2018.00474.

He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016; pp.770-778. doi: 10.1109/CVPR.2016.90.

Lin T Y, Goyal P, Girshick R, He K, Dollar P. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020; 42(2): 318-327. doi: 10.1109/TPAMI.2018.2858826.

Ge Z, Liu S, Wang F, Li Z, Sun J. YOLOX: Exceeding YOLO series in 2021. arXiv preprint, 2021. arXiv: 2107.08430. doi: 10.48550/arXiv.2107.08430.




Copyright (c) 2023 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office