AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

A Real-Time Multi-Stage Architecture for Pose Estimation of Zebrafish Head with Convolutional Neural Networks

School of Computer Science and Technology, University of Science and Technology of China, Hefei 230027, China
School of Data Science, University of Science and Technology of China, Hefei 230027, China
Anhui Province Key Laboratory of Software in Computing and Communication, Hefei 230027, China

Recommended by CAD/Graphcis 2019

Show Author Information

Abstract

In order to conduct optical neurophysiology experiments on a freely swimming zebrafish, it is essential to quantify the zebrafish head to determine exact lighting positions. To efficiently quantify a zebrafish head's behaviors with limited resources, we propose a real-time multi-stage architecture based on convolutional neural networks for pose estimation of the zebrafish head on CPUs. Each stage is implemented with a small neural network. Specifically, a light-weight object detector named Micro-YOLO is used to detect a coarse region of the zebrafish head in the first stage. In the second stage, a tiny bounding box refinement network is devised to produce a high-quality bounding box around the zebrafish head. Finally, a small pose estimation network named tiny-hourglass is designed to detect keypoints in the zebrafish head. The experimental results show that using Micro-YOLO combined with RegressNet to predict the zebrafish head region is not only more accurate but also much faster than Faster R-CNN which is the representative of two-stage detectors. Compared with DeepLabCut, a state-of-the-art method to estimate poses for user-defined body parts, our multi-stage architecture can achieve a higher accuracy, and runs 19x faster than it on CPUs.

Electronic Supplementary Material

Download File(s)
jcst-36-2-434-Highlights.pdf (560.2 KB)

References

[1]

Cong L, Wang Z, Chai Y, Han W, Shang C, Yang W, Bai L, Du J, Wang K, Wen Q. Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (Danio rerio). Elife, 2017, 6: Article No. e28158. https://doi.org/10.7554/elife.28158.

[2]

Xu Z P, Cheng X E. Zebrafish tracking using convolutional neural networks. Scientific Reports, 2017, 7: Article No. 42815. https://doi.org/10.1038/srep42815.

[3]

Mathis A, Mamidanna P, Cury K M, Abe T, Murthy V N, Mathis M W, Bethge M. DeepLabCut: Markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience, 2018, 21: 1281-1289. https://doi.org/10.1038/s41593-018-0209-y.

[4]
Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.580-587. https://doi.org/10.1109/CVPR.2014.81.
[5]
Girshick R. Fast R-CNN. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.1440-1448. https://doi.org/10.1109/ICCV.2015.169.
[6]
Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proc. the 29th Annual Conference on Neural Information Processing Systems, December 2015, pp.91-99.
[7]
Dai J, Li Y, He K, Sun J. R-FCN: Object detection via region-based fully convolutional networks. In Proc. the 30th Annual Conference on Neural Information Processing Systems, December 2016, pp.379-387.
[8]

Uijlings J R, van de Sande K E, Gevers T, Smeulders A W. Selective search for object recognition. International Journal of Computer Vision, 2013, 104(2): 154-171. https://doi.org/10.1007/s11263-013-0620-5.

[9]
Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.779-788. https://doi.org/10.1109/CVPR.2016.91.
[10]
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, Berg A C. SSD: Single shot multibox detector. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.21-37. https://doi.org/10.1007/978-3-319-46448-0_2.
[11]
Cai Z, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In Proc. the 2018 IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.6154-6162. https://doi.org/10.1109/CVPR.2018.00644.
[12]
Toshev A, Szegedy C. DeepPose: Human pose estimation via deep neural networks. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.1653-1660. https://doi.org/10.1109/CVPR.2014.214.
[13]
Pfister T, Simonyan K, Charles J, Zisserman A. Deep convolutional neural networks for efficient pose estimation in gesture videos. In Proc. the 12th Asian Conference on Computer Vision, November 2014, pp.538-552. https://doi.org/10.1007/978-3-319-16865-4_35.
[14]
Carreira J, Agrawal P, Fragkiadaki K, Malik J. Human pose estimation with iterative error feedback. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.4733-4742. https://doi.org/10.1109/CVPR.2016.512.
[15]
Pfister T, Charles J, Zisserman A. Flowing ConvNets for human pose estimation in videos. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.1913-1921. https://doi.org/10.1109/ICCV.2015.222.
[16]
Wei S E, Ramakrishna V, Kanade T, Sheikh Y. Convolutional pose machines. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.4724-4732. https://doi.org/10.1109/CVPR.2016.511.
[17]
Newell A, Yang K, Deng J. Stacked hourglass networks for human pose estimation. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.483-499. https://doi.org/10.1007/978-3-319-46484-8_29.
[18]
Pishchulin L, Insafutdinov E, Tang S, Andres B, Andriluka M, Gehler P V, Schiele B. DeepCut: Joint subset partition and labeling for multi person pose estimation. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.4929-4937. https://doi.org/10.1109/CVPR.2016.533.
[19]
Insafutdinov E, Pishchulin L, Andres B, Andriluka M, Schiele B. Deepercut: A deeper, stronger, and faster multi-person pose estimation model. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.34-50. https://doi.org/10.1007/978-3-319-46466-4_3.
[20]
Cao Z, Simon T, Wei S E, Sheikh Y. Realtime multi-person 2D pose estimation using part affinity fields. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 2017, pp.1302-1310. https://doi.org/10.1109/CVPR.2017.143.
[21]
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778. https://doi.org/10.1109/CVPR.2016.90.
[22]

Li S, Fang Z, Song W, Hao A, Qin H. Bidirectional optimization coupled lightweight networks for efficient and robust multi-person 2D pose estimation. Journal of Computer Science and Technology, 2019, 34(3): 522-536. https://doi.org/10.1007/s11390-019-1924-x.

Journal of Computer Science and Technology
Pages 434-444
Cite this article:
Huang Z-J, He X-X, Wang F-J, et al. A Real-Time Multi-Stage Architecture for Pose Estimation of Zebrafish Head with Convolutional Neural Networks. Journal of Computer Science and Technology, 2021, 36(2): 434-444. https://doi.org/10.1007/s11390-021-9599-5

324

Views

2

Crossref

1

Web of Science

2

Scopus

0

CSCD

Altmetrics

Received: 30 March 2019
Accepted: 03 June 2020
Published: 05 March 2021
©Institute of Computing Technology, Chinese Academy of Sciences 2021
Return