A LIGHTWEIGHT MULTI-PERSON POSE ESTIMATION SCHEME BASED ON JETSON NANO
Lei Liu
13605545615@163.comCollege of Computing and Information Technologies, National University Manila (Philippines)
https://orcid.org/0000-0002-1807-6906
Eric B. Blancaflor
Mapua University, School of Information Technology, Philippines (Philippines)
Mideth Abisado
National University, College of Computing and Information Technologies, Philippines (Philippines)
https://orcid.org/0000-0003-4215-7260
Abstract
As the basic technology of human action recognition, pose estimation is attracting more and more researchers' attention, while edge application scenarios pose a higher challenge. This paper proposes a lightweight multi-person pose estimation scheme to meet the needs of real-time human action recognition on the edge end. This scheme uses AlphaPose to extract human skeleton nodes, and adds ResNet and Dense Upsampling Revolution to improve its accuracy. Meanwhile, we use YOLO to enhance AlphaPose’s support for multi-person pose estimation, and optimize the proposed model with TensorRT. In addition, this paper sets Jetson Nano as the Edge AI deployment device of the proposed model and successfully realizes the model migration to the edge end. The experimental results show that the speed of the optimized object detection model can reach 20 FPS, and the optimized multi-person pose estimation model can reach 10 FPS. With the image resolution of 320×240, the model’s accuracy is 73.2%, which can meet the real-time requirements. In short, our scheme can provide a basis for lightweight multi-person action recognition scheme on the edge end.
Supporting Agencies
Keywords:
human pose estimation, lightweight model, Edge AI, deep learning, computer visionReferences
Akshatha, K. R., Karunakar, A. K., Shenoy, S. B., Pai, A. K., Nagaraj, N. H., & Rohatgi, S. S. (2022). Human detection in aerial thermal images using faster R-CNN and SSD algorithms. Electronics, 11(7), 1151. https://doi.org/10.3390/electronics11071151
DOI: https://doi.org/10.3390/electronics11071151
Google Scholar
Alnuaim, A. A., Zakariah, M., Hatamleh, W. A., Tarazi, H., Tripathi, V., & Amoatey, E. T. (2022). Humancomputer interaction with hand gesture recognition using ResNet and MobileNet. Computational
Google Scholar
Intelligence Neuroscience, 2022, 8777355. https://doi.org/10.1155/2022/8777355
DOI: https://doi.org/10.1155/2022/8777355
Google Scholar
Bertasius, G., Feichtenhofer, C., Tran, D., Shi, J., & Torresani, L. (2019). Learning temporal pose estimation from sparsely-labeled Videos. ArXiv, abs/1906.04016. https://doi.org/10.48550/arXiv.1906.04016
Google Scholar
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2016). Realtime multi-person 2D pose estimation using part affinity fields. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 (pp. 1302–1310). IEEE. https://doi.org/10.1109/CVPR.2017.143.
DOI: https://doi.org/10.1109/CVPR.2017.143
Google Scholar
Chen, W., Jiang, Z., Guo, H., & Ni, X. (2020). Fall Detection Based on Key Points of Human-Skeleton Using OpenPose. Symmetry, 12(5), 744. https://doi.org/10.3390/sym12050744
DOI: https://doi.org/10.3390/sym12050744
Google Scholar
Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., & Sun, J. (2018). Cascaded pyramid network for multi-person pose estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision Pattern Recognition (pp. 7103–7112). IEEE. https://doi.org/10.1109/CVPR.2018.00742
DOI: https://doi.org/10.1109/CVPR.2018.00742
Google Scholar
Chung, J.-L., Ong, L.-Y., & Leow, M. C. (2022). Comparative analysis of skeleton-based human pose estimation. Future Internet, 14(12), 380. https://doi.org/10.3390/fi14120380
DOI: https://doi.org/10.3390/fi14120380
Google Scholar
Dewangan, D. K., & Sahu, S. P. (2021). Deep learning-based speed bump detection model for intelligent vehicle system using raspberry pi. IEEE Sensors Journal, 21, 3570–3578. https://doi.org/10.1109/JSEN.2020.3027097
DOI: https://doi.org/10.1109/JSEN.2020.3027097
Google Scholar
Fang, H., Li, J., Tang, H., Xu, C., Zhu, H., Xiu, Y., Li, Y.-L., & Lu, C. (2022). AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time. ArXiv, abs/2211.03375. https://doi.org/10.48550/arXiv.2211.03375
DOI: https://doi.org/10.1109/TPAMI.2022.3222784
Google Scholar
Fang, H., Xie, S., Tai, Y.-W., & Lu, C. (2017). RMPE: Regional multi-person pose estimation. IEEE International Conference on Computer Vision (pp. 2353–2362). IEEE. https://doi.org/10.48550/arXiv.1612.00137
DOI: https://doi.org/10.1109/ICCV.2017.256
Google Scholar
Gamra, M. B., & Akhloufi, M. A. (2021). A review of deep learning techniques for 2D and 3D human pose estimation. Image Vis. Comput, 114, 104282. https://doi.org/10.1016/j.imavis.2021.104282
DOI: https://doi.org/10.1016/j.imavis.2021.104282
Google Scholar
Gautam, B. P., Noda, Y., Gautam, R., Sharma, H. P., Sato, K., & Neupane, S. B. (2020). Body part localization and pose tracking by using deepercut algorithm for king cobra's BBL (Biting Behavior Learning). International Conference on Networking Network Applications (pp. 422–429). IEEE. https://doi.org/10.1109/NaNA51271.2020.00078
DOI: https://doi.org/10.1109/NaNA51271.2020.00078
Google Scholar
Ge, Z., Liu, S., Wang, F., Li, Z., & Sun, J. (2021). YOLOX: Exceeding YOLO series in 2021. ArXiv, abs/2107.08430. https://doi.org/10.48550/arXiv.2107.08430
Google Scholar
Jegham, I., Khalifa, A. B., Alouani, I., & Mahjoub, M. A. (2020). Vision-based human action recognition: An overview and real world challenges. Forensic Science International: Digital Investigation, 32, 200901. https://doi.org/10.1016/j.fsidi.2019.200901
DOI: https://doi.org/10.1016/j.fsidi.2019.200901
Google Scholar
Jeong, E., Kim, J., & Ha, S. (2022). TensorRT-Based framework and optimization methodology for deep learning inference on jetson boards. ACM Transactions on Embedded Computing Systems, 21, 1–26. https://doi.org/10.1145/3508391
DOI: https://doi.org/10.1145/3508391
Google Scholar
Khirodkar, R., Chari, V., Agrawal, A., & Tyagi, A. (2021). Multi-Instance pose networks: rethinking top-down pose estimation. IEEE/CVF International Conference on Computer Vision (pp. 3102-3111). IEEE. https://doi.org/10.48550/arXiv.2101.11223
DOI: https://doi.org/10.1109/ICCV48922.2021.00311
Google Scholar
Kong, Y., & Fu, Y. (2022). Human action recognition and prediction: A survey. International Journal of Computer Vision, 130(5), 1366-1401. https://doi.org/10.48550/arXiv.1806.11230
DOI: https://doi.org/10.1007/s11263-022-01594-9
Google Scholar
Kreiss, S., Bertoni, L., & Alahi, A. (2021). OpenPifPaf: Composite fields for semantic keypoint detection and spatio-temporal association. IEEE Transactions on Intelligent Transportation Systems, 23, 13498–13511. https://doi.org/10.48550/arXiv.2103.02440
DOI: https://doi.org/10.1109/TITS.2021.3124981
Google Scholar
Liu, M.-J., Wan, L., Wang, B., & Wang, T.-L. (2023). SE-YOLOv4: shuffle expansion YOLOv4 for pedestrian detection based on PixelShuffle. Applied Intelligence, 2023. https://doi.org/10.1007/s10489-023-04456-0
DOI: https://doi.org/10.1007/s10489-023-04456-0
Google Scholar
Nguyen, S.-H., Le, T.-T.-H., Nguyen, H.-B., Phan, T.-T., Nguyen, C.-T., & Vu, H. (2022). Improving the Hand Pose Estimation from Egocentric Vision via HOPE-Net and Mask R-CNN. International Conference on Multimedia Analysis Pattern Recognition (pp. 1-6). IEEE. https://doi.org/10.1109/MAPR56351.2022.9924768
DOI: https://doi.org/10.1109/MAPR56351.2022.9924768
Google Scholar
Park, K., Jang, W., Lee, W., Nam, K., Seong, K., Chai, K., & Li, W.-S. (2020). Real-time mask detection on google edge TPU. ArXiv, abs/2010.04427. https://doi.org/10.48550/arXiv.2010.04427
Google Scholar
Pishchulin, L., Insafutdinov, E., Tang, S., Andres, B., Andriluka, M., Gehler, P., & Schiele, B. (2016). DeepCut: Joint subset partition and labeling for multi person pose estimation. Conference on Computer Vision Pattern Recognition (pp. 4929–4937). IEEE. https://doi.org/10.1109/CVPR.2016.533
DOI: https://doi.org/10.1109/CVPR.2016.533
Google Scholar
Sediqi, K. M., & Lee, H. J. (2021). A novel upsampling and context convolution for image semantic segmentation. Sensors, 21(6), 2170. https://doi.org/10.3390/s21062170
DOI: https://doi.org/10.3390/s21062170
Google Scholar
Shiraishi, Y. (2020). Latest trend of edge aI devices. Journal of The Japan Institute of Electronics Packaging, 23(2), 145-149. https://doi.org/10.5104/jiep.23.145
DOI: https://doi.org/10.5104/jiep.23.145
Google Scholar
Sipola, T., Alatalo, J., Kokkonen, T., & Rantonen, M. (2022). Artificial intelligence in the IoT Era: A Review of Edge AI Hardware and Software. 31st Conference of Open Innovations Association (pp. 320-331). IEEE. https://doi.org/10.23919/FRUCT54823.2022.9770931
DOI: https://doi.org/10.23919/FRUCT54823.2022.9770931
Google Scholar
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019). Deep high-resolution representation learning for human pose estimation. IEEE/CVF Conference on Computer Vision Pattern Recognition (pp. 5686–5696.) IEEE. https://doi.org/10.1109/CVPR.2019.00584.
DOI: https://doi.org/10.1109/CVPR.2019.00584
Google Scholar
Süzen, A. A., Duman, B., & Şen, B. (2020). Benchmark analysis of jetson TX2, jetson nano and raspberry PI using Deep-CNN. International Congress on Human-Computer Interaction, Optimization Robotic Applications (pp.1–5.) IEEE. https://doi.org/10.1109/HORA49412.2020.9152915
DOI: https://doi.org/10.1109/HORA49412.2020.9152915
Google Scholar
Tran, H. Y., Bui, T. M., Pham, T.-L., & Le, V.-H. (2022). An evaluation of 2D human pose estimation based on ResNet backbone. Journal of Engineering Research and Sciences, 1(2), 59–67. https://doi.org/10.55708/js0103007
DOI: https://doi.org/10.55708/js0103007
Google Scholar
Xiao, B., Wu, H., & Wei, Y. (2018). Simple baselines for human pose estimation and tracking. European Conference on Computer Vision. Lecture Notes in Computer Science (pp. 472–487). Springer. https://doi.org/10.1007/978-3-030-01231-1_29
DOI: https://doi.org/10.1007/978-3-030-01231-1_29
Google Scholar
Zhang, H.-B., Zhang, Y.-X., Zhong, B., Lei, Q., Yang, L., Du, J.-X., & Chen, D.-S. (2019). A comprehensive survey of vision-based human action recognition methods. Sensors, 19(5), 1005–1016. https://doi.org/10.3390/s19051005
DOI: https://doi.org/10.3390/s19051005
Google Scholar
Authors
Lei Liu13605545615@163.com
College of Computing and Information Technologies, National University Manila Philippines
https://orcid.org/0000-0002-1807-6906
Authors
Eric B. BlancaflorMapua University, School of Information Technology, Philippines Philippines
Authors
Mideth AbisadoNational University, College of Computing and Information Technologies, Philippines Philippines
https://orcid.org/0000-0003-4215-7260
Statistics
Abstract views: 612PDF downloads: 364
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
Similar Articles
- Łukasz SEMKŁO, Łukasz GIERZ, NUMERICAL AND EXPERIMENTAL ANALYSIS OF A CENTRIFUGAL PUMP WITH DIFFERENT ROTOR GEOMETRIES , Applied Computer Science: Vol. 18 No. 4 (2022)
- Lucian LUPŞA-TĂTARU, CUSTOMIZING AUDIO FADES WITH A VIEW TO REAL-TIME PROCESSING , Applied Computer Science: Vol. 15 No. 4 (2019)
- Konrad BIERCEWICZ, Mariusz BORAWSKI, Anna BORAWSKA, Jarosław DUDA, DETERMINING THE DEGREE OF PLAYER ENGAGEMENT IN A COMPUTER GAME WITH ELEMENTS OF A SOCIAL CAMPAIGN USING COGNITIVE NEUROSCIENCE TECHNIQUES , Applied Computer Science: Vol. 18 No. 4 (2022)
- Wojciech DANILCZUK, THE USE OF SIMULATION ENVIRONMENT FOR SOLVING THE ASSEMBLY LINE BALANCING PROBLEM , Applied Computer Science: Vol. 14 No. 1 (2018)
- Evans BAIDOO, FIREWORKS ALGORITHM FOR UNCONSTRAINED FUNCTION OPTIMIZATION PROBLEMS , Applied Computer Science: Vol. 13 No. 1 (2017)
You may also start an advanced similarity search for this article.
