VIOLENCE PREDICTION IN SURVEILLANCE VIDEOS
Esraa Alaa MAHAREEK
esraa.mahareek@azhar.edu.egAl-Azhar University (Egypt)
https://orcid.org/0000-0002-9042-248X
Doaa Rizk FATHY
Al-Azhar University (Egypt)
https://orcid.org/0000-0002-5625-0282
Eman Karm ELSAYED
School of Computer science in Canadian International College CIC (Egypt)
https://orcid.org/0000-0001-7870-927X
Nahed ELDESOUKY
Al-Azhar University (Egypt)
https://orcid.org/0009-0008-4547-3051
Kamal Abdelraouf ELDAHSHAN
Al-Azhar University (Egypt)
https://orcid.org/0000-0002-9953-5480
Abstract
Forecasting violence has become a critical obstacle in the field of video monitoring to guarantee public safety. Lately, YOLO (You Only Look Once) has become a popular and effective method for detecting weapons. However, identifying and forecasting violence remains a challenging endeavor. Additionally, the classification results had to be enhanced with semantic information. This study suggests a method for forecasting violent incidents by utilizing Yolov9 and ontology. The authors employed Yolov9 to identify and categorize weapons and individuals carrying them. Ontology is utilized for semantic prediction to assist in predicting violence. Semantic prediction happens through the application of a SPARQL query to the identified frame label. The authors developed a Threat Events Ontology (TEO) to gain semantic significance. The system was tested with a fresh dataset obtained from a variety of security cameras and websites. The VP Dataset comprises 8739 images categorized into 9 classes. The authors examined the outcomes of using Yolov9 in conjunction with ontology in comparison to using Yolov9 alone. The findings show that by combining Yolov9 with ontology, the violence prediction system's semantics and dependability are enhanced. The suggested system achieved a mean Average Precision (mAP) of 83.7 %, 88% for precision, and 76.4% for recall. However, the mAP of Yolov9 without TEO ontology achieved a score of 80.4%. It suggests that this method has a lot of potential for enhancing public safety. The authors finished all training and testing processes on Google Colab's GPU. That reduced the average duration by approximately 90.9%. The result of this work is a next level of object detectors that utilize ontology to improve the semantic significance for real-time end-to-end object detection.
Keywords:
Violence prediction system, YOLO v8, Ontology, Surveillance cameras, Anomaly predictionReferences
Arslan, A. N., Hempelmann, C. F., Attardo, S., Blount, G. P., & Sirakov, N. M. (2015). Threat assessment using visual hierarchy and conceptual firearms ontology. Optical Engineering, 54(5), 053109. https://doi.org/10.1117/1.oe.54.5.053109
Google Scholar
Arslan, A. N., Sirakov, N. M., & Attardo, S. (2012). Weapon ontology annotation using boundary describing sequences. 2012 IEEE Southwest Symposium on Image Analysis and Interpretation (pp. 101-104). https://doi.org/10.1109/SSIAI.2012.6202463
Google Scholar
Ashraf, A. H., Imran, M., Qahtani, A. M., Alsufyani, A., Almutiry, O., Mahmood, A., Attique, M., & Habib, M. (2022). Weapons detection for security and video surveillance using CNN and YOLO-V5s. Computers, Materials and Continua, 70(2), 2761–2775. https://doi.org/10.32604/cmc.2022.018785
Google Scholar
Benjumea, A., Teeti, I., Cuzzolin, F., & Bradley, A. (2021). YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles. ArXiv, abs/2112.11798. https://doi.org/10.48550/arXiv.2112.11798
Google Scholar
Bisong, E. (2019). Building Machine Learning and Deep Learning models on Google Cloud platform: A Comprehensive Guide for Beginners. Apress Berkeley.
Google Scholar
Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. ArXiv, abs/2004.10934. https://doi.org/10.48550/arXiv.2004.10934
Google Scholar
Dugyala, R., Vishnu Vardhan Reddy, M., Tharun Reddy, C., & Vijendar, G. (2023). Weapon detection in surveillance videos using YOLOV8 and PELSF-DCNN. 4th International Conference on Design and Manufacturing Aspects for Sustainable Energy (ICMED-ICMPC 2023) (pp. 01071). E3S Web of Conferences. https://doi.org/10.1051/e3sconf/202339101071
Google Scholar
Elsayed, E. K., & Fathy, D. R. (2020a). Semantic Deep Learning to translate dynamic sign language. International Journal of Intelligent Engineering and Systems, 14(1), 316-325. https://doi.org/10.22266/IJIES2021.0228.30
Google Scholar
Elsayed, E. K., & Fathy, D. R. (2020b). Sign language semantic translation system using ontology and Deep Learning. International Journal of Advanced Computer Science and Applications, 11(1), 141-147. https://doi.org/10.14569/IJACSA.2020.0110118
Google Scholar
Glenn, J. (2022, November 22). Yolov5 release v7.0. https://github.com/ultralytics/yolov5/tree/v7.0
Google Scholar
Han, J., Liu, Y., Li, Z., Liu, Y., & Zhan, B. (2023). Safety helmet detection based on YOLOv5 driven by super-resolution reconstruction. Sensors, 23(4), 1822. https://doi.org/10.3390/s23041822
Google Scholar
Khalid, S., Waqar, A., Ain Tahir, H. U., Edo, O. C., & Tenebe, I. T. (2023). Weapon detection system for surveillance and security. 2023 International Conference on IT Innovation and Knowledge Discovery (ITIKD 2023) (pp. 1-7). IEEE. https://doi.org/10.1109/ITIKD56332.2023.10099733
Google Scholar
Lai, J., & Maples, S. (2017). Developing a real-time gun detection classifier. Stanford University.
Google Scholar
Li, C., Li, L., Geng, Y., Jiang, H., Cheng, M., Zhang, B., Ke, Z., Xu, X., & Chu, X. (2023). YOLOv6 v3.0: A full-scale reloading. ArXiv, abs/2301.05586. https://doi.org/10.48550/arXiv.2301.05586
Google Scholar
Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., Nie, W., Li, Y., Zhang, B., Liang, Y., Zhou, L., Xu, X., Chu, X., Wei, X., & Wei, X. (2022). YOLOv6: A single-stage object detection framework for industrial applications. ArXiv, abs/2209.02976. https://doi.org/10.48550/arXiv.2209.02976
Google Scholar
Li, X., Wang, W., Wu, L., Chen, S., Hu, X., Li, J., Tang, J., & Yang, J. (2020). Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. ArXiv, abs/2006.04388. https://doi.org/10.48550/arXiv.2006.04388
Google Scholar
Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017). Focal loss for dense object detection. 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 2999–3007). IEEE. https://doi.org/10.1109/ICCV.2017.324
Google Scholar
Lou, H., Duan, X., Guo, J., Liu, H., Gu, J., Bi, L., & Chen, H. (2023). DC-YOLOv8: Small-size object detection algorithm based on camera sensor. Electronics, 12(10), 2323. https://doi.org/10.3390/electronics12102323
Google Scholar
Mahareek, E. A. (2024). VP Dataset. https://Universe.Roboflow.Com/al-Azhar-Unversity/Violence-Prediction-in-Surveillance-Videos.
Google Scholar
Mahareek, E. A., Elsayed, E. K., Eldesouky, N. M., & Eldahshan, K. A. (2024). Detecting anomalies in security cameras with 3D-convolutional neural network and convolutional long short-term memory. International Journal of Electrical and Computer Engineering, 14(1), 993–1004. https://doi.org/10.11591/ijece.v14i1.pp993-1004
Google Scholar
Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, faster, stronger. Proceedings. 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), (pp. 6517-6525). IEEE. https://doi.org/10.1109/CVPR.2017.690
Google Scholar
Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement. ArXiv, abs/1804.02767. https://doi.org/10.48550/arXiv.1804.02767
Google Scholar
Redmon, J. (2016). Darknet: Open source neural networks in c. http://pjreddie.com/darknet/
Google Scholar
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 779-788). IEEE. https://doi.org/10.1109/CVPR.2016.91
Google Scholar
Reis, D., Kupec, J., Hong, J., & Daoudi, A. (2023). Real-Time flying object detection with YOLOv8. ArXiv, abs/2305.09972. https://doi.org/10.48550/arXiv.2305.09972
Google Scholar
Solawetz, J. F. (2023, January 11). What is YOLOv8? The Ultimate Guide. https://blog.roboflow.com/whats-new-in-yolov8/
Google Scholar
Songire, S. B., Chandrakant Patkar, U., Chate, P. J., Patil, M. A., Wani, L. K., Pathak, A. S., Bhardwaj Shrivas, S., & Patil, U. (2023). Using Yolo V7 development of complete vids solution based on latest requirements to provide highway traffic and incident real time info to the atms control room using Artificial Intelligence. Journal of Survey in Fisheries Sciences, 10(4S), 3444-3456.
Google Scholar
Tian, Z., Shen, C., Chen, H., & He, T. (2022). FCOS: A simple and strong anchor-free object detector. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4), 1922–1933. https://doi.org/10.1109/TPAMI.2020.3032166
Google Scholar
Verma, R., & Jayant, S. (2022). Cyber crime prediction using Machine Learning. In M. Singh, V. Tyagi, P. K. Gupta, J. Flusser, & T. Ören (Eds.), Advances in Computing and Data Sciences (Vol. 1614, pp. 160–172). Springer International Publishing. https://doi.org/10.1007/978-3-031-12641-3_14
Google Scholar
Wang, C. Y., Mark Liao, H. Y., Wu, Y. H., Chen, P. Y., Hsieh, J. W., & Yeh, I. H. (2020). CSPNet: A new backbone that can enhance learning capability of CNN. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (pp. 1571-1580). IEEE. https://doi.org/10.1109/CVPRW50498.2020.00203
Google Scholar
Wang, C., He, W., Nie, Y., Guo, J., Liu, C., Han, K., & Wang, Y. (2023). Gold-YOLO: Efficient object detector via Gather-and-Distribute mechanism. ArXiv, abs/2309.11331. https://doi.org/10.48550/arXiv.2309.11331
Google Scholar
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. ArXiv, abs/2207.02696. https://doi.org/10.48550/arXiv.2207.02696
Google Scholar
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2023). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 7464–7475). IEEE. https://doi.org/10.1109/cvpr52729.2023.00721
Google Scholar
Wang, C.-Y., Yeh, I.-H., & Liao, H.-Y. M. (2024). YOLOv9: Learning what you want to learn using programmable gradient information. ArXiv, abs/2402.13616. https://doi.org/10.48550/arXiv.2402.13616
Google Scholar
Zhang, X., Fang, S., Shen, Y., Yuan, X., & Lu, Z. (2024). Hierarchical velocity optimization for connected automated vehicles with cellular vehicle-to-everything communication at continuous signalized intersections. IEEE Transactions on Intelligent Transportation Systems, 25(3), 2944–2955. https://doi.org/10.1109/TITS.2023.3274580
Google Scholar
Zhang, S., Chi, C., Yao, Y., Lei, Z., & Li, S. Z. (2019). Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. ArXiv, abs/1912.02424. https://doi.org/10.48550/arXiv.1912.02424
Google Scholar
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2020). Distance-IoU loss: Faster and better learning for bounding box regression. 34th AAAI Conference on Artificial Intelligence (AAAI 2020) (pp. 12993-13000). https://doi.org/10.1609/aaai.v34i07.6999
Google Scholar
Authors
Esraa Alaa MAHAREEKesraa.mahareek@azhar.edu.eg
Al-Azhar University Egypt
https://orcid.org/0000-0002-9042-248X
Authors
Eman Karm ELSAYEDSchool of Computer science in Canadian International College CIC Egypt
https://orcid.org/0000-0001-7870-927X
Statistics
Abstract views: 205PDF downloads: 59
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
Similar Articles
- Katarzyna BARAN, APPLICATION OF THERMAL IMAGING CAMERAS FOR SMARTPHONE: SEEK THERMAL COMPACT PRO AND FLIR ONE PRO FOR HUMAN STRESS DETECTION – COMPARISON AND STUDY , Applied Computer Science: Vol. 20 No. 1 (2024)
- Rowell HERNANDEZ, Robert ATIENZA, CAREER TRACK PREDICTION USING DEEP LEARNING MODEL BASED ON DISCRETE SERIES OF QUANTITATIVE CLASSIFICATION , Applied Computer Science: Vol. 17 No. 4 (2021)
- Saha RENO, Sheikh Surfuddin Reza Ali CHOWDHURY, Iqramuzzaman SADI, MITIGATING LOAN ASSOCIATED FINANCIAL RISK USING BLOCKCHAIN BASED LENDING SYSTEM , Applied Computer Science: Vol. 17 No. 2 (2021)
- Shadan Mohammed Jihad ABDALWAHID, Raghad Zuhair YOUSIF, Shahab Wahhab KAREEM, ENHANCING APPROACH USING HYBRID PAILLER AND RSA FOR INFORMATION SECURITY IN BIGDATA , Applied Computer Science: Vol. 15 No. 4 (2019)
- Mohanad ABDULHAMID, Otieno ODONDI, Muaayed AL-RAWI, COMPUTER VISION BASED ON RASPBERRY PI SYSTEM , Applied Computer Science: Vol. 16 No. 4 (2020)
- Sahar ZAMANI KHANGHAH, Keivan MAGHOOLI, EMOTION RECOGNITION FROM HEART RATE VARIABILITY WITH A HYBRID SYSTEM COMBINED HIDDEN MARKOV MODEL AND POINCARE PLOT , Applied Computer Science: Vol. 20 No. 1 (2024)
- Zahid Zamir, CAN THE SYSTEM, INFORMATION, AND SERVICE QUALITIES IMPACT EMPLOYEE LEARNING, ADAPTABILITY, AND JOB SATISFACTION? , Applied Computer Science: Vol. 19 No. 1 (2023)
- Rumesh Edirimanne, W Madushan Fernando, Peter Nielsen, H. Niles Perera, Amila Thibbotuwawa, OPTIMIZING UNMANNED AERIAL VEHICLE BASED FOOD DELIVERY THROUGH VEHICLE ROUTING PROBLEM: A COMPARATIVE ANALYSIS OF THREE DELIVERY SYSTEMS. , Applied Computer Science: Vol. 20 No. 1 (2024)
- Thanh-Nghia NGUYEN, Thanh-Hai NGUYEN, Ba-Viet NGO, R PEAK DETERMINATION USING A WDFR ALGORITHM AND ADAPTIVE THRESHOLD , Applied Computer Science: Vol. 18 No. 3 (2022)
- Sebastian CYGAN, Barbara BOROWIK, Bohdan BOROWIK, STREET LIGHTS INTELLIGENT SYSTEM, BASED ON THE INTERNET OF THINGS CONCEPT , Applied Computer Science: Vol. 14 No. 1 (2018)
You may also start an advanced similarity search for this article.
