Abstract
Abnormal crowd behaviors in high density situations can pose great danger to public safety. Despite the extensive installation of closed-circuit television (CCTV) cameras, it is still difficult to achieve real-time alerts and automated responses from current systems. Two major breakthroughs have been reported in this research. Firstly, a spatial-temporal texture extraction algorithm is developed. This algorithm is able to effectively extract video textures with abundant crowd motion details. It is through adopting Gabor-filtered textures with the highest information entropy values. Secondly, a novel scheme for defining crowd motion patterns (signatures) is devised to identify abnormal behaviors in the crowd by employing an enhanced gray level co-occurrence matrix model. In the experiments, various classic classifiers are utilized to benchmark the performance of the proposed method. The results obtained exhibit detection and accuracy rates which are, overall, superior to other techniques.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
B. Solmaz, B. E. Moore, M. Shah. Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 10, pp. 2064–2070, 2012. DOI: 10.1109/TPAMI.2012.123.
B. Krausz, C. Bauckhage. Loveparade 2010: Automatic video analysis of a crowd disaster. Computer Vision and Image Understanding, vol. 116, no. 3, pp. 307–319, 2012. DOI: 10.1016/j.cviu.2011.08.006.
X. Y. Cui, Q. S. Liu, M. C. Gao, D. N. Metaxas. Abnormal detection using interaction energy potentials. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 3161–3167, 2011. DOI: 10.1109/CVPR.2011.5995558.
D. D. Ma, Q. Wang, Y. Yuan. Anomaly detection in crowd scene via online learning. In Proceedings of the 14th International Conference on Internet Multimedia Computing and Service, ACM, Xiamen, China, pp. 158–162, 2014. DOI: 10.1145/2632856.2632862.
R. Raghavendra, A. Del Bue, M. Cristani, V. Murino. Optimizing interaction force for global anomaly detection in crowded scenes. In Proceedings of IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, pp. 136–143, 2011. DOI: 10.1109/ICCVW.2011. 6130235.
A. Zeroual, N. Messai, S. Kechida, F. Hamdi. A piecewise switched linear approach for traffic flow modeling. International Journal of Automation and Computing, vol. 14, no. 6, pp. 729–741, 2017. DOI: 10.1007/s11633-017-1060-4.
Z. C. Song, Y. Z. Ge, H. Duan, X. G. Qiu. Agent-based simulation systems for emergency management. International Journal of Automation and Computing, vol. 13, no. 2, pp. 89–98, 2016. DOI: 10.1007/s11633-016-0958-6.
G. Csurka, C. R. Dance, L. X. Fan, J. Willamowski, C. Bray. Visual categorization with bags of keypoints. In Proceedings of ECCV International Workshop on Statistical Learning in Computer Vision, Grenoble, France, pp. 145–146, 2004.
D. Helbing, P. Molnar. Social force model for pedestrian dynamics. Physical Review E, vol. 51, no. 5, pp. 4282–4294, 1995. DOI: 10.1103/PhysRevE.51.4282.
W. Yan, Z. Zou, J. B. Xie, T. Liu, P. Q. Li. The detecting of abnormal crowd activities based on motion vector. Optik, vol. 166, pp. 248–256, 2018. DOI: 10.1016/j.ijleo.2017.11.187.
Y. Hao, Y. Liu, J. L. Fan. A crowd behavior feature descriptor based on optical flow field. Journal of Xi’an University of Posts and Telecommunications, vol. 21, no. 6, pp. 55–59, 2016. DOI: 10.13682/j.issn.2095-6533.2016.06.011. (In Chinese)
S. A. Niyogi, E. H. Adelson. Analyzing and recognizing walking figures in XYT. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 469–474, 1994. DOI: 10.1109/CVP R. 1994.323868.
M. Kass, A. Witkin, D. Terzopoulon. Snakes: Active contour models. International Journal of Computer Vision, vol. 1, no. 4, pp. 321–331, 1988. DOI: 10.1007/BF00133570.
J. Wang, Z. J. Xu. STV-based video feature processing for action recognition. Signal Processing, vol. 93, no. 8, pp. 2151–2168, 2012. DOI: 10.1016/j.sigpro.2012.06.009.
C. Van Gemeren, R. Poppe, R. C. Veltkamp. Hands-on: deformable pose and motion models for spatiotemporal localization of fine-grained dyadic interactions. EURASIP Journal on Image and Video Processing, vol. 2018, Article number 16, 2018. DOI: 10.1186/s13640-018-0255-0.
X. F. Ji, Q. Q. Wu, Z. J. Ju, Y. Y. Wang. Study of human action recognition based on improved spatio-temporal features. International Journal of Automation and Computing, vol. 11, no. 5, pp. 500–509, 2014. DOI: 10.1007/s11633-014-0831-4.
E. H. Aldelson, J. R. Bergen. Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, vol. 2, no. 2, pp. 284–299, 1985. DOI: 10.1364/JOSAA.2.000284.
Y. Iwashita, M. Petrou. Person identification from spatiotemporal volumes. In Proceedings of the 23rd International Conference Image and Vision Computing, IEEE, Christchurch, New Zealand, 2008. DOI: 10.1109/IVCNZ.2008.4762086.
R. C. Bolles, H. H. Baker, D. H. Marimont. Epipolar-plane image analysis: an approach to determining structure from motion. International Journal of Computer Vision, vol. 1, no. 1, pp. 7–55, 1987. DOI: 10.1007/BF00128525.
H. H. Baker, R. C. Bolles. Generalizing epipolar-plane image analysis on the spatiotemporal surface. In Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, Ann Arbor, USA, pp. 33–49, 1988. DOI: 10.1109/CVPR.1988.196209.
G. Kuhne G, S. Richter, M. Beier. Motion-based segmentation and contour-based classification of video objects. In Proceedings of the 9th ACM international conference on Multimedia, Ottawa, Canada, pp. 41–50, 2001. DOI: 10.1145/500141.500150.
C. W. Ngo, T. C. Pong, R. T. Chin. Detection of gradual transitions through temporal slice analysis. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, USA, pp. 41–46, 1999. DOI: 10.1109/CVPR.1999.786914.
Y. Hao, Z. J. Xu, J. Wang, Y. Liu, J. L. Fan. An approach to detect crowd panic behavior using flow-based feature. In Proceedings of the 22nd International Conference on Automation and Computing, IEEE, Colchester, UK, pp. 462–466, 2016. DOI: 10.1109/IConAC.2016.7604963.
J. H. Xiang, H. Fan, J. Xu. Abnormal behavior detection based on spatial-temporal features. In Proceedings of International Conference on Machine Learning and Cybernetics, IEEE, Tianjin, China, pp. 871–876, 2013. DOI: 10.1109/ICMLC.2013.6890406.
H. H. Alqaysi, S. Sasi. Detection of abnormal behavior in dynamic crowded gatherings. In Proceedings of IEEE Applied Imagery Pattern Recognition Workshop, IEEE, Washington DC, USA, pp. 1–6, 2013. DOI: 10.1109/AIP R. 2013.6749309.
C. Li, Z. J. Han, Q. X. Ye, J. B. Jiao. Abnormal behavior detection via sparse reconstruction analysis of trajectory. In Proceedings of the 6th International Conference on Image and Graphics, IEEE, Hefei, China, pp. 807–810, 2011. DOI: 10.1109/ICIG.2011.104.
C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, vol. 27, no. 3, pp. 379–423, 1948. DOI: 10.1002/j.1538-7305.1948.tb01338.x.
K. He, S. X. Wang. Study on denoising of fractal signal based on Shannon entropy. In Proceedings of International Conference on Neural Networks and Signal Processing, IEEE, Nanjing, China, pp. 751–755, 2003. DOI:10. 1109/ICNNSP.2003.1279384.
S. Liang, Y. Ma, Y. Y. Huang, J. Guo, C. F. Jia. The scheme of detecting encoded malicious web pages based on information entropy. In Proceedings of the 10th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, IEEE, Fukuoka, Japan, pp. 310–312, 2016. DOI: 10.1109/IMIS.2016.82.
Z. J. Zhang, X. N. Wang, L. Sun. Mobile payment anomaly detection mechanism based on information entropy. IET Networks, vol. 5, no. 1, pp. 1–7, 2014. DOI: 10.1049/iet-net.2014.0101.
D. K. Panda, S. Meher. Hierarchical background subtraction algorithm using Gabor filter. In Proceedings of IEEE International Conference on Electronics, Computing and Communication Technologies, Bangalore, India, pp. 1–6, 2015. DOI: 10.1109/CONECCT.2015.7383876.
D. X. Zhou, H. Zhang, N. Ray. Texture based background subtraction. In Proceedings of IEEE International Conference on Information and Automation, Changsha, China, pp. 20–23, 2008. DOI: 10.1109/ICINFA.2008.4608070.
J. C. S. J. Junior, S. R. Musse, C. R. Jung. Crowd analysis using computer vision techniques. IEEE Signal Processing Magazine, vol. 27, no. 5, pp. 66–77, 2010. DOI: 10.1109/ MSP.2010.937394.
R. M. Haralick, K. Shanmugam, I. Dinstein. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-3, no. 6, pp. 610–621, 1973. DOI: 10.1109/TSMC.1973.4309314.
Y. Benezeth, P. M. Jodoin, V. Saligrama, C. Rosenberger. Abnormal events detection based on spatio-temporal Cooccurences. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, pp. 2458–2465, 2009. DOI: 10.1109/CVPR.2009.5206686.
J. B. Shen, X. G. Jin, C. Zhou, H. L. Zhao. Dynamic textures using wavelet analysis. In Proceedings of International Conference, Springer, Berlin Heidelberg, Germany, pp. 1070–1073, 2006. DOI: 10.1007/11736639—132.
The GLCM Tutorial. [Online], Available: https://doi.org/www.fp.ucalgary.ca/mhallbey/tutorial.htm, June 28, 2018.
S. B. Nikam, S. Agarwal. Wavelet energy signature and GLCM features-based fingerprint anti-spoofing. In Proceedings of the International Conference on Wavelet Analysis and Pattern Recognition, IEEE, Hong Kong, China, pp. 717–723, 2008. DOI: 10.1109/ICWAPR.2008.4635872.
G. Mukherjee, A. Chatterjee, B. Tudu. Study on the potential of combined GLCM features towards medicinal plant classification. In Proceedings of the 2nd International Conference on Control, Instrumentation, Energy and Communication, IEEE, Kolkata, India, pp. 98–102, 2016. DOI: 10.1109/CIEC.2016.7513746.
R. K. Ranjan, A. Agrawal. Video summary based on Fsift, Tamura textural and middle level semantic feature. Procedia Computer Science, vol. 89, pp. 870–876, 2016. DOI: 10.1016/j.procs.2016.06.075.
Acknowledgements
This research is funded by Chinese National Natural Science Foundation (No. 61671377) and Shaanxi Smart City Technology Project of Xianyang (No. 2017k01-25–5).
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Associate Editor Jie Zhang
Yu Hao received the B. Sc. degree in electronic engineering from Xidian University, China in 2008, and the M. Sc. degree in computer science from the Wichita State University, USA in 2011, and he is the Ph. D. degree candidate in computing and engineering from the University of Huddersfield, UK since 2015. Currently, he is a lecturer in School of Computer Science and Technology, Xi’an University of Posts and Telecommunications, China. He has published about 7 refereed journal and conference papers during his Ph. D. program.
His research interest is crowd abnormal behavior analysis.
Zhi-Jie Xu received the B. Sc. degree in communication engineering from the Xi’an University of Science and Technology, China in 1991. After graduation, he has worked for one of the major Chinese Electrical and Machinery Companies–HH Yellow River Ltd for four years as an electronics engineer. He then traveled to the UK and spent a year working in a robotics labratory in Derby, UK. In 1996, he registered and became a postgraduate student at the University of Derby, UK. His research topic is virtual reality for manufacturing simulations. In 2000, he has completed his Ph. D. study and immediately been offered a tenured academic post at the University of Huddersfield, UK. He has published over 100 peer-reviewed journal and conference papers as well as editing 5 books in the relevant fields. He has supervised 11 postgraduate (including 8 Ph. D.) students to completion and been continuously winning substantial research and development grants in his career to date. He is a member of the IEEE, Institution of Engineering and Technology (IET), British Computer Society (BCS), The British Machine Vision Association (BMVA) and a fellow of Higher Education Academy (HEA). In addition, he has been serving as an editor, reviewer and chair for many prestigious academic journals and conferences.
His research interests include visual computing, vision systems, data science and machine learning.
Ying Liu received the Ph. D. degree in computer vision from the Monash University, Australia in 2007. And she worked as a post doctor researcher at Nanyang Technological University, Singapore until 2010. She is the chief engineer of Shaanxi Forensic Science Digital Information Laboratory Research Center, China since 2012. Currently, she is the assistant dean of School of Communications and Information Engineering at Xi’an University of Posts and Telecommunications, China. She has published over 60 peer-reviewed journal and conference papers in the relevant fields. She was grant annual best paper of Pattern Recognition and Tier A paper from Australia Research Council.
Her research interest include pattern recognition, machine learning and forensic science.
Jing Wang received the B. Sc. degree in machine and electronic technology from the Xidian University, China in 2006. After graduation, he was appointed as software engineer and carried out development work on computer vision (CV)-based quality control systems, such as assembly line monitoring and industrial robotic controls. In 2008, he began his postgraduate study at the University of Huddersfield and received his Ph. D. degree in computer vision from University of Huddersfield, UK in 2012. He then became a research fellow and carried out independent researches on image processing, analysing and understanding. Since 2008, He has published more than 20 journal and conference papers in the relative fields. He is a member of the British Machine Vision Association (BMVA) and British Computer Society (BCS). He has also served as chair and editor for the International Conference on Automation and Computing.
His research interest is real-world applications of computer vision systems.
Jiu-Lun Fan received the B. Sc. and M. Sc. degrees in mathematics from the Shaanxi Normal University, China in 1985 and 1988, respectively, and the Ph. D. degree in electronic engineering from the Xidian University, China in 1998. Currently, he is the president of Xi’an University of Posts and Telecommunications, China since 2015. He has published over 200 peer-reviewed journal and conference papers in the relevant fields.
His research interests include signal processing, pattern recognition and communications security.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://doi.org/creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Hao, Y., Xu, ZJ., Liu, Y. et al. Effective Crowd Anomaly Detection Through Spatio-temporal Texture Analysis. Int. J. Autom. Comput. 16, 27–39 (2019). https://doi.org/10.1007/s11633-018-1141-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-018-1141-z