The Crowd monitoring module is an important tool for ensuring public security in streets, squares, railway stations. A video surveillance system administrator sets a threshold value. If this value exceeds within a certain area under surveillance, this particular area will be recognised as a crowd. The system will send a warning.
All methods of crowd monitoring (crowd detection and people counting in groups) can be divided into two larger groups. The first includes detectors which analyse characteristics of a crowd in general, builds its model, and determines number of people in it. The second cluster of detectors work with the individual objects in a crowd and counts them.
Groups of methods
First group of methods
- Counting based on square
This method is suitable for use with crowds, where people are in motion and making movements. It is based on the use of motion detector. Rectangular areas are allocated within a screen. They correspond to an accumulation of people. There are also masks of movement inside these areas. Masks correspond to changing pixels, which characterise a crowd’s outline. This method defines a crowd’s area by highlighting its contours.
Counting people in crowds based on square
A detector determines the number of people within a crowd by dividing the moving area (crowd) in an image by an area of one person.
Because a crowd can occupy any position in a snapshot, and a camera’s surveillance area can be rather large, the detector must take into account the effects of perspective in an area occupied by one person. That is why, before counting even starts, a system administrator should specify a size of a person in near and in far sight.
People can also overlap each other. This affects the square. Some of them are captured on screen as a whole (the first row of a crowd), while others are only partially captured. In order to calculate the number of people, the method analyses the movements area and determines the degree of its filling with changing pixels. Then an overlapping ratio is calculated.
The method, which is based on the square of the area, is only 70% accurate in crowd monitoring. The results depend greatly on the quality of the motion detector.
- Counting based on special points
Special points are spots of an image where the colour gradient reaches a local maximum. They are located in body corners, at the intersections of contours, garments’ curves, etc.
People counting based on special points
The method works with a frame moving areas and searches for special points in them, where colour gradient changes are happening in both directions simultaneously. The module determines the number of people in a crowd by comparing a number of specific points in moving areas with a number of special points per one person.
The main difficulty lies in determining the number of special points per one person because it is not a constant value. It greatly depends on the degree of overlap between people, lighting, the camera’s position, its settings, etc. That is why this method is often used in conjunction with other methods of counting, such as the method of square footage, for example. In the process of such collaboration, "learning" of a special points method’s algorithms is taking place based on a frames sequence. The results produced by the square method are analysed and compared to results of the special point’s method. An average number of special points per one person is calculated.
This method imposes a number of limitations. First, people can be counted only in the moving crowd. Second, this method’s accuracy depends greatly on other methods accuracy, because of the "training", which is taking place in special points method’s algorithms.
- Counting based on textures
This method assumes that the camera covers a large surveillance area and the crowd’s density is high and located in a particular part of a screen. If an image of such a crowd is being analysed, it presents a combination of components, which are emerging into a certain pattern. It means that an image has a special structure, which can be detected by this method’s algorithm. The number of people can be calculated after that by analysing this special structure and determining the frequency arrangement of its crowd pattern elements.
It is the most inaccurate of the methods presented. It has about a 50% margin of error. However, it can be successfully used to estimate the number of participants at mass events, for example, where tens and hundreds of thousands of people are present in a frame.
Second group of methods
- Counting based on heads detection
This method analyses how many head images are present in a particular frame and calculates the number of people based on them. A special classifier is used to search for head images. It can be trained to recognise any objects or their parts. The head-counting search was picked for a reason: in spite of the fact that people can be in any part of a snapshot, and can be placed randomly inside of a crowd, they overlap each other. Despite that, their heads usually fall within a camera's vision. In addition, a head has a special shape, which is the same for everybody. That is why its image can be used as a universal counting tool. This method’s algorithms analyse each frame. Perspective is taken into consideration.
Counting based on head images
The difficulty of applying this method is that it requires high-resolution images from a camera. A head image should be at least not more or less than certain dimensions. Images which are too small or too large cannot be applicable for people counting in groups. There can be false starts by a classifier for round-shaped objects in a frame (for example, lanterns, etc.) or skipping of head images within a complex background or in a dense crowd.
Statistics demonstrate that underestimations occur more often than over-estimations in cases of incorrect calculations of head images. At the same time, in cases of false starts, the counting method based on square often gives over-estimations. These under- and over-estimations have approximately the same value for similar scenes. This feature is used to compensate for error, when both methods are used together. Detectors are counting the number of people in a crowd based on square footage and the number of heads. An average value is the result of those calculations.
- Counting based on movement’s pathway
This method works in conjunction with a Tracking module. Algorithms analyse the nature of objects’ movements in a pre-set area of a screen, build their pathways, and an objects’ movement direction map. Furthermore, some algorithms require classification and allow detection of only people’s movement. Others can work with any moving objects. Counting is performed based on detected movement pathways within a certain area.
The advantages of this method lie primarily in its high degree of accuracy. The method can count people even if a crowd is packed together and people overlap each other because movements are being analysed not the images themselves. People move in their individual manner. It helps to detect and divide almost every person.
Accuracy complexity, errors
Various software products implement various methods for counting people in groups. They all demonstrate different results in terms of accuracy, and impose different requirements and restrictions on cameras placement, their settings, and a crowd’s position on a screen.
- The most accurate, but also most difficult for implementation is a method based on people’s movement pathways.
- Texture analysis produces the most errors.
- Three remaining methods, namely, counting based on square, on heads images, and special points are accurate in 70% of cases on average. They are often used together to increase the counting accuracy because some methods overestimate, and some underestimate the results approximately by an equal amount.