The use of video analytics is growing rapidly in the surveillance market. It has proven indispensable in high-risk security projects, and is becoming increasingly popular in commercial jobs for a wide range of applications, including outdoor protection, customer service measurements, people counting, crowd monitoring, and many others.



We are not far from the day when most video products will include embedded content analysis, making all video products smarter. However, there is a lack of solid technical information to help compare available technologies. The problem is that companies often make grand claims, but many fall far short in performance, leading to widespread disappointment.

The purpose of this article, therefore, is to outline the general principles of how video analytics works, in non-technical language, and examine how competing technologies try to solve these problems. The results vary dramatically, and a closer look shows why there are such big differences in performance.

Video analytics systems are built on three core components:

- Motion detection and object segmentation: This is where the video is processed to separate foreground objects from the background. This is the most processor intensive part of video analytics, accounting for up to 80% of the computational resources. However, there is a wide range in how well different products segment moving objects from the rest of the video.

- Object Tracking: This step tracks groups of pixels that are foreground objects as they move from frame to frame. If a group of pixels moves across the scene, it is probably a foreground object. The challenge is to track this blob of changing pixels. Once again, there is a huge range in performance from the different approaches taken.

- Object Classification: This function identifies the type of object detected. If a group of pixels moves across the scene, it is probably a foreground object. The challenge is to track this blob of changing pixels. Once again, there is a huge range in performance from the different approaches taken.

Systems vary in how well they perform each of these three steps. For example, there are many products being sold today as video analytics that are better described as “advanced video motion detection.†They can extract blobs of pixels moving across a scene, but cannot tell whether these objects are people, vehicles or anything else. In other words, they have no object classification.

Such systems have a harder time distinguishing a branch blowing in the wind from a person, but some of them have added object size calibration to help ignore things that are too big or too small. Large animals and tree branches are still a problem with these technologies, and they also require someone manually calibrating each camera for object sizes. Advanced video motion detection systems might be able to perform all the behavior detection types you see in the best video analytics systems, such as tripwires, direction of travel, intrusion detection, etc., but their results cannot compare to the accuracy of systems with superior object classification.

This is just one example of where some products skimp to reduce the amount of processing required. The problem is that you can't easily tell the difference by just looking at such solutions or reading their literature. They claim to offer all of the same features you see in high end systems. However, the differences in results are stark.

Even the best of the advanced video motion detection systems are at least 10X worse compared to even the poorest true video analytics products. Extensive testing easily proves this out, but most integrators and users don't have the time to test dozens of products to find one that works.

For a more detailed description of how video analytics works, read All Analytics are not Created Equal by Doug Marman, CTO and VP of Products at VideoIQ, at http://www.videoiq.net/products/resources.