Video analytics (also known as video content analysis, or VCA) are a type of technology that automatically analyses video content. They do this by using algorithms that process video in order to carry out a specific task — for example, identifying moving objects or reading vehicle licence plates. When artificial intelligence is involved, they are often referred to as intelligent video analytics.

In the physical security industry, video analytics software is used to automate the job of watching hours of CCTV footage in search of potential threats, such as intruders. Video surveillance is becoming more and more popular, so integrating video analytics can help security operators to identify incidents during long shifts, where they have to stay alert and monitor multiple camera feeds at once.

How do video analytics work?

The simplest video analytics use rule-based algorithms that follow a decision tree of “if/then” questions to predict whether an object in a video could be a possible threat. After a process of elimination, if the software has decided the incident is a threat, it generates an alert for a human operator to verify.

Analytics software isn’t watching a continuous stream of video, instead it’s isolating freeze frames as individual images, then analysing them in sequence. A rule-based algorithm will ask multiple questions of each image, testing for an outcome before it is able to determine whether there is a security threat present or not.

To help you visualise how this works, below is an example of the kinds of “if/then” rules analytics software might use to identify a moving object in a video:

The algorithm asks multiple questions, such as whether there was movement and then the size of that movement. At each step, the algorithm goes down this decision tree in order to make a prediction.

While they have their uses, rule-based algorithms present us with some serious limitations:

1. The questions that the software asks of a video are set by engineers. No matter how thorough these algorithms are, there will always be a case that doesn’t conform, which slips through the cracks.
2. To get around this problem, engineers might tweak the algorithms’ parameters to ensure nothing is missed — however this in turn can compromise the software’s overall accuracy by imposing too many rules.
3. Simpler algorithms could fail to detect more complex security incidents. For example, if you were to set up motion detection, how sensitive should it be? A trigger that is too sensitive will send alarms all the time, most of them false. However by reducing the sensitivity, you run the risk of missing genuine incidents.