Artificial Intelligence

VXG VMS has a unique pipeline for Cloud AI processing that makes video analytics cost-effective, scalable, and flexible.

Cloud AI processing pipeline

To make the AI operations cost-efficient, VXG will first filter video by events, such as motion detection. Then sample data - extract short clips and images and then send this to AI engines. This will result in the processing of a fraction of data compared to the input streams.

Cloud VMS provides a variety of pre-processing functions and an interface for integration with Cloud AI engines.

You can connect the same camera to multiple AI engines, or switch between different AI engines based on certain conditions. This provides unmatched flexibility to video analytics.

Event filtering and data sampling are key operations for Cloud video analytics, they make Cloud AI scalable and cost-efficient.

The pipeline has 3 main stages.

Event filtering

Event filtering is usually done by motion events generated on a camera. It can also be any type of event produced by a camera, external sensor, or software service. VXG Cloud VMS can process millions of events simultaneously and filter data based on selected criteria per camera.

Video sampling

Sampling is a process of extracting video fragments and/or images from videos that contain the most significant information. VXG Cloud VMS can extract images and clips from 100s of thousands of cameras simultaneously.

Communicating to AI

After video from a camera is filtered by events and then sampled (extracted images or short clips) we’ll get a small fraction of data that will be sent to a Cloud AI and the results of AI processing - metadata will be stored in the VMS.

Cloud AI services

As a rule, Cloud AI engines only charge for actually processed data. If after filtering and sampling, we’ll get 10 images in a day, we’ll only pay for the processing of these 10 images.

For example, when we need to detect a person in a front yard, we’ll first filter video by motion, then from each video duration with motion we’ll extract several images and send these images to Cloud AI for object detection. If Cloud AI returns “human” as one of the objects then we’ll trigger an alarm.

VXG is pre-integrated with the following Cloud AI engines:

1. Amazon Rekognition

2. Google Vision AI

3. Azure Cognitive Services

4. OpenAI / ChatGPT (in development)

Features of Cloud AI services

For the full list of AI features please visit the corresponding AI services websites:

Amazon Rekognition:

• For images: https://aws.amazon.com/rekognition/image-features/

• For video: https://aws.amazon.com/rekognition/video-features/

 Azure Cognitive Services:

https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/#features

 Google Vision API:

https://cloud.google.com/vision#section-11

VMS operations with AI

Cloud VMS is integrated with the following Cloud AI functions:

• Object recognition (this includes detection of humans, cars, pets, weapons, and other objects)

• Face recognition (in development)

• Other functions can be integrated using the API or by requesting VXG.

VMS can perform the following operations with the metadata produced by Cloud AI:

Visualize metadata

Bounding boxes around objects and other visualization.

Search in metadata

For example, find all events with persons, or find all detected guns:

Generate alerts

The VMS can generate alerts based on the metadata received from Cloud AI. For example, generate an alert when a gun is detected, or when more than 10 people are detected.