Visualizing Digital Video Stream into 2D real-time model

Just a few months ago, I wrote a blog about modeling a physical retail store to drive digital experiences online on the Sitecore Commerce platform. Today, I want to cover how to enhance it by using the Spatial Analytics Container available from Microsoft in Preview mode.

Spatial Analytics is part of the Azure Computer Vision services and is designed to gain an understanding of how people move throughout the space. The Platform can use a live RTSP stream from the camera or a video file to generate events as people move and cross predefined areas. The Platform can generate entrance and people counting events and provide details on mask-wearing and social distancing in almost real-time. Additional metadata is captured to indicate a person’s relative geo-position coordinates.

Spatial Analytics Container can be installed on Stack Edge Device, a physical computer with NVIDIA Tesla T4 GPU, or a VM running NC series running K80 GPU. Microsoft provides an affordable version of this highly customized GPU-first VM at $290 a month for evaluation and development purposes. I used Standard_NC6_Promo, and I was able to develop my PoC for under $50, well below my Visual Studio monthly Azure allowance.

Provisioning and configuring spatial container could take an hour and is well documented here. The installation steps provide institutions on how to configure docker and NVIDIA drivers and how to configure IoT Edge on VM. Configuration of container could be the most complex part of the installation process. I use Operation Guide to create my Deployment Manifest file – this file allows us to preconfigure all required elements, such as video URL, zone configuration, and operations for Spatial Analytics. Once the file is created, we simply need to execute Azure CLI to deploy the container as IoT Edge Module

az iot edge set-modules –hub-name “xxxx” –device-id “iot” –content deployment.json -–subscription “xxx”

In order to start visualizing geo-spatial analytics or positions of people in the video, I opted to use IoT hub, along with Digital Twin, to persist coordinates and people counts. Digital Twin is an excellent choice to model physical objects for digital consumption and can be used in a low-code or codeless setting. Finally, to present data in a 2d visual way, i created a simple canvas application that uses ajax, with a predefined refresh rate – SignalR is a significantly better choice to present data in near-real-time.

I was amazed at how accurate the Spatial Analytics container is. As you can see in the video, the system was able to detect counts very accurately and was only stumbled to detect a person in the last frame (as the person took a seating position ). The accuracy of geo location was sufficient for most usages, but I am sure it can be improved by dividing the video stream into more clearly divided areas. The most impressive finding was how easy it is to get geo-spatial analytics from a video – I was able to assemble this simple PoC in just one weekend.