Utility-Aware Load Shedding for Real-time Video Analytics at the Edge

Real-time video analytics typically require video frames to be processed by a query to identify objects or activities of interest while adhering to an end-to-end frame processing latency constraint. Such applications impose a continuous and heavy load on backend compute and network infrastructure because of the need to stream and process all video frames. Video data has inherent redundancy and does not always contain an object of interest for a given query. We leverage this property of video streams to propose a lightweight Load Shedder that can be deployed on edge servers or on inexpensive edge devices co-located with cameras and drop uninteresting video frames. The proposed Load Shedder uses pixel-level color-based features to calculate a utility score for each ingress video frame, which represents the frame's utility toward the query at hand. The Load Shedder uses a minimum utility threshold to select interesting frames to send for query processing. Dropping unnecessary frames enables the video analytics query in the backend to meet the end-to-end latency constraint with fewer compute and network resources. To guarantee a bounded end-to-end latency at runtime, we introduce a control loop that monitors the backend load for the given query and dynamically adjusts the utility threshold. Performance evaluations show that the proposed Load Shedder selects a large portion of frames containing each object of interest while meeting the end-to-end frame processing latency constraint. Furthermore, the Load Shedder does not impose a significant latency overhead when running on edge devices with modest compute resources.