Lightweight optimization strategies for modern video applications over wireless networks

Abstract

The various video applications running over wireless networks face a common set of challenges, such as inconsistent bandwidth, high network variability, and sudden latency spikes. Considering such constraints, this thesis explores the possible strategies and design choices for developing data-intensive video applications such as real-time traffic surveillance, live video streaming, and cloud-assisted autonomous driving. Our focus is primarily on two techniques across three distinct video-streaming applications. The first technique is the intelligent use of tiled encoding available in modern video codecs, where the encoded video has independent rectangular spatial regions that can be manipulated in real time without re-encoding. The second technique is to develop data filtering strategies to minimize ingestion costs by pruning extraneous information. In traffic surveillance, cameras stream video to servers for computer vision algorithms, consuming significant bandwidth. To reduce bandwidth usage, we use tile sampling to select spatial regions of frames that contain only moving objects, since the rest of the frame mostly has static backgrounds, such as the sky and buildings. To select such tiles, we propose an adaptive tile selection algorithm that samples only tiles with moving objects by leveraging their correlation with tile bitrates. Our evaluations across different lighting, weather, and traffic conditions, both using benchmark videos and a live deployment, show improved accuracy, reduced bandwidth usage, and lower overhead than existing systems. For live streaming, we propose using the network interface of a nearby helper device to get an aggregated bandwidth. To stream videos, we design a tile-level aggregation strategy that partitions tiles into two subsets and schedules them independently across available network interfaces, based on their importance. We demonstrate, through extensive experiments, that such an aggregation strategy provides better QoE than conventional multipath strategies. Lastly, we address the high data ingestion cost in self-driving cars. Autonomous vehicles rely on compressed models whose accuracy degrades over time as the distribution of real-world test data changes, requiring frequent retraining on a server. This necessitates limiting the number of selected training frames to minimize the end-to-end delay in obtaining the retrained model without impairing post-training accuracy. We sample only the most useful frames and adaptively encode them into a video based on the network bandwidth. Our lightweight sampling strategy seamlessly integrates with the existing workflow, achieving superior accuracy while minimizing update delay compared to baseline strategies across different network conditions