VAlue proposition
The value of this technology lies in its ability to create input-adaptive neural networks that dynamically adjust their computational complexity based on the characteristics of input data, such as video frames, while maintaining high accuracy. This adaptability allows for significant reductions in computational demand and power consumption, making it particularly suitable for resource-constrained mobile devices like smartphones, drones, and augmented reality headsets. By employing an early exit mechanism that is automatically determined, the system can efficiently process video data with minimal latency and energy usage, addressing the critical need for high-throughput, low-latency, and low-energy on-device video stream analytics. This technology not only enhances the efficiency of deep neural networks in mobile applications but also provides a scalable solution that minimizes memory footprint and computational overhead, thereby enabling a wide range of continuous mobile vision applications in real-time scenarios.
Description of Technology
The technology involves the development and implementation of Deep Neural Networks (DNNs) with flexible size and dynamic early exit strategies, specifically tailored for processing video data in mobile devices. This system employs an input-adaptive neural network that adjusts computational demand based on context characteristics such as computational resource availability, power resource availability, user settings, or characteristics of the input data units. The neural network architecture includes an early exit mechanism, which is determined through an automated process, allowing for reduced computational demand and power consumption while maintaining accuracy. The system can be applied to various tasks, including video processing, where it processes input data units (e.g., image frames) and outputs results, adjusting computational and energy requirements based on resource demand indications such as frame rate or real-time computational resource availability. Additionally, the technology provides methods for generating input-adaptive DNNs by assessing filter importance, determining early exit architectures, and optimizing accuracy-resource tradeoffs for specific devices. Furthermore, it outlines a system for modifying existing neural networks by adding early exit branches and determining confidence thresholds for early exits, enhancing the adaptability and efficiency of neural network processing in diverse applications.
Benefits
Applications
IP Status
US Patent12,346,818
LICENSING RIGHTS AVAILABLE
Full licensing rights available
INVENTORs: Mi Zhang, Biyi Fang and Xiao Zeng
Tech ID: TEC2020-0001
For more information about this technology, contact Jon Debling PhD at deblingj@msu.edu or 1(517)884-1653