Slides
The premise of continuous mobile vision is that a wearable device can latently observe its environment through a camera sensor. This can assist a user’s attention and memory through object tracking, face recognition, and other vision operations. Unfortunately, such tasks are hindered by a large energy footprint on a small mobile battery. Furthermore, applications with access to wearable camera streams have the potential to invade user and subject privacy by capturing and/or distributing image frames. To address this, we motivate a complete redesign of the sensing pipeline with energy efficiency and user/subject privacy as first-order concerns. Our proposed redesign spans a range of optimizations throughout the pipeline: (1) performing early vision operations in the analog domain of the image sensor, (2) configuring image sensors for proportional energy consumption, and (3) reducing the redundancy of library framework operations. Thus, we investigate efficiency and privacy bottlenecks and opportunities in the mobile vision pipeline, moving towards a practical implementation of continuous mobile sensing.