Operating in the photovoltaic mode, the photodiodes harvest energy from ambient light, using the instantaneously harvested power from individual photodiodes for sensing nearby finger gestures. The harvested power from all photodiodes are aggregated to drive the whole gesture-recognition module including a microcontroller running the recognition algorithm.
The team led by Yichen Li, post-doctoral researcher, and Tianxing Li, a PhD student, designed a robust, lightweight algorithm to recognize finger gestures in the presence of ambient light fluctuations. The algorithm runs on a MSP432P401R microcontroller operating in three modes: 1) LPM3 mode (660 nA/3.3V, CPU idle); 2) active mode (80 µA/MHz/3.3V, 48 MHz clock) running CFAR; and 3) ADC_DMA mode (1.4 mA/3.3V, 25 MHz clock) controlling the decoder and sampling voltage number.
The team built two prototypes to test out interactions with smart glasses and smart watches. The microcontroller is in the active mode for 0.14% (glasses) and 0.36% (watch) of the time, in the ADC_DMA mode for 0.28% (glasses) and 0.39% (watch) of the time, and in the LPM3 mode otherwise.
Given that an ADC conversion takes 5 µs, collecting voltage numbers from all units takes 80 µs on the glasses and 110 µs on the watch, so the photodiodes harvest power in more than 99.5% of the time. The microcontroller runs the gesture recognition algorithm to output detected gesture. The measurements show that the recognizing a gesture takes 10 µs on the glasses prototype and 30 µs on the watch. To minimize the power consumption, the team removed unrelated units such as USB bridge chip and LED indicators on the board. To further reduce the computation overhead, they replaced all of the multiplications and divisions to shift operations, since the multipliers and dividers are factor of two. The energy harvested by photodiodes powers the whole system, including SPDT switches, decoders and the micro-controller.
The results show 99.7%/98.3% overall precision/recall in recognizing five gestures on glasses and 99.2%/97.5% precision/recall in recognizing