420 Kent Avenue
Brooklyn, NY 11249

hi at kota bk dot com

+1 917-746-8852

Jul 2020

Solenoids & Hand Gesture Control

A device that reacts to the fingers you raise

Following my work on Computer Vision applied to a robotic paper arm, I played with a few solenoids triggered by the fingers the viewer raised in front of a Raspberry Pi camera. I used the really well written C++ framework Mediapipe from Google, that particularly suits power-hungry applications such as this one (2 DeepNet models used here + real-time relative joint position computation). The whole thing runs on a Raspberry 4 and the solenoids are controlled by an Arduino Mega.

Each raised finger triggers its matching column in the solenoid array.

Where to go from there?

First, I'd like to get rid of the 2" lag between the gesture and the solenoid activation. For that, I think that removing one layer of computation (joint position) to only focus on the hand as a whole - thus removing some functionality but preserving the magical effect of real-time trigger - would be worth exploring.

Instead of an array, it would also be interesting to set up the solenoids in concentric circles. They could respond to the viewer's gestures, spreading wider as she/he widen her/his arm's reach.

A reflective material could be mounted on each solenoid to play with the slanted nature of those motors.