Spatial identification of manipulable objects for a bionic hand prosthesis

This article presents a method for the spatial identification of objects for bionic upper limb prostheses, utilizing the analysis of digital images captured by an optoelectronic module based on the ESP32-CAM and classified using neural network algorithms, specifically FOMO (MobileNetV2). Modern bionic prostheses that imitate natural limb functions, as well as their advantages and significance for restoring the functionality of the human body, are analysed. An algorithm for a grip-type recognition system is proposed, integrating spatial identification of object shapes with the analysis of myographic signals to enable accurate selection and execution of appropriate manipulations. The neural network was trained on a set of images of basic shapes (spherical, rectangular, cylindrical), which achieved an average identification accuracy of over 89% with a processing time of one image of 2 ms. Due to its compactness and low cost, the developed system is suitable for integration into low-cost prostheses, ensuring adaptation of the movements of the artificial limb to the shape of the objects of manipulation and minimizing the risk of slipping objects. The proposed approach helps to increase the accuracy of movement execution and reduce dependence on expensive and complex technologies. The system has potential for further improvement, as it can operate with objects of complex shapes and handle scenarios involving multiple objects within the camera's field of view simultaneously.