Evaluating Gesture Recognition in Virtual Reality (2401.04545v1)
Abstract: Human-Robot Interaction (HRI) has become increasingly important as robots are being integrated into various aspects of daily life. One key aspect of HRI is gesture recognition, which allows robots to interpret and respond to human gestures in real-time. Gesture recognition plays an important role in non-verbal communication in HRI. To this aim, there is ongoing research on how such non-verbal communication can strengthen verbal communication and improve the system's overall efficiency, thereby enhancing the user experience with the robot. However, several challenges need to be addressed in gesture recognition systems, which include data generation, transferability, scalability, generalizability, standardization, and lack of benchmarking of the gestural systems. In this preliminary paper, we want to address the challenges of data generation using virtual reality simulations and standardization issues by presenting gestures to some commands that can be used as a standard in ground robots.
- BlazePose: On-device Real-time Body Pose tracking. (2020). https://doi.org/10.48550/ARXIV.2006.10204
- Alessandro Carfì and Fulvio Mastrogiovanni. 2023. Gesture-Based Human–Machine Interaction: Taxonomy, Problem Definition, and Analysis. IEEE Transactions on Cybernetics 53, 1 (2023), 497–513. https://doi.org/10.1109/TCYB.2021.3129119
- Dynamic Hand Gesture Recognition Using 3DCNN and LSTM with FSM Context-Aware Model. Sensors 19, 24 (2019). https://doi.org/10.3390/s19245429
- Intel RealSense. n.d.. Intel RealSense Depth Camera D435i. https://www.intelrealsense.com/depth-camera-d435i.html.
- A brief analysis of gesture recognition in VR. SID Symposium Digest of Technical Papers 51, S1 (2020), 190–195. https://doi.org/10.1002/sdtp.13787 arXiv:https://sid.onlinelibrary.wiley.com/doi/pdf/10.1002/sdtp.13787
- Kenneth Lai and Svetlana Yanushkevich. 2018. CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition. 3451–3456. https://doi.org/10.1109/ICPR.2018.8545718
- Animated 3D human avatars from a single image with GAN-based texture inference. Computers & Graphics 95 (2021), 81–91. https://doi.org/10.1016/j.cag.2021.01.002
- MediaPipe’s Landmarks with RNN for Dynamic Sign Language Recognition. Electronics 11 (10 2022), 3228. https://doi.org/10.3390/electronics11193228
- Dushyant Kumar Singh. 2021. 3D-CNN based Dynamic Gesture Recognition for Indian Sign Language Modeling. Procedia Computer Science 189 (2021), 76–83. https://doi.org/10.1016/j.procs.2021.05.071 AI in Computational Linguistics.
- GAN-Based Style Transformation to Improve Gesture-Recognition Accuracy. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4, Article 154 (dec 2020), 20 pages. https://doi.org/10.1145/3432199
- A Proposed Set of Communicative Gestures for Human Robot Interaction and an RGB Image-based Gesture Recognizer Implemented in ROS. (2021). https://doi.org/10.48550/ARXIV.2109.09908
- GestureGAN for Hand Gesture-to-Gesture Translation in the Wild. https://doi.org/10.48550/ARXIV.1808.04859
- Real Time Hand Gesture Recognition in Depth Image using CNN. International Journal of Computer Applications 174 (01 2021), 28–32. https://doi.org/10.5120/ijca2021921040
- Dynamic gesture recognition based on 2D convolutional neural network and feature fusion. Scientific Reports 12 (03 2022), 4345. https://doi.org/10.1038/s41598-022-08133-z
- AvatarGen: A 3D Generative Model for Animatable Human Avatars. In Computer Vision – ECCV 2022 Workshops, Leonid Karlinsky, Tomer Michaeli, and Ko Nishino (Eds.). Springer Nature Switzerland, Cham, 668–685.
- Wenjin Zhang and Jiacun Wang. 2019. Dynamic Hand Gesture Recognition Based on 3D Convolutional Neural Network Models. In 2019 IEEE 16th International Conference on Networking, Sensing and Control (ICNSC). 224–229. https://doi.org/10.1109/ICNSC.2019.8743159
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.