Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Designing Upper-Body Gesture Interaction with and for People with Spinal Muscular Atrophy in VR (2403.16107v1)

Published 24 Mar 2024 in cs.HC

Abstract: Recent research proposed gaze-assisted gestures to enhance interaction within virtual reality (VR), providing opportunities for people with motor impairments to experience VR. Compared to people with other motor impairments, those with Spinal Muscular Atrophy (SMA) exhibit enhanced distal limb mobility, providing them with more design space. However, it remains unknown what gaze-assisted upper-body gestures people with SMA would want and be able to perform. We conducted an elicitation study in which 12 VR-experienced people with SMA designed upper-body gestures for 26 VR commands, and collected 312 user-defined gestures. Participants predominantly favored creating gestures with their hands. The type of tasks and participants' abilities influence their choice of body parts for gesture design. Participants tended to enhance their body involvement and preferred gestures that required minimal physical effort, and were aesthetically pleasing. Our research will contribute to creating better gesture-based input methods for people with motor impairments to interact with VR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. RePlay: Touchscreen Interaction Substitution Method for Accessible Gaming. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction (New York, NY, USA) (MobileHCI ’21). Association for Computing Machinery, 1–12. https://doi.org/10.1145/3447526.3472044
  2. Exploring Non-touchscreen Gestures for Smartwatches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, San Jose California USA, 3822–3833. https://doi.org/10.1145/2858036.2858385
  3. Métamorphe: augmenting hotkey usage with actuated keys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Paris France, 563–572. https://doi.org/10.1145/2470654.2470734
  4. An Integrated CNN-LSTM Model for Micro Hand Gesture Recognition. In Intelligent Computing and Optimization (Cham) (Advances in Intelligent Systems and Computing), Pandian Vasant, Ivan Zelinka, and Gerhard-Wilhelm Weber (Eds.). Springer International Publishing, 379–392. https://doi.org/10.1007/978-3-030-68154-8_35
  5. On-the-fly calibration for improved on-device eye tracking. US Patent 11,106,280.
  6. The Rehabilitation Gaming System: A Review. Studies in Health Technology and Informatics 145 (2009), 65–83. arXiv:19592787
  7. John W. Creswell. 2007. Qualitative Inquiry and Research Design: Choosing among Five Approaches, 2nd Ed. Sage Publications, Inc. xvii, 395 pages.
  8. Spinal Muscular Atrophies. In Neuromuscular Disorders of Infancy, Childhood, and Adolescence. Elsevier, 117–145. https://doi.org/10.1016/B978-0-12-417044-5.00008-1
  9. Heiko Drewes and Albrecht Schmidt. 2007. Interacting with the Computer Using Gaze Gestures. In Human-Computer Interaction – INTERACT 2007, Cécilia Baranauskas, Philippe Palanque, Julio Abascal, and Simone Diniz Junqueira Barbosa (Eds.). Vol. 4663. Springer Berlin Heidelberg, Berlin, Heidelberg, 475–488. https://doi.org/10.1007/978-3-540-74800-7_43 Series Title: Lecture Notes in Computer Science.
  10. Eyelid Gestures on Mobile Devices for People with Motor Impairments. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Virtual Event Greece, 1–8. https://doi.org/10.1145/3373625.3416987
  11. Eyelid gestures for people with motor impairments. Commun. ACM 65, 1 (Jan. 2022), 108–115. https://doi.org/10.1145/3498367
  12. Virtual Reality Games for People Using Wheelchairs. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–11. https://doi.org/10.1145/3313831.3376265
  13. Head Gesture-based Control for Assistive Robots. In Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference. ACM, Corfu Greece, 379–383. https://doi.org/10.1145/3197768.3201574
  14. Henna Heikkilä and Kari-Jouko Räihä. 2012. Simple gaze gestures and the closure of the eyes as an interaction technique. In Proceedings of the Symposium on Eye Tracking Research and Applications. ACM, Santa Barbara California, 147–154. https://doi.org/10.1145/2168556.2168579
  15. In the blink of an eye: combining head motion and eye blink frequency for activity recognition with Google Glass. In Proceedings of the 5th Augmented Human International Conference. ACM, Kobe Japan, 1–4. https://doi.org/10.1145/2582051.2582066
  16. Clinical Application of Virtual Reality for Upper Limb Motor Rehabilitation in Stroke: Review of Technologies and Clinical Evidence. Journal of Clinical Medicine 9, 10 (2020-10), 3369. Issue 10. https://doi.org/10.3390/jcm9103369
  17. Georgios Kouroupetroglou and Priti Das (Eds.). 2014. Assistive Technologies and Computer Access for Motor Disabilities:. IGI Global. https://doi.org/10.4018/978-1-4666-4438-0
  18. User-defined gestures for connecting mobile phones, public displays, and tabletops. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. ACM, Lisbon Portugal, 239–248. https://doi.org/10.1145/1851600.1851640
  19. Studying user-defined iPad gestures for interaction in multi-display environment. In Proceedings of the 2012 ACM international conference on Intelligent User Interfaces. ACM, Lisbon Portugal, 93–96. https://doi.org/10.1145/2166966.2166984
  20. Michael F. Land and Mary Hayhoe. 2001. In what ways do eye movements contribute to everyday activities? Vision Research 41, 25-26 (Nov. 2001), 3559–3565. https://doi.org/10.1016/S0042-6989(01)00102-X
  21. Rena B Lewis. 1998. Assistive Technology and Learning Disabilities: Today’s Realities and Tomorrow’s Promises. JOURNAL OF LEARNING DISABILITIES 31, 1 (1998).
  22. “I Choose Assistive Devices That Save My Face” A Study on Perceptions of Accessibility and Assistive Technology Use Conducted in China. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
  23. Freedom to Choose: Understanding Input Modality Preferences of People with Upper-body Motor Impairments for Activities of Daily Living. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility. 1–16.
  24. Gaze-Pinch Menu: Performing Multiple Interactions Concurrently in Mixed Reality. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, Lisbon, Portugal, 536–537. https://doi.org/10.1109/VRW52623.2021.00150
  25. User-Defined Gestures with Physical Props in Virtual Reality. Proceedings of the ACM on Human-Computer Interaction 5, ISS (Nov. 2021), 1–23. https://doi.org/10.1145/3486954
  26. “I just went into it assuming that I wouldn’t be able to have the full experience”: Understanding the Accessibility of Virtual Reality for People with Limited Mobility. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Virtual Event Greece, 1–13. https://doi.org/10.1145/3373625.3416998
  27. Memorability of pre-designed and user-defined gesture sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Paris France, 1099–1108. https://doi.org/10.1145/2470654.2466142
  28. User-elicited dual-hand interactions for manipulating 3D objects in virtual reality environments. Human-centric Computing and Information Sciences 8, 1 (Dec. 2018), 31. https://doi.org/10.1186/s13673-018-0154-5
  29. Evaluation of HeadTurn: An Interaction Technique Using the Gaze and Head Turns. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction. ACM, Gothenburg Sweden, 1–8. https://doi.org/10.1145/2971485.2971490
  30. Genetic risk assessment in carrier testing for spinal muscular atrophy. American Journal of Medical Genetics 110, 4 (July 2002), 301–307. https://doi.org/10.1002/ajmg.10425
  31. Embodied Exploration: Facilitating Remote Accessibility Assessment for Wheelchair Users with Virtual Reality. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility. 1–17.
  32. Gaze + pinch interaction in virtual reality. In Proceedings of the 5th Symposium on Spatial User Interaction. ACM, Brighton United Kingdom, 99–108. https://doi.org/10.1145/3131277.3132180
  33. Empirical Evaluation of Gaze-enhanced Menus in Virtual Reality. In 26th ACM Symposium on Virtual Reality Software and Technology. ACM, Virtual Event Canada, 1–11. https://doi.org/10.1145/3385956.3418962
  34. User-defined gestures for augmented reality. Augmented Reality (2013).
  35. Gaze tracking and non-touch gesture based interaction method for mobile 3D virtual spaces. In Proceedings of the 24th Australian Computer-Human Interaction Conference. ACM, Melbourne Australia, 505–512. https://doi.org/10.1145/2414536.2414614
  36. DigiTap: An Eyes-Free VR/AR Symbolic Input Device. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology (Edinburgh, Scotland) (VRST ’14). Association for Computing Machinery, New York, NY, USA, 9–18. https://doi.org/10.1145/2671015.2671029
  37. Newborn and carrier screening for spinal muscular atrophy. American Journal of Medical Genetics Part A 152A, 7 (May 2010), 1608–1616. https://doi.org/10.1002/ajmg.a.33474
  38. Look & Turn: One-handed and Expressive Menu Interaction by Gaze and Arm Turns in VR. In 2022 Symposium on Eye Tracking Research and Applications. ACM, Seattle WA USA, 1–7. https://doi.org/10.1145/3517031.3529233
  39. Immersion of Virtual Reality for Rehabilitation - Review. Applied Ergonomics 69 (2018), 153–161. https://doi.org/10.1016/j.apergo.2018.01.009
  40. FaceSwitch - Low-Cost Accessibility Software for Computer Control Combining Gaze Interaction and Face Gestures. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction. ACM, Parkville VIC Australia, 197–201. https://doi.org/10.1145/2838739.2838809
  41. User-defined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vancouver BC Canada, 197–206. https://doi.org/10.1145/1978942.1978971
  42. The eye wink control interface: using the computer to provide the severely disabled with increased flexibility and comfort. In [1990] Proceedings. Third Annual IEEE Symposium on Computer-Based Medical Systems. IEEE Comput. Soc. Press, Chapel Hill, NC, USA, 105–111. https://doi.org/10.1109/CBMSYS.1990.109386
  43. Outline Pursuits: Gaze-assisted Selection of Occluded Objects in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–13. https://doi.org/10.1145/3313831.3376438
  44. Gaze and gesture based object manipulation in virtual worlds. In Proceedings of the 18th ACM symposium on Virtual reality software and technology. ACM, Toronto Ontario Canada, 203–204. https://doi.org/10.1145/2407336.2407380
  45. FingerInput: Capturing Expressive Single-Hand Thumb-to-Finger Microgestures. In Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces (Tokyo, Japan) (ISS ’18). Association for Computing Machinery, New York, NY, USA, 177–187. https://doi.org/10.1145/3279778.3279799
  46. A Review of Interaction Techniques for Immersive Environments. IEEE Transactions on Visualization and Computer Graphics 29, 9 (Sept. 2023), 3900–3921. https://doi.org/10.1109/TVCG.2022.3174805
  47. Teethtap: Recognizing discrete teeth gestures using motion and acoustic sensing on an earpiece. In 26th International Conference on Intelligent User Interfaces. 161–169.
  48. Gesture Input for Users with Motor Impairments on Touchscreens: Empirical Results based on the Kinematic Theory. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1–6. https://doi.org/10.1145/3170427.3188619
  49. Exploring the design space of gestural interaction with active tokens through user-defined gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Toronto Ontario Canada, 4107–4116. https://doi.org/10.1145/2556288.2557373
  50. Radu-Daniel Vatavu and Ovidiu-Ciprian Ungurean. 2019. Stroke-Gesture Input for People with Motor Impairments: Empirical Results & Research Roadmap. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1–14. https://doi.org/10.1145/3290605.3300445
  51. Radu-Daniel Vatavu and Ovidiu-Ciprian Ungurean. 2022. Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor Impairments. In CHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–16. https://doi.org/10.1145/3491102.3501964
  52. Systematic literature review of hand gestures used in human computer interaction interfaces. International Journal of Human-Computer Studies 129 (Sept. 2019), 74–94. https://doi.org/10.1016/j.ijhcs.2019.03.011
  53. M[Eye]Cro: Eye-Gaze+Microgestures for Multitasking and Interruptions. Proc. ACM Hum.-Comput. Interact. 5, EICS, Article 210 (may 2021), 22 pages. https://doi.org/10.1145/3461732
  54. Intelligent Wearable Virtual Reality (VR) Gaming Controller for People with Motor Disabilities. In 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). IEEE, Taichung, Taiwan, 161–164. https://doi.org/10.1109/AIVR.2018.00034
  55. Florian Weidner and Wolfgang Broll. 2019. Interact with your car: a user-elicited gesture set to inform future in-car user interfaces. In Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia. ACM, Pisa Italy, 1–12. https://doi.org/10.1145/3365610.3365625
  56. Understanding How People with Limited Mobility Use Multi-Modal Input. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 4, 17 pages. https://doi.org/10.1145/3491102.3517458
  57. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Boston MA USA, 1083–1092. https://doi.org/10.1145/1518701.1518866
  58. User-defined gesture interaction for immersive VR shopping applications. Behaviour & Information Technology 38, 7 (July 2019), 726–741. https://doi.org/10.1080/0144929X.2018.1552313
  59. Two-In-One: A Design Space for Mapping Unimanual Input into Bimanual Interactions in VR for Users with Limited Movement. ACM Trans. Access. Comput. 15, 3, Article 23 (jul 2022), 25 pages. https://doi.org/10.1145/3510463
  60. Gaze-Supported 3D Object Manipulation in Virtual Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–13. https://doi.org/10.1145/3411764.3445343
  61. Smartphone-Based Gaze Gesture Communication for People with Motor Disabilities. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, Denver Colorado USA, 2878–2889. https://doi.org/10.1145/3025453.3025790
  62. “I Don’t Want People to Look At Me Differently”: Designing User-Defined Above-the-Neck Gestures for People with Upper Body Motor Impairments. In CHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–15. https://doi.org/10.1145/3491102.3517552
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets