Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed Reality (2403.06891v1)

Published 11 Mar 2024 in cs.HC

Abstract: Tangible interfaces in mixed reality (MR) environments allow for intuitive data interactions. Tangible cubes, with their rich interaction affordances, high maneuverability, and stable structure, are particularly well-suited for exploring multi-dimensional data types. However, the design potential of these cubes is underexplored. This study introduces a design space for tangible cubes in MR, focusing on interaction space, visualization space, sizes, and multiplicity. Using spatio-temporal data, we explored the interaction affordances of these cubes in a workshop (N=24). We identified unique interactions like rotating, tapping, and stacking, which are linked to augmented reality (AR) visualization commands. Integrating user-identified interactions, we created a design space for tangible-cube interactions and visualization. A prototype visualizing global health spending with small cubes was developed and evaluated, supporting both individual and combined cube manipulation. This research enhances our grasp of tangible interaction in MR, offering insights for future design and application in diverse data contexts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. Spatio-temporal aspects and dimensions in integrated disaster risk management. Natural Hazards 68 (2013), 1205–1216. https://doi.org/10.1007/s11069-013-0619-9
  2. A Review of Temporal Data Visualizations Based on Space-Time Cube Operations. In EuroVis - STARs, R. Borgo, R. Maciejewski, and I. Viola (Eds.). The Eurographics Association, Swansea, Wales, United Kingdom, 1–19. https://doi.org/10.2312/eurovisstar.20141171
  3. A Descriptive Framework for Temporal Data Visualizations Based on Generalized Space-Time Cubes. Computer Graphics Forum 36 (2017), 36–61. https://doi.org/10.1111/cgf.12804
  4. Out of the cube: augmented Rubik’s cube. International Journal of Computer Games Technology 2011 (2011), 2–2. https://doi.org/10.1155/2011/570210
  5. The State of the Art of Spatial Interfaces for 3D Visualization. Computer Graphics Forum 40, 1 (Feb. 2021), 293–326. https://doi.org/10.1111/cgf.14189
  6. Evren Bozgeyikli and Lal Lila Bozgeyikli. 2021. Evaluating Object Manipulation Interaction Techniques in Mixed Reality: Tangible User Interfaces and Gesture. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, Lisboa, Portugal, 778–787. https://doi.org/10.1109/VR50410.2021.00105
  7. CAPTIVE: A Cube with Augmented Physical Tools. In CHI ’14 Extended Abstracts on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI EA ’14). Association for Computing Machinery, New York, NY, USA, 1315–1320. https://doi.org/10.1145/2559206.2581340
  8. CodeCubes: Coding with Augmented Reality. In First International Computer Programming Education Conference (ICPEC 2020) (OpenAccess Series in Informatics (OASIcs), Vol. 81), Ricardo Queirós, Filipe Portela, Mário Pinto, and Alberto Simões (Eds.). Schloss Dagstuhl–Leibniz-Zentrum für Informatik, Dagstuhl, Germany, 7:1–7:9. https://doi.org/10.4230/OASIcs.ICPEC.2020.7
  9. Design space for spatio-data coordination: Tangible interaction devices for immersive information visualisation. In 2017 IEEE Pacific Visualization Symposium (PacificVis). IEEE, Soeul, Korea, 46–50. https://doi.org/10.1109/PACIFICVIS.2017.8031578
  10. CubTile: a multi-touch cubic interface. In Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology (Bordeaux, France) (VRST ’08). Association for Computing Machinery, New York, NY, USA, 69–72. https://doi.org/10.1145/1450579.1450593
  11. Urška Demšar and Kirsi Virrantaus. 2010. Space–time density of trajectories: exploring spatio-temporal patterns in movement data. International Journal of Geographical Information Science 24, 10 (2010), 1527–1542. https://doi.org/10.1080/13658816.2010.511223
  12. Spatio-temporal dynamics and evolution of land use change and landscape pattern in response to rapid urbanization. Landscape and urban planning 92, 3-4 (2009), 187–198. https://doi.org/10.1016/j.landurbplan.2009.05.001
  13. Uplift: A tangible and immersive tabletop system for casual collaborative visual analytics. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1193–1203. https://doi.org/10.1109/TVCG.2020.3030334
  14. Min Fan and Alissa N. Antle. 2015. Tactile Letters: A Tangible Tabletop with Texture Cues Supporting Alphabetic Learning for Dyslexic Children. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (Stanford, California, USA) (TEI ’15). Association for Computing Machinery, New York, NY, USA, 673–678. https://doi.org/10.1145/2677199.2688806
  15. Tangi: Tangible proxies for embodied object exploration and manipulation in virtual reality. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Cambridge, UK, 195–206. https://doi.org/10.1109/ismar50242.2020.00042
  16. Touch and Feel Soft Hardware. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (Kingston, Ontario, Canada) (TEI ’12). Association for Computing Machinery, New York, NY, USA, 359–362. https://doi.org/10.1145/2148131.2148217
  17. A visualization environment for the space-time-cube. In Developments in Spatial Data Handling: 11 th International Symposium on Spatial Data Handling. Springer, Leicester, UK, 189–200. https://doi.org/10.1007/3-540-26772-7_15
  18. Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’95). ACM Press/Addison-Wesley Publishing Co., USA, 442–449. https://doi.org/10.1145/223904.223964
  19. Interactive analysis of event data using space-time cube. In Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004. IEEE, Austin, Texas, 145–152. https://doi.org/10.1109/IV.2004.1320137
  20. Grey Island: Immersive tangible interaction through augmented reality. In 2019 IEEE Fourth International Conference on Data Science in Cyberspace (DSC). IEEE, Hangzhou, China, 656–661. https://doi.org/10.1109/dsc.2019.00107
  21. Uttam Grandhi and Ina Yosun Chang. 2019. PlayGAMI: augmented reality origami creativity platform. In ACM SIGGRAPH 2019 Appy Hour. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3305365.3329729
  22. The mixed reality book: a new multimedia reading experience. In CHI ’07 Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA) (CHI EA ’07). Association for Computing Machinery, New York, NY, USA, 1953–1958. https://doi.org/10.1145/1240866.1240931
  23. Joachim Gudmundsson and Michael Horton. 2017. Spatio-temporal analysis of team sports. ACM Computing Surveys (CSUR) 50, 2 (2017), 1–34. https://doi.org/10.2139/ssrn.3060266
  24. Taejin Ha and Woontack Woo. 2010. An empirical evaluation of virtual hand techniques for 3D object manipulation in a tangible augmented reality environment. In 2010 IEEE Symposium on 3D User Interfaces (3DUI). IEEE Computer Society, Waltham, MA, 91–98. https://doi.org/10.1109/3DUI.2010.5444713
  25. Data mining: concepts and techniques. Morgan kaufmann, Waltham, MA. https://doi.org/10.4018/978-1-59904-951-9.ch143
  26. Ian Hickson. 2021. The WebSocket API. W3C Working Group Note. W3C Working Group. Copyright © 2021 W3C® (MIT, ERCIM, Keio, Beihang). W3C liability, trademark and permissive document license rules apply. The bulk of the text of this specification is also available in the WHATWG Web Applications 1.0 specification, under a license that permits reuse of the specification text..
  27. Using an augmented reality-based training system to promote spatial visualization ability for the elderly. Universal Access in the Information Society 18 (2019), 327–342. https://doi.org/10.1007/s10209-017-0597-x
  28. Eva Hornecker and Jacob Buur. 2006. Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada) (CHI ’06). Association for Computing Machinery, New York, NY, USA, 437–446. https://doi.org/10.1145/1124772.1124838
  29. Tangible design support system using RFID technology. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (Bonn, Germany) (TEI ’08). Association for Computing Machinery, New York, NY, USA, 75–78. https://doi.org/10.1145/1347390.1347408
  30. Hui-Ni Hsu. 2022. Paper-based Tangible User Interface as interaction in mixed reality. Master’s thesis. University of Stuttgart. https://doi.org/10.18419/opus-12718
  31. Simultaneous Worlds: Supporting Fluid Exploration of Multiple Data Sets via Physical Models. In Proceedings of Graphics Interface 2022 (Montréal, Quebec) (GI 2022). Canadian Information Processing Society, Montreal, Quebec, Canada, 24 – 34. https://doi.org/10.20380/GI2022.04
  32. Torsten Ilägcrstrand. 1970. What about people in regional science. Papers in Regional Science 24 (1970), 7–24. https://doi.org/10.1111/j.1435-5597.1970.tb01464.x
  33. Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’97). Association for Computing Machinery, New York, NY, USA, 234–241. https://doi.org/10.1145/258549.258715
  34. A tangible volume for portable 3D interaction. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE, Merida, Yucatan, Mexico, 215–220. https://doi.org/10.1109/ismar-adjunct.2016.0079
  35. ShARe: Enabling Co-Located Asymmetric Multi-User Interaction for Augmented Reality Head-Mounted Displays. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’20). Association for Computing Machinery, New York, NY, USA, 459–471. https://doi.org/10.1145/3379337.3415843
  36. A tangible user interface for interactive data visualization. In Proceedings of the 25th Annual International Conference on Computer Science and Software Engineering (Markham, Canada) (CASCON ’15). IBM Corp., USA, 244–247.
  37. Augmented reality interactive storytelling systems using tangible cubes for edutainment. In 2008 Eighth IEEE International Conference on Advanced Learning Technologies. IEEE, Santander, Spain, 233–235. https://doi.org/10.1109/ICALT.2008.122
  38. Tangible cubes used as the user interface in an augmented reality game for edutainment. In 2010 10th IEEE international conference on advanced learning technologies. IEEE, Sousse, Tunisia, 599–603. https://doi.org/10.1109/ICALT.2010.170
  39. Sketched Reality: Sketching Bi-Directional Interactions Between Virtual and Physical Worlds with AR and Actuated Tangible UI. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 1, 12 pages. https://doi.org/10.1145/3526113.3545626
  40. Ju-Young Kang and Hwan-Seung Yong. 2010. Mining spatio-temporal patterns in trajectory data. Journal of Information Processing Systems 6, 4 (2010), 521–536. https://doi.org/10.4018/978-1-59904-387-6.ch008
  41. Inside-reachable and see-through augmented reality Shell for 3D visualization and tangible interaction. Multimedia Tools and Applications 79 (2020), 5941–5963. https://doi.org/10.1007/s11042-019-08324-3
  42. An evaluation of space time cube representation of spatiotemporal patterns. IEEE Transactions on visualization and computer graphics 15, 4 (2008), 696–702. https://doi.org/10.1109/TVCG.2008.194
  43. Krzysztof J. Kruszynski and Robert van Liere. 2008. Tangible Interaction for 3D Widget Manipulation in Virtual Environments. In Eurographics Symposium on Virtual Environments, Robert van Liere and Betty Mohler (Eds.). The Eurographics Association, Crete, Greece, 89–95. https://doi.org/10.2312/EGVE/EGVE08/089-095
  44. CubeQuery: Tangible Interface for Creating and Manipulating Database Queries. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (Dresden, Germany) (ITS ’14). Association for Computing Machinery, New York, NY, USA, 423–426. https://doi.org/10.1145/2669485.2669526
  45. A Design Space For Data Visualisation Transformations Between 2D And 3D In Mixed-Reality Environments. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (<conf-loc>, <city>New Orleans</city>, <state>LA</state>, <country>USA</country>, </conf-loc>) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 25, 14 pages. https://doi.org/10.1145/3491102.3501859
  46. NFCStack: Identifiable Physical Building Blocks that Support Concurrent Construction and Frictionless Interaction. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 26, 12 pages. https://doi.org/10.1145/3526113.3545658
  47. Two-handed tangible interaction techniques for composing augmented blocks. Virtual Reality 15, 2-3 (2011), 133–146. https://doi.org/10.1007/s10055-010-0163-9
  48. Hyeongmook Lee and Woontack Woo. 2010. Tangible spin cube for 3D ring menu in real space. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 4147–4152. https://doi.org/10.1145/1753846.1754117
  49. I-Jui Lee. 2020. Using augmented reality to train students to visualize three-dimensional drawings of mortise–tenon joints in furniture carpentry. Interactive Learning Environments 28, 7 (2020), 930–944. https://doi.org/10.1080/10494820.2019.1572629
  50. Spatio-temporal distribution of flood disasters and analysis of influencing factors in Africa. Natural Hazards 82 (2016), 721–731. https://doi.org/10.1007/s11069-016-2181-8
  51. A soft cube capable of controllable continuous jumping. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Hamburg, 1712–1717. https://doi.org/10.1109/IROS.2015.7353598
  52. Visual exploration of eye movement data using the space-time-cube. In Geographic Information Science: 6th International Conference, GIScience 2010, Zurich, Switzerland, September 14-17, 2010. Proceedings 6. Springer, Springer Berlin Heidelberg, Berlin, Heidelberg, 295–309. https://doi.org/10.1007/978-3-642-15300-6_21
  53. Hongtao Ma. 2020. A Mixed Modeling Environment Exploring Tangible and Digital Interactions for Iterative Design Modeling. Ph. D. Dissertation. Carnegie Mellon University. https://doi.org/10.1184/R1/14135807.v1
  54. Sifteo cubes. In CHI ’12 Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1015–1018. https://doi.org/10.1145/2212776.2212374
  55. Augmented reality: a class of displays on the reality-virtuality continuum. In Telemanipulator and Telepresence Technologies, Hari Das (Ed.), Vol. 2351. International Society for Optics and Photonics, SPIE, Bellingham, WA, U.S.A., 282 – 292. https://doi.org/10.1117/12.197321
  56. Eva Murray. 2020. A Global Perspective on Health Expenditure. https://www.kaggle.com/datasets/thedevastator/a-global-perspective-on-health-expenditure Original data source: OECD Data.
  57. Jo Nakashima. 2019. Origami Rubik’s Cube & Origami Infinity Cube. https://jonakashima.com.br Accessed: 2019-08-21.
  58. Sandra Câmara Olim and Valentina Nisi. 2020. Augmented Reality Towards Facilitating Abstract Concepts Learning. In Entertainment Computing – ICEC 2020, Nuno J. Nunes, Lizhuang Ma, Meili Wang, Nuno Correia, and Zhigeng Pan (Eds.). Springer International Publishing, Cham, 188–204.
  59. Multidimensional data visualization. Behavior Research Methods, Instruments, & Computers 34, 2 (2002), 158–162. https://doi.org/10.3758/BF03195437
  60. Pol Pla and Pattie Maes. 2013. Display blocks: a set of cubic displays for tangible, multi-perspective data exploration. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (Barcelona, Spain) (TEI ’13). Association for Computing Machinery, New York, NY, USA, 307–314. https://doi.org/10.1145/2460625.2460677
  61. TangibleTouch: A Toolkit for Designing Surface-based Gestures for Tangible Interfaces. In Proceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction (<conf-loc>, <city>Daejeon</city>, <country>Republic of Korea</country>, </conf-loc>) (TEI ’22). Association for Computing Machinery, New York, NY, USA, Article 39, 14 pages. https://doi.org/10.1145/3490149.3502263
  62. Wen Qi and Jean-Bernard Martens. 2005. Tangible user interfaces for 3D clipping plane interaction with volumetric data: a case study. In Proceedings of the 7th International Conference on Multimodal Interfaces (Torento, Italy) (ICMI ’05). Association for Computing Machinery, New York, NY, USA, 252–258. https://doi.org/10.1145/1088463.1088507
  63. Cubes. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (Barcelona, Spain) (TEI ’13). Association for Computing Machinery, New York, NY, USA, 397–398. https://doi.org/10.1145/2460625.2460711
  64. Benjamin Salem and Harold Peeters. 2007. InterCUBE: A Study into Merging Action and Interaction Spaces. In Human-Computer Interaction – INTERACT 2007, Cécilia Baranauskas, Philippe Palanque, Julio Abascal, and Simone Diniz Junqueira Barbosa (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 57–70.
  65. Active Proxy Dashboard: Binding Physical Referents and Abstract Data Representations in Situated Visualization through Tangible Interaction. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 23, 7 pages. https://doi.org/10.1145/3544549.3585797
  66. Tangible Globes for Data Visualisation in Augmented Reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (<conf-loc>, <city>New Orleans</city>, <state>LA</state>, <country>USA</country>, </conf-loc>) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 505, 16 pages. https://doi.org/10.1145/3491102.3517715
  67. Orit Shaer and Eva Hornecker. 2010. Tangible User Interfaces: Past, Present, and Future Directions. Foundations and Trends® in Human–Computer Interaction 3, 1–2 (2010), 4–137. https://doi.org/10.1561/1100000026
  68. TurtleGO: Application with Cubes for Children’s Spatial Ability Based on AR Technology. In Virtual, Augmented and Mixed Reality. Applications and Case Studies, Jessie Y.C. Chen and Gino Fragomeni (Eds.). Springer International Publishing, Cham, 372–383.
  69. Unity Asset Store. 2023. E2Chart. https://assetstore.unity.com/packages/tools/gui/e2chart-147893. License type: Extension Asset. File size: 3.1 MB. Latest version: 3.2. Original Unity version: 2019.4.30 or higher. License agreement: Standard Unity Asset Store EULA.
  70. Exploring Interactions with Printed Data Visualizations in Augmented Reality. IEEE transactions on visualization and computer graphics 29, 1 (2023), 418–428. https://doi.org/10.1109/TVCG.2022.3209386
  71. John Underkoffler and Hiroshi Ishii. 1999. Urp: A Luminous-Tangible Workbench for Urban Planning and Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI ’99). Association for Computing Machinery, New York, NY, USA, 386–393. https://doi.org/10.1145/302979.303114
  72. Design for Playfulness with Interactive Soft Materials: Description Document. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction (Salzburg, Austria) (TEI ’21). Association for Computing Machinery, New York, NY, USA, Article 67, 4 pages. https://doi.org/10.1145/3430524.3442702
  73. Exploring the design space of gestural interaction with active tokens through user-defined gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 4107–4116. https://doi.org/10.1145/2556288.2557373
  74. I3-Eye-Cube: Interactive Intuitive Mixed Reality Interface for Virtual Architecture. In Computing in Architecture / Re-Thinking the Discourse: The Second International Conference of the Arab Society for Computer Aided Architectural Design (ASCAAD 2006) (25-27 April) (ASCAAD). The Arab Society for Computer Aided Architectural Design, Sharjah, United Arab Emirates, 365–380. https://api.semanticscholar.org/CorpusID:59936978
  75. Let’s Get Vysical: Perceptual Accuracy In Visual and Tactile Encodings. arXiv:2308.04392 [cs.HC]
  76. Toward a Deeper Understanding of the Role of Interaction in Information Visualization. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1224–1231. https://doi.org/10.1109/TVCG.2007.70515
  77. What Is Mixed Reality, Anyway? Considering the Boundaries of Mixed Reality in the Context of Robots. Springer Netherlands, Dordrecht, 1–11. https://doi.org/10.1007/978-94-007-0582-1_1
  78. TimeTables: Embodied Exploration of Immersive Spatio-Temporal Data. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Christchurch, New Zealand, 599–605. https://doi.org/10.1109/VR51125.2022.00080
  79. Magic Story Cube: An Interactive Tangible Interface for Storytelling. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (Singapore) (ACE ’04). Association for Computing Machinery, New York, NY, USA, 364–365. https://doi.org/10.1145/1067343.1067404
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Shuqi He (5 papers)
  2. Haonan Yao (2 papers)
  3. Luyan Jiang (1 paper)
  4. Kaiwen Li (15 papers)
  5. Nan Xiang (4 papers)
  6. Yue Li (219 papers)
  7. Hai-Ning Liang (42 papers)
  8. Lingyun Yu (25 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets