Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

360TripleView: 360-Degree Video View Management System Driven by Convergence Value of Viewing Preferences (2306.08089v2)

Published 13 Jun 2023 in cs.MM

Abstract: 360-degree video has become increasingly popular in content consumption. However, finding the viewing direction for important content within each frame poses a significant challenge. Existing approaches rely on either viewer input or algorithmic determination to select the viewing direction, but neither mode consistently outperforms the other in terms of content-importance. In this paper, we propose 360TripleView, the first view management system for 360-degree video that automatically infers and utilizes the better view mode for each frame, ultimately providing viewers with higher content-importance views. Through extensive experiments and a user study, we demonstrate that 360TripleView achieves over 90\% accuracy in inferring the better mode and significantly enhances content-importance compared to existing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. The prefetch aggressiveness tradeoff in 360 video streaming. In Proceedings of the 9th ACM Multimedia Systems Conference, pp. 258–269, 2018.
  2. Enhanced interactive 360 viewing via automatic guidance. ACM Transactions on Graphics (TOG), 39(5):1–15, 2020.
  3. Full uhd 360-degree video dataset and modeling of rate-distortion characteristics and head movement navigation. In Proceedings of the 12th ACM Multimedia Systems Conference, pp. 267–273, 2021.
  4. Transformer-based long-term viewport prediction in 360° video: Scanpath is all you need. In MMSP, pp. 1–6, 2021.
  5. Cube padding for weakly-supervised saliency prediction in 360 videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1420–1429, 2018.
  6. 360-degree video head movement dataset. In Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 199–204, 2017.
  7. Atsal: An attention based architecture for saliency prediction in 360 videos. In International Conference on Pattern Recognition, pp. 305–320. Springer, 2021.
  8. A dataset of head and eye movements for 360 videos. In Proceedings of the 9th ACM Multimedia Systems Conference, pp. 432–437, 2018.
  9. Avtrack360: an open dataset and software recording people’s head rotations watching 360° videos on an hmd. In Proceedings of the 9th ACM Multimedia Systems Conference, pp. 403–408, 2018.
  10. Pem360: A dataset of 360 videos with continuous physiological measurements, subjective emotional ratings and motion traces. In Proceedings of the 13th ACM Multimedia Systems Conference, pp. 252–258, 2022.
  11. Deep variational learning for multiple trajectory prediction of 360° head movements. In Proceedings of the 13th ACM Multimedia Systems Conference, pp. 12–26, 2022.
  12. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
  13. Spatial audio in 360° videos: does it influence visual attention? In Proceedings of the 13th ACM Multimedia Systems Conference, pp. 39–51, 2022.
  14. Deep 360 pilot: Learning a deep agent for piloting through 360 sports videos. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1396–1405. IEEE, 2017.
  15. Where are you looking? a large-scale dataset of head and gaze behavior for 360-degree videos and a pilot study. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 1025–1034, 2022.
  16. K. Kang and S. Cho. Interactive and automatic navigation for 360 video playback. ACM Transactions on Graphics (TOG), 38(4):1–11, 2019.
  17. Semantic-driven generation of hyperlapse from 360 degree video. IEEE transactions on visualization and computer graphics, 24(9):2610–2621, 2017.
  18. A memory network approach for story-based temporal summarization of 360 videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1410–1419, 2018.
  19. Tell me where to look: Investigating ways for assisting focus in 360 video. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 2535–2545, 2017.
  20. Outside-in: Visualizing out-of-sight regions-of-interest in a 360 video using spatial picture-in-picture previews. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, pp. 255–265, 2017.
  21. 360 video viewing dataset in head-mounted virtual reality. In Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 211–216, 2017.
  22. A taxonomy and dataset for 360 videos. In Proceedings of the 10th ACM Multimedia Systems Conference, pp. 273–278, 2019.
  23. Viewport prediction for 360 videos: a clustering approach. In Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, pp. 34–39, 2020.
  24. A. Nguyen and Z. Yan. A saliency dataset for 360-degree videos. In Proceedings of the 10th ACM Multimedia Systems Conference, pp. 279–284, 2019.
  25. J. Park and K. Nahrstedt. Navigation graph for tiled media streaming. In Proceedings of the 27th ACM International Conference on Multimedia, pp. 447–455, 2019.
  26. Seaware: Semantic aware view prediction system for 360-degree video streaming. In 2020 IEEE International Symposium on Multimedia (ISM), pp. 57–64. IEEE, 2020.
  27. Shot orientation controls for interactive cinematography with 360 video. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, pp. 289–297, 2017.
  28. Flare: Practical viewport-adaptive 360-degree video streaming for mobile devices. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking, pp. 99–114, 2018.
  29. A dataset of head and eye movements for 360 degree images. In Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 205–210, 2017.
  30. Track: A new method from a re-examination of deep architectures for head motion prediction in 360 videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):5681–5699, 2021.
  31. Spherical clustering of users navigating 360 content. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4020–4024. IEEE, 2019.
  32. S. Rossi and L. Toni. Understanding user navigation in immersive experience: An information-theoretic analysis. In Proceedings of the 12th ACM International Workshop on Immersive Mixed and Virtual Environment Systems, pp. 19–24, 2020.
  33. A. Samiei and R. Prakash. Improving 360-degree video field-of-view prediction and edge caching. In 2021 IEEE International Symposium on Multimedia (ISM), pp. 9–16. IEEE, 2021.
  34. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  35. Saliency in vr: How do people explore virtual environments? IEEE transactions on visualization and computer graphics, 24(4):1633–1642, 2018.
  36. Y.-C. Su and K. Grauman. Making 360 video watchable in 2d: Learning videography for click free viewing. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1368–1376. IEEE, 2017.
  37. Pano2vid: Automatic cinematography for watching 360 videos. In Asian Conference on Computer Vision, pp. 154–171. Springer, 2016.
  38. Flocking-based live streaming of 360-degree video. In Proceedings of the 11th ACM Multimedia Systems Conference, pp. 26–37, 2020.
  39. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826, 2016.
  40. Semantic-aware view prediction for 360-degree videos at the 5g edge. In 2022 IEEE International Symposium on Multimedia (ISM), pp. 121–128. IEEE, 2022.
  41. Attention-based deep reinforcement learning for virtual cinematography of 360 videos. IEEE Transactions on Multimedia, 23:3227–3238, 2020.
  42. Transitioning360: Content-aware nfov virtual camera paths for 360° video playback. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 185–194. IEEE, 2020.
  43. A dataset for exploring user behaviors in vr spherical video streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 193–198, 2017.
  44. A deep ranking model for spatio-temporal highlight detection from a 360◦ video. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, 2018.
  45. 360broadview: Viewer management for viewport prediction in 360-degree video live broadcast. In Proceedings of the 4th ACM International Conference on Multimedia in Asia, pp. 1–7, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now