Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Perceptual Quality Assessment of Virtual Reality Videos in the Wild (2206.08751v3)

Published 13 Jun 2022 in cs.CV and eess.IV

Abstract: Investigating how people perceive virtual reality (VR) videos in the wild (i.e., those captured by everyday users) is a crucial and challenging task in VR-related applications due to complex authentic distortions localized in space and time. Existing panoramic video databases only consider synthetic distortions, assume fixed viewing conditions, and are limited in size. To overcome these shortcomings, we construct the VR Video Quality in the Wild (VRVQW) database, containing $502$ user-generated videos with diverse content and distortion characteristics. Based on VRVQW, we conduct a formal psychophysical experiment to record the scanpaths and perceived quality scores from $139$ participants under two different viewing conditions. We provide a thorough statistical analysis of the recorded data, observing significant impact of viewing conditions on both human scanpaths and perceived quality. Moreover, we develop an objective quality assessment model for VR videos based on pseudocylindrical representation and convolution. Results on the proposed VRVQW show that our method is superior to existing video quality assessment models. We have made the database and code available at https://github.com/limuhit/VR-Video-Quality-in-the-Wild.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. R. G. d. A. Azevedo, N. Birkbeck, F. De Simone, I. Janatra, B. Adsumilli, and P. Frossard, “Visual distortions in 360° videos,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 8, pp. 2524–2537, 2020.
  2. Z. Wang, L. Lu, and A. C. Bovik, “Video quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, vol. 19, no. 2, pp. 121–132, 2004.
  3. Y. Wang, T. Jiang, S. Ma, and W. Gao, “Novel spatio-temporal structural information based video quality metric,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 7, pp. 989–998, 2012.
  4. M. Xu, J. Chen, H. Wang, S. Liu, G. Li, and Z. Bai, “C3DVQA: Full-reference video quality assessment with 3D convolutional neural network,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2020, pp. 4447–4451.
  5. D. Li, T. Jiang, and M. Jiang, “Quality assessment of in-the-wild videos,” in ACM Multimedia, 2019, pp. 2351–2359.
  6. J. Korhonen, “Two-level approach for no-reference consumer video quality assessment,” IEEE Transactions on Image Processing, vol. 28, no. 12, pp. 5923–5938, 2019.
  7. Z. Ying, M. Mandal, D. Ghadiyaram, and A. C. Bovik, “Patch-VQ: ‘Patching Up’ the video quality problem,” in IEEE Conference on Computer Vision and Pattern Recognition, 2021, pp. 14019–14029.
  8. F. Götz-Hahn, V. Hosu, H. Lin, and D. Saupe, “KonVid-150k: A dataset for no-reference video quality assessment of videos in-the-wild,” IEEE Access, vol. 9, pp. 72139–72160, 2021.
  9. A. Singla, S. Fremerey, W. Robitza, P. Lebreton, and A. Raake, “Comparison of subjective quality evaluation for HEVC encoded omnidirectional videos at different bit-rates for UHD and FHD resolution,” in ACM Multimedia Workshops, 2017, pp. 511–519.
  10. I. D. D. Curcio, H. Toukomaa, and D. Naik, “Bandwidth reduction of omnidirectional viewport-dependent video streaming via subjective quality assessment,” in International Workshop on Multimedia Alternate Realities, 2017, pp. 9–14.
  11. H. T. T. Tran, N. P. Ngoc, C. M. Bui, M. H. Pham, and T. C. Thang, “An evaluation of quality metrics for 360 videos,” in International Conference on Ubiquitous and Future Networks, 2017, pp. 7–11.
  12. H. Duan, G. Zhai, X. Yang, D. Li, and W. Zhu, “IVQAD 2017: An immersive video quality assessment database,” in International Conference on Systems, Signals and Image Processing, 2017, pp. 1–5.
  13. B. Zhang, J. Zhao, S. Yang, Y. Zhang, J. Wang, and Z. Fei, “Subjective and objective quality assessment of panoramic videos in virtual reality environments,” in IEEE International Conference on Multimedia Expo Workshops, 2017, pp. 163–168.
  14. Y. Zhang, Y. Wang, F. Liu, Z. Liu, Y. Li, D. Yang, and Z. Chen, “Subjective panoramic video quality assessment database for coding applications,” IEEE Transactions on Broadcasting, vol. 64, no. 2, pp. 461–473, 2018.
  15. F. Lopes, J. Ascenso, A. Rodrigues, and M. P. Queluz, “Subjective and objective quality assessment of omnidirectional video,” in Applications of Digital Image Processing, 2018, pp. 249–265.
  16. C. Li, M. Xu, X. Du, and Z. Wang, “Bridge the gap between VQA and human behavior on omnidirectional video: A large-scale dataset and a deep learning model,” in ACM Multimedia, 2018, pp. 932–940.
  17. Y. Meng and Z. Ma, “Viewport-based omnidirectional video quality assessment: Database, modeling and inference,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 1, pp. 120–134, 2022.
  18. M. Li, K. Ma, J. Li, and D. Zhang, “Pseudocylindrical convolutions for learned omnidirectional image compression,” arXiv preprint arXiv:2112.13227, 2021.
  19. M. Graf, C. Timmerer, and C. Mueller, “Towards bandwidth efficient adaptive streaming of omnidirectional video over HTTP: Design, implementation, and evaluation,” in ACM on Multimedia Systems Conference, 2017, pp. 261–271.
  20. X. Sui, K. Ma, Y. Yao, and Y. Fang, “Perceptual quality assessment of omnidirectional images as moving camera videos,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 8, pp. 3022–3034, 2021.
  21. Y. Sun, A. Lu, and L. Yu, “Weighted-to-spherically-uniform quality evaluation for omnidirectional video,” IEEE Signal Processing Letters, vol. 24, no. 9, pp. 1408–1412, 2017.
  22. V. Zakharchenko, K. P. Choi, and J. H. Park, “Quality metric for spherical panoramic video,” in Optics and Photonics for Information Processing, 2016, pp. 57–65.
  23. H. G. Kim, H.-T. Lim, and Y. M. Ro, “Deep virtual reality image quality assessment with human perception guider for omnidirectional image,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 4, pp. 917–928, 2019.
  24. M. Yu, H. Lakshman, and B. Girod, “A framework to evaluate omnidirectional video coding schemes,” in IEEE International Symposium on Mixed and Augmented Reality, 2015, pp. 31–36.
  25. S. Chen, Y. Zhang, Y. Li, Z. Chen, and Z. Wang, “Spherical structural similarity index for objective omnidirectional video quality assessment,” in IEEE International Conference on Multimedia and Expo, 2018, pp. 1–6.
  26. J. Xu, W. Zhou, and Z. Chen, “Blind omnidirectional image quality assessment with viewport oriented graph convolutional networks,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 5, pp. 1724–1737, 2021.
  27. C. Li, M. Xu, L. Jiang, S. Zhang, and X. Tao, “Viewport proposal CNN for 360° video quality assessment,” in IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 10169–10178.
  28. T. S. Cohen, M. Geiger, J. Köhler, and M. Welling, “Spherical CNNs,” in International Conference on Learning Representations, 2018, pp. 1–15.
  29. H. G. Kim, H.-T. Lim, S. Lee, and Y. M. Ro, “VRSA Net: VR sickness assessment considering exceptional motion for 360° VR video,” IEEE Transactions on Image Processing, vol. 28, no. 4, pp. 1646–1660, 2019.
  30. R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, “Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness,” The International Journal of Aviation Psychology, vol. 3, no. 3, pp. 203–220, 1993.
  31. D. Ghadiyaram and A. C. Bovik, “Massive online crowdsourced study of subjective and objective picture quality,” IEEE Transactions on Image Processing, vol. 25, no. 1, pp. 372–387, 2016.
  32. K. Zeng, T. Zhao, A. Rehman, and Z. Wang, “Characterizing perceptual artifacts in compressed video streams,” in SPIE, 2014, pp. 173–182.
  33. ITU-R BT.500-13, “Methodology for the subjective assessment of the quality of television pictures,” 2012. [Online]. Available: https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.500-13-201201-S!!PDF-E.pdf
  34. Z. Duanmu, K. Ma, and Z. Wang, “Quality-of-experience for adaptive streaming videos: An expectation confirmation theory motivated approach,” IEEE Transactions on Image Processing, vol. 27, no. 12, pp. 6135–6146, 2018.
  35. VQEG, “Final report from the Video Quality Experts Group on the validation of objective models of video quality assessment,” 2000. [Online]. Available: https://www.its.bldrdoc.gov/media/8212/frtv_phase1_final_report.doc
  36. V. Hosu, H. Lin, T. Sziranyi, and D. Saupe, “KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment,” IEEE Transactions on Image Processing, vol. 29, pp. 4041–4056, 2020.
  37. V. Sitzmann, A. Serrano, A. Pavel, M. Agrawala, D. Gutierrez, B. Masia, and G. Wetzstein, “Saliency in VR: How do people explore virtual environments?” IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 4, pp. 1633–1642, 2018.
  38. N. C. Anderson, F. Anderson, A. Kingstone, and W. F. Bischof, “A comparison of scanpath comparison methods,” Behavior Research Methods, vol. 47, no. 4, pp. 1377–1392, 2015.
  39. I. D. D. Curcio, H. Toukomaa, and D. Naik, “360-Degree video streaming and its subjective quality,” in SMPTE Annual Technical Conference and Exhibition, 2017, pp. 1–23.
  40. M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density based algorithm for discovering density varied clusters in large spatial databases,” in International Conference on Knowledge Discovery and Data Mining, 1996, pp. 226–231.
  41. F.-Y. Chao, C. Ozcinar, C. Wang, E. Zerman, L. Zhang, W. Hamidouche, O. Deforges, and A. Smolic, “Audio-visual perception of omnidirectional video for virtual reality applications,” in IEEE International Conference on Multimedia Expo Workshops, 2020, pp. 1–6.
  42. M. Xu, Y. Song, J. Wang, M. Qiao, L. Huo, and Z. Wang, “Predicting head movement in panoramic video: A deep reinforcement learning approach,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 11, pp. 2693–2708, 2019.
  43. Y. Fang, Y. Yao, X. Sui, and K. Ma, “Subjective quality assessment of user-generated 360∘superscript360360^{\circ}360 start_POSTSUPERSCRIPT ∘ end_POSTSUPERSCRIPT videos,” in IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, 2023, pp. 723–724.
  44. D. S. Hands and S. E. Avons, “Recency and duration neglect in subjective assessment of television picture quality,” Applied Cognitive Psychology, vol. 15, no. 6, pp. 639–657, 2001.
  45. M. Yu, H. Lakshman, and B. Girod, “Content adaptive representations of omnidirectional videos for cinematic virtual reality,” in International Workshop on Immersive Media Experiences, 2015, pp. 1–6.
  46. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
  47. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
  48. Z. Tu, C.-J. Chen, L.-H. Chen, N. Birkbeck, B. Adsumilli, and A. C. Bovik, “A comparative evaluation of temporal pooling methods for blind video quality assessment,” in IEEE International Conference on Image Processing, 2020, pp. 141–145.
  49. A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a ‘completely blind’ image quality analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013.
  50. B. Li, W. Zhang, M. Tian, G. Zhai, and X. Wang, “Blindly assess quality of in-the-wild videos via quality-aware pre-training and motion perception,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 9, pp. 5944–5958, 2022.
  51. W. Sun, X. Min, G. Zhai, K. Gu, H. Duan, and S. Ma, “MC360IQA: A multi-channel CNN for blind 360-degree image quality assessment,” IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 1, pp. 64–77, 2020.
  52. M. Xu, Y. Song, J. Wang, M. Qiao, L. Huo, and Z. Wang, “Predicting head movement in panoramic video: A deep reinforcement learning approach,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 11, pp. 2693–2708, 2018.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now