An Accurate Viewport Estimation Method for 360 Video Streaming using Deep Learning
DOI:
https://doi.org/10.4108/eetinis.v9i4.2218Abstract
Nowadays, Virtual Reality is becoming more and more popular, and 360 video is a very important part of the system. 360 video transmission over the Internet faces many difficulties due to its large size. Therefore, to reduce the network bandwidth requirement of 360-degree video, Viewport Adaptive Streaming (VAS) was proposed. An important issue in VAS is how to estimate future user viewing direction. In this paper, we propose an algorithm called GLVP (GRU-LSTM-based-Viewport-Prediction) to estimate the typical view for the VAS system. The results show that our method can improve viewport estimation from 9.5% to near 20%compared with other methods.
Downloads
References
D. V. Nguyen, H. T. T. Tran, and T. C. Thang, “An evaluation of tile selection methods for viewport-adaptive streaming of 360-degree video,” ACM Trans. Multimedia Comput. Commun. Appl., vol. 16, no. 1, mar 2020. [Online]. Available: https://doi.org/10.1145/3373359 DOI: https://doi.org/10.1145/3373359
D. V. Nguyen, H. T. Tran, and T. C. Thang, “Impact of delays on 360-degree video communications,” in 2017 TRON Symposium (TRONSHOW). IEEE, 2017, pp. 1–6. DOI: https://doi.org/10.23919/TRONSHOW.2017.8275075
F. Qian, L. Ji, B. Han, and V. Gopalakrishnan, “Optimizing 360 video delivery over cellular networks,” in Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges, ser. ATC ’16. New York, NY, USA: Association for Computing [4] Machinery, 2016, p. 1–6. [Online]. Available: https://doi.org/10.1145/2980055.2980056 DOI: https://doi.org/10.1145/2980055.2980056
Y. Bao, H. Wu, T. Zhang, A. A. Ramli, and X. Liu, “Shooting a moving target: Motion-prediction-based transmission for 360-degree videos,” in 2016 IEEE International Conference on Big Data (Big Data), 2016, pp. 1161–1170. DOI: https://doi.org/10.1109/BigData.2016.7840720
C.-L. Fan, J. Lee, W.-C. Lo, C.-Y. Huang, K.-T. Chen, and C.-H. Hsu, “Fixation prediction for 360 video streaming in head-mounted virtual reality,” in Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video, 2017, pp. 67–72. DOI: https://doi.org/10.1145/3083165.3083180
Y. Ban, L. Xie, Z. Xu, X. Zhang, Z. Guo, and Y. Wang, “: Exploiting cross-users behaviors for viewport prediction in 360 video adaptive streaming,” in 2018 IEEE International Conference on Multimedia and Expo (ICME), 2018, pp. 1–6. DOI: https://doi.org/10.1109/ICME.2018.8486606
C. Wu, R. Zhang, Z. Wang, and L. Sun, “A spherical convolution approach for learning long term viewport prediction in 360 immersive video,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, 2020, pp. 14 003–14 040. DOI: https://doi.org/10.1609/aaai.v34i01.7377
M. Hosseini and V. Swaminathan, “Adaptive 360 vr video streaming: Divide and conquer,” in 2016 IEEE International Symposium on Multimedia (ISM), 2016, pp. 107–110. DOI: https://doi.org/10.1109/ISM.2016.0028
D. V. Nguyen, H. T. T. Tran, A. T. Pham, and T. C. Thang, “A new adaptation approach for viewport-adaptive 360-degree video streaming,” in 2017 IEEE International Symposium on Multimedia (ISM), 2017, pp. 38–44. DOI: https://doi.org/10.1109/ISM.2017.16
A. Zare, A. Aminlou, M. M. Hannuksela, and M. Gabbouj, “Hevc-compliant tile-based streaming of panoramic video for virtual reality applications,” in Proceedings of the 24th ACM International Conference on Multimedia, ser. MM ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 601–605.[Online]. Available: https://doi.org/10.1145/2964284. 2967292 DOI: https://doi.org/10.1145/2964284.2967292
D. V. Nguyen, H. T. T. Tran, A. T. Pham, and T. C. Thang, “An optimal tile-based approach for viewport-adaptive 360-degree video streaming,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 1, pp. 29–42, 2019. DOI: https://doi.org/10.1109/JETCAS.2019.2899488
S. Petrangeli, V. Swaminathan, M. Hosseini, and
F. De Turck, “An http/2-based adaptive streaming framework for 360° virtual reality videos,” in Proceedings of the 25th ACM International Conference on Multimedia, ser. MM ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 306–314. [Online]. Available: https://doi.org/10.1145/3123266.3123453 DOI: https://doi.org/10.1145/3123266.3123453
Z. Xu, X. Zhang, K. Zhang, and Z. Guo, “Probabilistic viewport adaptive streaming for 360-degree videos,” in 2018 IEEE International Symposium on Circuits and Systems (ISCAS), 2018, pp. 1–5. DOI: https://doi.org/10.1109/ISCAS.2018.8351404
C. Li, W. Zhang, Y. Liu, and Y. Wang, “Very long term field of view prediction for 360-degree video streaming,” in 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE, 2019, pp. 297–302. DOI: https://doi.org/10.1109/MIPR.2019.00060
X. Feng, V. Swaminathan, and S. Wei, “Viewport prediction for live 360-degree mobile video streaming using user-content hybrid motion tracking,” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 3, no. 2, jun 2019. [Online]. Available: https://doi.org/10.1145/3328914 DOI: https://doi.org/10.1145/3328914
J. Heyse, M. T. Vega, F. de Backere, and F. de Turck, “Contextual bandit learning-based viewport prediction for 360 video,” in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019, pp. 972–973. DOI: https://doi.org/10.1109/VR.2019.8797830
L. Sun, Y. Mao, T. Zong, Y. Liu, and Y. Wang, “Flocking-based live streaming of 360-degree video,” in Proceedings of the 11th ACM Multimedia Systems Conference, ser. MMSys ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 26–37. [Online]. Available: https://doi.org/10.1145/3339825.3391856 DOI: https://doi.org/10.1145/3339825.3391856
A. T. Nasrabadi, A. Samiei, and R. Prakash, “Viewport prediction for 360° videos: A clustering approach,” in Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, ser. NOSSDAV ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 34–39. [Online]. Available: https://doi.org/10.1145/3386290.3396934 DOI: https://doi.org/10.1145/3386290.3396934
J. Park, M. Wu, K.-Y. Lee, B. Chen, K. Nahrstedt, M. Zink, and R. Sitaraman, “Seaware: Semantic aware view prediction system for 360-degree video streaming,” in 2020 IEEE International Symposium on Multimedia (ISM), 2020, pp. 57–64. DOI: https://doi.org/10.1109/ISM.2020.00016
T. C. Thang, H. T. Le, A. T. Pham, and Y. M. Ro, “An evaluation of bitrate adaptation methods for http live streaming,” IEEE Journal on Selected Areas in Communications, vol. 32, no. 4, pp. 693–705, April 2014. DOI: https://doi.org/10.1109/JSAC.2014.140403
X. Corbillon, F. De Simone, and G. Simon, “360-degree video head movement dataset,” in Proceedings of the 8th ACM on Multimedia Systems Conference, ser. MMSys’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 199–204. [Online]. Available: https://doi.org/10.1145/3083187.3083215 DOI: https://doi.org/10.1145/3083187.3083215
D. Nguyen, “An evaluation of viewport estimation methods in 360-degree video streaming,” in 2022 7th International Conference on Business and Industrial Research (ICBIR), 2022, pp. 161–166. DOI: https://doi.org/10.1109/ICBIR54589.2022.9786513
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
