Influence of video content type on the usefulness of reinforcement learning algorithms in DASH systems

Przemyslaw Markiewicz

przemyslaw.markiewicz@pollub.edu.pl
Politechnika Lubelska (Poland)

Sławomir Przyłucki


(Poland)

Abstract

The article presents the result of research on DASH (Dynamic Adaptive Streaming over HTTP) systems. In the proposed solution, the adaptive algorithm is based on the RL (Reinforcement Learning) paradigm. The Pensieve algorithm was chosen as the basis for the tests. This algorithm is widely discussed in the scientific literature and therefore the study and analysis of its properties is useful in a wide range of solutions using DASH. The main contribution of the presented test results to the development of knowledge on video streaming services consists in the analysis of the impact of the characteristics of video materials on the effectiveness of the adaptation process implemented by the developed RL model. The presented results show that this influence should not be omitted in any in-depth analyses of the characteristics of DASH systems.


Keywords:

DASH, reinforcement learning, video streaming, QoE

Cisco Visual Networking Index: Forecast and Methodology 2016-2021, High Efficiency Video Coding (HEVC) Algorithms and Architectures (2017).
  Google Scholar

K. u. R. Laghari, O. Issa, F. Speranza, T. H. Falk, Quality-of-Experience perception for video streaming services: Preliminary subjective and objective results, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, Hollywood, CA, USA, (2012) 1-9.
  Google Scholar

A. Reuban, MPEG-DASH Enhanced Multimedia Streaming, International Journal of Advanced Research in Computer Science and Software Engineering, 4 (2014) 848-851.
  Google Scholar

D. You, S. -H. Kim, D. H. Kim, ATSC 3.0 ROUTE/DASH Signaling for Immersive Media: New Perspectives and Examples, in IEEE Access, 9 (2021) 164503-164509, https://dx.doi.org/10.1109/ACCESS.2021.3133626.
DOI: https://doi.org/10.1109/ACCESS.2021.3133626   Google Scholar

I. Sodagar, The MPEG-DASH Standard for Multimedia Streaming Over the Internet, IEEE MultiMedia, 18 (2011) 62-67, https://doi.org/10.1109/MMUL.2011.71.
DOI: https://doi.org/10.1109/MMUL.2011.71   Google Scholar

O. Izima, R. de Fréin, A. Malik, A Survey of Machine Learning Techniques for Video Quality Prediction from Quality of Delivery Metrics, Electronics, 10 (2021) 2851, https://dx.doi.org/10.3390/electronics10222851.
DOI: https://doi.org/10.3390/electronics10222851   Google Scholar

T-Y. Huang, R. Johari, N. McKeown, M. Trunnell, W. Mark, A buffer-based approach to rate adaptation: Evidence from a Large Video Streaming Service, SIGCOMM Computer Communication Review, New York, NY, USA, 44 (2014) 187-198, https://dx.doi.org/10.1145/2619239.2626296.
DOI: https://doi.org/10.1145/2740070.2626296   Google Scholar

X. Yin, A. Jindal, V. Sekar, B. Sinopoli, A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP, SIGCOMM Computer Communication Review, New York, NY, USA, 45 (2015) 325-338, https://doi.org/10.1145/2829988.2787486.
DOI: https://doi.org/10.1145/2829988.2787486   Google Scholar

K. Spiteri, R. Sitaraman, D. Sparacio, From Theory to Practice: Improving Bitrate Adaptation in the DASH Reference Player, ACM Transactions on Multimedia Computing, Communications, and Applications, New York, NY, USA, 15 (2019) 1-29, https://doi.org/10.1145/3336497.
DOI: https://doi.org/10.1145/3336497   Google Scholar

M. Otterlo, M. Wiering, Reinforcement Learning and Markov Decision Processes, Reinforcement Learning: State of the Art, (2012) 3-42, https://doi.org/10.1145/2829988.2787486.
DOI: https://doi.org/10.1007/978-3-642-27645-3_1   Google Scholar

S. Bhatt, Reinforcement Learning 101, Learn the essentials of Reinforcement Learning!, https://towardsdatascience.com/reinforcement-learning-101-e24b50e1d292, [03.11.2022].
  Google Scholar

H. Mao, R. Netravali, M. Alizadeh, Neural Adaptive Video Streaming with Pensieve, Association for Computing Machinery, Los Angeles, CA, USA, (2017) 197–210, https://doi.org/10.1145/3098822.3098843.
DOI: https://doi.org/10.1145/3098822.3098843   Google Scholar

I. Grondman, L. Busoniu, G. Lopes, R. Babuska, A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients, IEEE Transactions on Systems, Man and Cybernetics Part B-Cybernetics, 42 (2012) 1291-1307, https://doi.org/10.1109/TSMCC.2012.2218595
DOI: https://doi.org/10.1109/TSMCC.2012.2218595   Google Scholar

H. -C. Jang, Y. -C. Huang, H. -A. Chiu, A Study on the Effectiveness of A2C and A3C Reinforcement Learning in Parking Space Search in Urban Areas Problem, International Conference on Information and Communication Technology Convergence, Jeju, South Korea, (2020) 567-571, https://doi.org/10.1109/ICTC49870.2020.9289269.
DOI: https://doi.org/10.1109/ICTC49870.2020.9289269   Google Scholar

J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks, 61 (2015) 85-117, https://doi.org/10.1016/j.neunet.2014.09.003.
DOI: https://doi.org/10.1016/j.neunet.2014.09.003   Google Scholar

Z. Tian, L. Zhao, L. Nie, P. Chen, S. Chen, Deeplive: QoE Optimization for Live Video Streaming through Deep Reinforcement Learning, IEEE 25th International Conference on Parallel and Distributed Systems, Tianjin, China, (2019) 827-831, https://doi.org/10.1109/ICPADS47876.2019.00122.
DOI: https://doi.org/10.1109/ICPADS47876.2019.00122   Google Scholar

Y. Liu, D. Wei, C. Zhang, W. Li, Distributed Bandwidth Allocation Strategy for QoE Fairness of Multiple Video Streams in Bottleneck Links, Future Internet, 14 (2022) 152, https://doi.org/10.3390/fi14050152.
DOI: https://doi.org/10.3390/fi14050152   Google Scholar

A. Dethise, M. Canini, S. Kandula, Cracking Open the Black Box: What Observations Can Tell Us About Reinforcement Learning Agents, Association for Computing Machinery, New York, NY, USA, (2019) 29-36, https://doi.org/10.1145/3341216.3342210.
DOI: https://doi.org/10.1145/3341216.3342210   Google Scholar

H. Riiser, P. Vigmostad, C. Griwodz, P. Halvorsen, Commute Path Bandwidth Traces from 3G Networks: Analysis and Applications, Association for Computing Machinery, New York, NY, USA, (2013) 114-118, https://doi.org/10.1145/2483977.2483991.
DOI: https://doi.org/10.1145/2483977.2483991   Google Scholar

M. Ribeiro, S. Singh, C. Guestrin, “Why Should I Trust You?”: Explaining the Predictions of Any Classifier, Association for Computational Linguistics: Demonstrations, San Diego, California, (2016) 97-101, http://dx.doi.org/10.18653/v1/N16-3020.
DOI: https://doi.org/10.18653/v1/N16-3020   Google Scholar

Raw Data - Measuring Broadband America 2016, Federal Communications Commission, https://www.fcc.gov/reports-research/reports/measuring-broadband-america/, [03.11.2022].
  Google Scholar

ITEC - Dynamic Adaptive Streaming over HTTP, https://dash.itec.aau.at/contact/, [03.11.2022].
  Google Scholar

C. Müller, S. Lederer, C. Timmerer, H. Hellwagner, Dynamic Adaptive Streaming over HTTP/2.0, IEEE International Conference on Multimedia and Expo, (2013) 1-6, http://dx.doi.org/10.1109/ICME.2013.6607498.
DOI: https://doi.org/10.1109/ICME.2013.6607498   Google Scholar

Download


Published
2023-06-30

Cited by

Markiewicz, P., & Przyłucki, S. (2023). Influence of video content type on the usefulness of reinforcement learning algorithms in DASH systems. Journal of Computer Sciences Institute, 27, 162–170. https://doi.org/10.35784/jcsi.3579

Authors

Przemyslaw Markiewicz 
przemyslaw.markiewicz@pollub.edu.pl
Politechnika Lubelska Poland

Authors

Sławomir Przyłucki 

Poland

Statistics

Abstract views: 60
PDF downloads: 111