Influence of video content type on the usefulness of reinforcement learning algorithms in DASH systems
Przemyslaw Markiewicz
przemyslaw.markiewicz@pollub.edu.plPolitechnika Lubelska (Poland)
Sławomir Przyłucki
(Poland)
Abstract
The article presents the result of research on DASH (Dynamic Adaptive Streaming over HTTP) systems. In the proposed solution, the adaptive algorithm is based on the RL (Reinforcement Learning) paradigm. The Pensieve algorithm was chosen as the basis for the tests. This algorithm is widely discussed in the scientific literature and therefore the study and analysis of its properties is useful in a wide range of solutions using DASH. The main contribution of the presented test results to the development of knowledge on video streaming services consists in the analysis of the impact of the characteristics of video materials on the effectiveness of the adaptation process implemented by the developed RL model. The presented results show that this influence should not be omitted in any in-depth analyses of the characteristics of DASH systems.
Keywords:
DASH, reinforcement learning, video streaming, QoEReferences
Cisco Visual Networking Index: Forecast and Methodology 2016-2021, High Efficiency Video Coding (HEVC) Algorithms and Architectures (2017).
Google Scholar
K. u. R. Laghari, O. Issa, F. Speranza, T. H. Falk, Quality-of-Experience perception for video streaming services: Preliminary subjective and objective results, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, Hollywood, CA, USA, (2012) 1-9.
Google Scholar
A. Reuban, MPEG-DASH Enhanced Multimedia Streaming, International Journal of Advanced Research in Computer Science and Software Engineering, 4 (2014) 848-851.
Google Scholar
D. You, S. -H. Kim, D. H. Kim, ATSC 3.0 ROUTE/DASH Signaling for Immersive Media: New Perspectives and Examples, in IEEE Access, 9 (2021) 164503-164509, https://dx.doi.org/10.1109/ACCESS.2021.3133626.
DOI: https://doi.org/10.1109/ACCESS.2021.3133626
Google Scholar
I. Sodagar, The MPEG-DASH Standard for Multimedia Streaming Over the Internet, IEEE MultiMedia, 18 (2011) 62-67, https://doi.org/10.1109/MMUL.2011.71.
DOI: https://doi.org/10.1109/MMUL.2011.71
Google Scholar
O. Izima, R. de Fréin, A. Malik, A Survey of Machine Learning Techniques for Video Quality Prediction from Quality of Delivery Metrics, Electronics, 10 (2021) 2851, https://dx.doi.org/10.3390/electronics10222851.
DOI: https://doi.org/10.3390/electronics10222851
Google Scholar
T-Y. Huang, R. Johari, N. McKeown, M. Trunnell, W. Mark, A buffer-based approach to rate adaptation: Evidence from a Large Video Streaming Service, SIGCOMM Computer Communication Review, New York, NY, USA, 44 (2014) 187-198, https://dx.doi.org/10.1145/2619239.2626296.
DOI: https://doi.org/10.1145/2740070.2626296
Google Scholar
X. Yin, A. Jindal, V. Sekar, B. Sinopoli, A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP, SIGCOMM Computer Communication Review, New York, NY, USA, 45 (2015) 325-338, https://doi.org/10.1145/2829988.2787486.
DOI: https://doi.org/10.1145/2829988.2787486
Google Scholar
K. Spiteri, R. Sitaraman, D. Sparacio, From Theory to Practice: Improving Bitrate Adaptation in the DASH Reference Player, ACM Transactions on Multimedia Computing, Communications, and Applications, New York, NY, USA, 15 (2019) 1-29, https://doi.org/10.1145/3336497.
DOI: https://doi.org/10.1145/3336497
Google Scholar
M. Otterlo, M. Wiering, Reinforcement Learning and Markov Decision Processes, Reinforcement Learning: State of the Art, (2012) 3-42, https://doi.org/10.1145/2829988.2787486.
DOI: https://doi.org/10.1007/978-3-642-27645-3_1
Google Scholar
S. Bhatt, Reinforcement Learning 101, Learn the essentials of Reinforcement Learning!, https://towardsdatascience.com/reinforcement-learning-101-e24b50e1d292, [03.11.2022].
Google Scholar
H. Mao, R. Netravali, M. Alizadeh, Neural Adaptive Video Streaming with Pensieve, Association for Computing Machinery, Los Angeles, CA, USA, (2017) 197–210, https://doi.org/10.1145/3098822.3098843.
DOI: https://doi.org/10.1145/3098822.3098843
Google Scholar
I. Grondman, L. Busoniu, G. Lopes, R. Babuska, A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients, IEEE Transactions on Systems, Man and Cybernetics Part B-Cybernetics, 42 (2012) 1291-1307, https://doi.org/10.1109/TSMCC.2012.2218595
DOI: https://doi.org/10.1109/TSMCC.2012.2218595
Google Scholar
H. -C. Jang, Y. -C. Huang, H. -A. Chiu, A Study on the Effectiveness of A2C and A3C Reinforcement Learning in Parking Space Search in Urban Areas Problem, International Conference on Information and Communication Technology Convergence, Jeju, South Korea, (2020) 567-571, https://doi.org/10.1109/ICTC49870.2020.9289269.
DOI: https://doi.org/10.1109/ICTC49870.2020.9289269
Google Scholar
J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks, 61 (2015) 85-117, https://doi.org/10.1016/j.neunet.2014.09.003.
DOI: https://doi.org/10.1016/j.neunet.2014.09.003
Google Scholar
Z. Tian, L. Zhao, L. Nie, P. Chen, S. Chen, Deeplive: QoE Optimization for Live Video Streaming through Deep Reinforcement Learning, IEEE 25th International Conference on Parallel and Distributed Systems, Tianjin, China, (2019) 827-831, https://doi.org/10.1109/ICPADS47876.2019.00122.
DOI: https://doi.org/10.1109/ICPADS47876.2019.00122
Google Scholar
Y. Liu, D. Wei, C. Zhang, W. Li, Distributed Bandwidth Allocation Strategy for QoE Fairness of Multiple Video Streams in Bottleneck Links, Future Internet, 14 (2022) 152, https://doi.org/10.3390/fi14050152.
DOI: https://doi.org/10.3390/fi14050152
Google Scholar
A. Dethise, M. Canini, S. Kandula, Cracking Open the Black Box: What Observations Can Tell Us About Reinforcement Learning Agents, Association for Computing Machinery, New York, NY, USA, (2019) 29-36, https://doi.org/10.1145/3341216.3342210.
DOI: https://doi.org/10.1145/3341216.3342210
Google Scholar
H. Riiser, P. Vigmostad, C. Griwodz, P. Halvorsen, Commute Path Bandwidth Traces from 3G Networks: Analysis and Applications, Association for Computing Machinery, New York, NY, USA, (2013) 114-118, https://doi.org/10.1145/2483977.2483991.
DOI: https://doi.org/10.1145/2483977.2483991
Google Scholar
M. Ribeiro, S. Singh, C. Guestrin, “Why Should I Trust You?”: Explaining the Predictions of Any Classifier, Association for Computational Linguistics: Demonstrations, San Diego, California, (2016) 97-101, http://dx.doi.org/10.18653/v1/N16-3020.
DOI: https://doi.org/10.18653/v1/N16-3020
Google Scholar
Raw Data - Measuring Broadband America 2016, Federal Communications Commission, https://www.fcc.gov/reports-research/reports/measuring-broadband-america/, [03.11.2022].
Google Scholar
ITEC - Dynamic Adaptive Streaming over HTTP, https://dash.itec.aau.at/contact/, [03.11.2022].
Google Scholar
C. Müller, S. Lederer, C. Timmerer, H. Hellwagner, Dynamic Adaptive Streaming over HTTP/2.0, IEEE International Conference on Multimedia and Expo, (2013) 1-6, http://dx.doi.org/10.1109/ICME.2013.6607498.
DOI: https://doi.org/10.1109/ICME.2013.6607498
Google Scholar
Authors
Sławomir PrzyłuckiPoland
Statistics
Abstract views: 60PDF downloads: 111
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.