Measuring the Effectiveness of Autonomous Network Functions that Use Artificial Intelligence Algorithms
DOI:
https://doi.org/10.13052/nbjict1902-097X.2020.014Keywords:
Network assurance, AI, ML, counters, stochastic algorithms, KPIsAbstract
Autonomous network functions such as Software Defined Networks (SDN), Self-Organizing Networks (SON), and virtual function network orchestrator plays a crucial role in 5G and beyond 5G wireless telecommunication network. Advancements in Artificial Intelligence (AI), Machine Learning (ML) algorithms, and frameworks have led to adequate adaption of stochastic algorithms for autonomous network functions, aimed at performing better than human capability. Measuring the effectiveness of such autonomous network functions is a challenge since stochastic algorithms are fundamentally generalized models and could potentially make malicious proposals. Traditionally effectiveness of network is measured through network assurance Key Performance Indicators (KPIs). Autonomous network functions are kept active when the KPIs are in the acceptable limit, and the network is showing improvement over time. This paper introduces
• Factors that are to be considered beyond KPIs for effective measurement of autonomous network functions that use stochastic algorithms.
• Adopting the right scales for measuring the effectiveness of autonomous network functions using the grading system from medical practices which are used for the treatment of critical illness.
References
Scott Mayer McKinney, Marcin Sieniek, Varun Godbole, Jonathan Godwin, Natasha Antropova, Hutan Ashrafian, Trevor Back, Mary Chesus, Greg C. Corrado, and Ara et al. Darzi. “International evaluation of an ai system for breast cancer screening”. Nature, 577.7788 (2020), pp. 89–94. DOI: 10.1038/s41586-019-1799-6.
Premnath K Narayanan and David K Harrison. “Explainable AI for Autonomous Network Functions in Wireless and Mobile Networks”. International Journal of Wireless and Mobile Networks, 12.3 (2020), pp. 31–44. DOI: 10.5121/ijwmn.2020.12303. URL: https://aircconline.com/ijwmn/V12N3/12320ijwmn03.pdf.
Scott M. Lundberg and Su-In Lee”. “A Unified Approach to Interpreting Model Predictions”. In: 31st Conference on Neural Information Processing Systems. NIPS, 2017.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why Should I Trust You?” Explaining the Predictions of Any Classifier”. In: 22nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 2016.
Gregory Plumb, Denali Molitor, and Ameet Talwalkar. “Model Agnostic Supervised Local Explanations”. In: 32nd Conference on Neural Information Processing Systems. NeurIPS, 2018.
CJ Willmott and K Matsuura. “Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance”. In: Climate Research 30 (2005), pp. 79–82. DOI: 10.3354/cr030079.
Lonnie Magee. “R2Measures Based on Wald and Likelihood Ratio Joint Significance Tests”. In: The American Statistician 44.3 (1990), pp. 250–253. DOI: 10.1080/00031305.1990.10475731.
Arnaud de Myttenaere et al. “Mean absolute percentage error for regression models”. Neurocomputing 192 (2016), pp. 38–48. DOI: 10.1016/j.neucom.2015.12.114.
Shen Yi. Loss Functions for Binary Classification and Class Probability Estimation, 2005. URL: http://stat.wharton.upenn.edu/~buja/PAPERS/yi-shen-dissertation.pdf.
2018. URL: https://medium.com/@ODSC/unsupervised-learning-evaluating-clusters-bd47eed175ce.
Peter J. Rousseeuw. “Silhouettes: A graphical aid to the interpretation and validation of cluster analysis”. Journal of Computational and Applied Mathematics 20 (1987), pp. 53–65. DOI: 10.1016/0377- 0427(87)90125-7.
2019. URL: https://www.o-ran.org/.
Diego Kreutz et al., “Software-Defined Networking: A Comprehensive Survey”. In: Proceedings of the IEEE 103.1, pp. 14–76. DOI: 10.1109/jproc.2014.2371999.
Graham Teasdale and Bryan Jennett. “Assessment of Coma and Impaired Consciousness”. In: The Lancet 304.7872 (1974), pp. 81–84. DOI: 10.1016/s0140-6736(74)91639-0.
Florence C.M. Reit et al. “Differential effects of the glasgow coma scale score and its components: An analysis of 54,069 patients with traumatic brain injury”. In: Injury 48.9 (2017), pp. 1932–1943. DOI: 10.1016/j.injury.2017.05.038.
Mieczyslaw Finster and Margaret Wood. “The Apgar Score Has Survived the Test of Time”. In: Anesthesiology 102.4, pp. 855–857. DOI: 10.1097/00000542-200504000-00022.
Anders Bjartell. “The 2005 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma”. In: European Urology 49.4 (2006), pp. 758–759. DOI: 10.1016/j.eururo.2006.02.007.
Premnath K Narayanan and David K Harrison. “BIAS mitigation methods for autonomous network functions in telecommunication networks (a presentation)”. In: Wireless World Research Forum. Wireless World Research Forum, 2019. URL: https://www.wwrf.ch/wwrf43.html.
Premnath K Narayanan and David K Harrison”. “Measuring the Effectiveness of Autonomous Network Functions that Use Artificial Intelligence Algorithms (a presentation)”. In: Wireless World Research Forum. Wireless World Research Forum, 2020. URL: http://wwrf.ch/wwrf44.html.
