A Comprehensive Review on Non-Neural Networks Collaborative Filtering Recommendation Systems
Keywords:
Recommendation systems, collaborative filtering, user-based collaborative filtering, item-based collaborative filtering, matrix factorization, non-negative matrix factorization, explainable matrix factorization, evaluation metricsAbstract
Over the past two decades, recommendation systems have attracted a lot of interest due to the massive rise of online applications. A particular attention has been paid to collaborative filtering, which is the most widely used in applications that involve information recommendations. Collaborative Filtering (CF) uses the known preference of a group of users to make predictions and recommendations about the unknown preferences of other users (recommendations are made based on the past behavior of users). First introduced in the 1990s, a wide variety of increasingly successful models have been proposed. Due to the success of machine learning techniques in many areas, there has been a growing emphasis on the application of such algorithms in recommendation systems. In this article, we present an overview of the CF approaches for recommendation systems, their two main categories, and their evaluation metrics. We focus on the application of classical Machine Learning algorithms to CF recommendation systems by presenting their evolution from their first use-cases to advanced Machine Learning models. We attempt to provide a comprehensive and comparative overview of CF systems (with python implementations) that can serve as a guideline for research and practice in this area.
Downloads
References
Abdollahi and Nasraoui (2016) Abdollahi, B., and O. Nasraoui, 2016: Explainable matrix factorization for collaborative filtering. Proceedings of the 25th International Conference Companion on World Wide Web, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 5–6, WWW ’16 Companion, URL https://doi.org/10.1145/2872518.2889405.
Abdollahi and Nasraoui (2017) Abdollahi, B., and O. Nasraoui, 2017: Using explainability for constrained matrix factorization. Proceedings of the Eleventh ACM Conference on Recommender Systems, ACM, New York, NY, USA, URL https://doi.org/10.1145/3109859.3109913.
Ali and Majeed (2021) Ali, S. I. M., and S. S. Majeed, 2021: A review of collaborative filtering recommendation system. Almuthanna Journal of Pure Science (MJPS), 8, 120–131, doi:10.52113/2/08.01.2021/120-131.
Ansari et al. (2000) Ansari, A., S. Essegaier, and R. Kohli, 2000: Internet recommendation systems. J. Mark. Res., 37 (3), 363–375, URL http://dx.doi.org/10.1509/jmkr.37.3.363.18779.
Baltrunas et al. (2010) Baltrunas, L., T. Makcinskas, and F. Ricci, 2010: Group recommendations with rank aggregation and collaborative filtering. Proceedings of the fourth ACM conference on Recommender systems, Association for Computing Machinery, New York, NY, USA, 119–126, RecSys ’10, URL https://doi.org/10.1145/1864708.1864733.
Billsus and Pazzani (1998) Billsus, D., and M. J. Pazzani, 1998: Learning collaborative information filters. Proceedings of the Fifteenth International Conference on Machine Learning (ICML 1998), Madison, Wisconsin, USA, July 24-27, 1998, J. W. Shavlik, Ed., Morgan Kaufmann, 46–54.
Bobadilla et al. (2011) Bobadilla, J., A. Hernando, F. Ortega, and J. Bernal, 2011: A framework for collaborative filtering recommender systems. Expert Syst. Appl., 38 (12), 14 609–14 623, URL https://doi.org/10.1016/j.eswa.2011.05.021.
Bobadilla et al. (2013) Bobadilla, J., F. Ortega, A. Hernando, and A. Gutiérrez, 2013: Recommender systems survey. Knowledge-Based Systems, 46, 109–132, URL https://doi.org/10.1016/j.knosys.2013.03.012.
Bokde et al. (2015) Bokde, D., S. Girase, and D. Mukhopadhyay, 2015: Matrix factorization model in collaborative filtering algorithms: A survey. Procedia Comput. Sci., 49, 136–146, URL https://doi.org/10.1016/j.procs.2015.04.237.
Breese et al. (1998) Breese, J. S., D. Heckerman, and C. Kadie, 1998: Empirical analysis of predictive algorithms for collaborative filtering. Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 43–52, UAI’98, URL https://arxiv.org/abs/1301.7363.
Cacheda et al. (2011) Cacheda, F., V. Carneiro, D. Fernández, and V. Formoso, 2011: Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems. ACM Trans. Web, 5 (1), 1–33, URL https://doi.org/10.1145/1921591.1921593.
Cai et al. (2008) Cai, D., X. He, X. Wu, and J. Han, 2008: Non-negative matrix factorization on manifold. Proceedings – 8th IEEE International Conference on Data Mining, ICDM 2008, 63–72, Proceedings – IEEE International Conference on Data Mining, ICDM, doi:10.1109/icdm.2008.57, URL https://doi.org/10.1109/ICDM.2008.57, copyright: Copyright 2012 Elsevier B.V., All rights reserved.; 8th IEEE International Conference on Data Mining, ICDM 2008 ; Conference date: 15-12-2008 Through 19-12-2008.
Caragea et al. (2009) Caragea, C., and Coauthors, 2009: Mean average precision. Encyclopedia of Database Systems.
Chee et al. (2001) Chee, S. H. S., J. Han, and K. Wang, 2001: RecTree: An efficient collaborative filtering method. Data Warehousing and Knowledge Discovery, Lecture notes in computer science, Springer Berlin Heidelberg, Berlin, Heidelberg, 141–151, URL https://doi.org/10.1007/3-540-44801-2“˙15.
Coba et al. (2019) Coba, L., P. Symeonidis, and M. Zanker, 2019: Personalised novel and explainable matrix factorisation. Data Knowl. Eng., 122, 142–158, URL https://doi.org/10.1016/j.datak.2019.06.003.
Deerwester et al. (1990) Deerwester, S., S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, 1990: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci., 41 (6), 391–407.
Deshpande and Karypis (2004) Deshpande, M., and G. Karypis, 2004: Item-based top- N recommendation algorithms. ACM Trans. Inf. Syst. Secur., 22 (1), 143–177, URL https://doi.org/10.1145/963770.963776.
Desrosiers and Karypis (2011) Desrosiers, C., and G. Karypis, 2011: A comprehensive survey of neighborhood-based recommendation methods. Recommender Systems Handbook, F. Ricci, L. Rokach, B. Shapira, and P. B. Kantor, Eds., Springer US, Boston, MA, 107–144, doi:10.1007/978-0-387-85820-3˙4, URL http://dx.doi.org/10.1007/978-0-387-85820-3˙4.
Dillon (1983) Dillon, M., 1983: Introduction to modern information retrieval. Inf. Process. Manag., 19 (6), 402–403.
Ekstrand et al. (2011) Ekstrand, M. D., J. T. Riedl, and J. A. Konstan, 2011: Collaborative filtering recommender systems. Foundations and Trends® in Human–Computer Interaction, 4 (2), 81–173, URL https://doi.org/10.1561/1100000009.
Feigl and Bogdan (2017) Feigl, J., and M. Bogdan, 2017: Collaborative filtering with neural networks. ESANN, URL http://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2017-23.pdf.
Gillis (2014) Gillis, N., 2014: The why and how of nonnegative matrix factorization. Regularization, Optimization, Kernels, and Support Vector Machines, URL http://arxiv.org/abs/1401.5226.
Goldberg et al. (1992) Goldberg, D., D. Nichols, B. M. Oki, and D. Terry, 1992: Using collaborative filtering to weave an information tapestry. Commun. ACM, 35 (12), 61–70, URL https://doi.org/10.1145/138859.138867.
Goldberg et al. (2001) Goldberg, K., T. Roeder, D. Gupta, and C. Perkins, 2001: Eigentaste: A constant time collaborative filtering algorithm. Inf. Retr. Boston., 4, 133–151, URL https://doi.org/10.1023/A:1011419012209.
He et al. (2018) He, X., X. Du, X. Wang, F. Tian, J. Tang, and T. Chua, 2018: Outer product-based neural collaborative filtering. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, J. Lang, Ed., ijcai.org, 2227–2233, doi:10.24963/ijcai.2018/308, URL https://doi.org/10.24963/ijcai.2018/308.
He et al. (2017a) He, X., L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua, 2017a: Neural collaborative filtering. Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, R. Barrett, R. Cummings, E. Agichtein, and E. Gabrilovich, Eds., ACM, 173–182, doi:10.1145/3038912.3052569, URL https://doi.org/10.1145/3038912.3052569.
He et al. (2017b) He, X., L. Liao, H. Zhang, L. Nie, X. Hu, and T.-S. Chua, 2017b: Neural collaborative filtering. Proceedings of the 26th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 173–182, WWW ’17, URL https://doi.org/10.1145/3038912.3052569.
Herlocker et al. (1999) Herlocker, J. L., J. A. Konstan, A. Borchers, and J. Riedl, 1999: An algorithmic framework for performing collaborative filtering. Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, Association for Computing Machinery, New York, NY, USA, 230–237, SIGIR ’99, URL https://doi.org/10.1145/312624.312682.
Herlocker et al. (2004) Herlocker, J. L., J. A. Konstan, L. G. Terveen, and J. T. Riedl, 2004: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. Secur., 22 (1), 5–53, URL https://doi.org/10.1145/963770.963772.
Hernando et al. (2016) Hernando, A., J. Bobadilla, and F. Ortega, 2016: A non negative matrix factorization for collaborative filtering recommender systems based on a bayesian probabilistic model. Knowledge-Based Systems, 97, 188–202, URL https://doi.org/10.1016/j.knosys.2015.12.018.
Hofmann (1999) Hofmann, T., 1999: Probabilistic latent semantic indexing. Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval – SIGIR ’99, ACM Press, New York, New York, USA, URL https://doi.org/10.1145/312624.312649.
Hu and Li (2018) Hu, J., and P. Li, 2018: Collaborative filtering via additive ordinal regression. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Association for Computing Machinery, New York, NY, USA, 243–251, WSDM ’18, URL https://doi.org/10.1145/3159652.3159723.
Karypis (2001) Karypis, G., 2001: Evaluation of Item-Based Top-N recommendation algorithms. Proceedings of the tenth international conference on Information and knowledge management, Association for Computing Machinery, New York, NY, USA, 247–254, CIKM ’01, URL https://doi.org/10.1145/502585.502627.
Khalifeh and Al-Mousa (2021) Khalifeh, S., and A. A. Al-Mousa, 2021: A book recommender system using collaborative filtering method. DATA’21: International Conference on Data Science, E-learning and Information Systems 2021, Petra, Jordan, 5-7 April, 2021, J. A. L. Torralbo, S. A. Aljawarneh, V. Radhakrishna, and A. N., Eds., ACM, 131–135, doi:10.1145/3460620.3460744, URL https://doi.org/10.1145/3460620.3460744.
Koren et al. (2009) Koren, Y., R. Bell, and C. Volinsky, 2009: Matrix factorization techniques for recommender systems. Computer, 42 (8), 30–37, URL https://doi.org/10.1109/MC.2009.263.
Kunegis and Albayrak (2007) Kunegis, J., and S. Albayrak, 2007: Adapting ratings in Memory-Based collaborative filtering using linear regression. 2007 IEEE International Conference on Information Reuse and Integration, 49–54, URL https://doi.org/10.1109/IRI.2007.4296596.
Lakshminarayanan et al. (2011) Lakshminarayanan, B., G. Bouchard, and C. Archambeau, 2011: Robust bayesian matrix factorisation. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 425–433, URL http://proceedings.mlr.press/v15/lakshminarayanan11a/lakshminarayanan11a.pdf.
Lee and Seung (2001) Lee, D., and H. Seung, 2001: Algorithms for non-negative matrix factorization. Advances in Neural Information Processing Systems 13 – Proceedings of the 2000 Conference, NIPS 2000, 13.
Lee and Seung (1999) Lee, D. D., and H. S. Seung, 1999: Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788–791, doi:10.1038/44565, URL https://doi.org/10.1038/44565.
Linden et al. (2003) Linden, G., B. Smith, and J. York, 2003: Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Comput., 7 (1), 76–80, URL https://doi.org/10.1109/MIC.2003.1167344.
Luo et al. (2014) Luo, X., M. Zhou, Y. Xia, and Q. Zhu, 2014: An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems. IEEE Trans. Ind. Informatics, 10 (2), 1273–1284, doi:10.1109/TII.2014.2308433, URL https://doi.org/10.1109/TII.2014.2308433.
Mild and Natter (2002) Mild, A., and M. Natter, 2002: Collaborative filtering or regression models for internet recommendation systems? Journal of Targeting, Measurement and Analysis for Marketing, 10 (4), 304–313, URL http://dx.doi.org/10.1057/palgrave.jt.5740055.
Mnih and Salakhutdinov (2008) Mnih, A., and R. R. Salakhutdinov, 2008: Probabilistic matrix factorization. Advances in Neural Information Processing Systems, J. Platt, D. Koller, Y. Singer, and S. Roweis, Eds., Curran Associates, Inc., Vol. 20, 1257–1264.
Mu (2018) Mu, R., 2018: A survey of recommender systems based on deep learning. IEEE Access, 6, 69 009–69 022, URL https://doi.org/10.1109/ACCESS.2018.2880197.
Parhi et al. (2017) Parhi, P., A. Pal, and M. Aggarwal, 2017: A survey of methods of collaborative filtering techniques. 2017 International Conference on Inventive Systems and Control (ICISC), 1–7, URL http://dx.doi.org/10.1109/ICISC.2017.8068603.
Paterek (2007) Paterek, A., 2007: Improving regularized singular value decomposition for collaborative filtering. Proceedings of KDD cup and workshop, Vol. 2007, 5–8, URL https://www.cs.uic.edu/~liub/KDD-cup-2007/proceedings/Regular-Paterek.pdf.
Pauca et al. (2006) Pauca, V. P., J. Piper, and R. J. Plemmons, 2006: Nonnegative matrix factorization for spectral data analysis. Linear Algebra and its Applications, 416 (1), 29–47, URL https://doi.org/10.1016/j.laa.2005.06.025, special Issue devoted to the Haifa 2005 conference on matrix theory.
Pazzani and Billsus (2007) Pazzani, M. J., and D. Billsus, 2007: Content-Based recommendation systems. The Adaptive Web: Methods and Strategies of Web Personalization, P. Brusilovsky, A. Kobsa, and W. Nejdl, Eds., Springer Berlin Heidelberg, Berlin, Heidelberg, 325–341.
Rawat and Wang (2017) Rawat, W., and Z. Wang, 2017: Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation, 29 (9), 2352–2449, URL https://doi.org/10.1162/NECO˙a˙00990.
Resnick et al. (1994) Resnick, P., N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl, 1994: GroupLens: an open architecture for collaborative filtering of netnews. Proceedings of the 1994 ACM conference on Computer supported cooperative work, Association for Computing Machinery, New York, NY, USA, 175–186, CSCW ’94, URL https://doi.org/10.1145/192844.192905.
Sarwar et al. (2000) Sarwar, B., G. Karypis, J. Konstan, and J. Riedl, 2000: Application of dimensionality reduction in recommender system: A case study. Computer Science and Engineering (CS E) Technical Reports, URL https://hdl.handle.net/11299/215429.
Sarwar et al. (2001) Sarwar, B. M., G. Karypis, J. A. Konstan, and J. Riedl, 2001: Item-based collaborative filtering recommendation algorithms. Proceedings of the Tenth International World Wide Web Conference, WWW 10, Hong Kong, China, May 1-5, 2001, V. Y. Shen, N. Saito, M. R. Lyu, and M. E. Zurko, Eds., ACM, 285–295, doi:10.1145/371920.372071, URL https://doi.org/10.1145/371920.372071.
Saxena et al. (2017) Saxena, A., and Coauthors, 2017: A review of clustering techniques and developments. Neurocomputing, 267, 664–681, URL https://doi.org/10.1016/j.neucom.2017.06.053.
Sedhain et al. (2015) Sedhain, S., A. K. Menon, S. Sanner, and L. Xie, 2015: AutoRec: Autoencoders meet collaborative filtering. Proceedings of the 24th International Conference on World Wide Web, Association for Computing Machinery, New York, NY, USA, 111–112, WWW ’15 Companion, URL https://doi.org/10.1145/2740908.2742726.
Seyednezhad et al. (2018) Seyednezhad, S. M. M., K. N. Cozart, J. A. Bowllan, and A. O. Smith, 2018: A review on recommendation systems: Context-aware to social-based. CoRR, abs/1811.11866, URL http://arxiv.org/abs/1811.11866, 1811.11866.
Shenbin et al. (2020) Shenbin, I., A. Alekseev, E. Tutubalina, V. Malykh, and S. I. Nikolenko, 2020: RecVAE: A new variational autoencoder for Top-N recommendations with implicit feedback. Proceedings of the 13th International Conference on Web Search and Data Mining, Association for Computing Machinery, New York, NY, USA, 528–536, WSDM ’20, URL https://doi.org/10.1145/3336191.3371831.
Srifi et al. (2020) Srifi, M., A. Oussous, A. Ait Lahcen, and S. Mouline, 2020: Recommender systems based on collaborative filtering using review texts—a survey. Information, 11 (6), doi:10.3390/info11060317, URL https://www.mdpi.com/2078-2489/11/6/317.
Steck (2019) Steck, H., 2019: Collaborative filtering via High-Dimensional regression. ArXiv, abs/1904.13033, URL http://arxiv.org/abs/1904.13033, 1904.13033.
Su and Khoshgoftaar (2009) Su, X., and T. M. Khoshgoftaar, 2009: A survey of collaborative filtering techniques. Advances in Artificial Intelligence, 2009, URL https://doi.org/10.1155/2009/421425.
Su et al. (2008) Su, X., T. M. Khoshgoftaar, X. Zhu, and R. Greiner, 2008: Imputation-boosted collaborative filtering using machine learning classifiers. Proceedings of the 2008 ACM symposium on Applied computing, Association for Computing Machinery, New York, NY, USA, 949–950, SAC ’08, URL https://doi.org/10.1145/1363686.1363903.
Ungar and Foster (1998) Ungar, L. H., and D. P. Foster, 1998: Clustering methods for collaborative filtering. AAAI workshop on recommendation systems, Vol. 1, 114–129, URL https://www.aaai.org/Papers/Workshops/1998/WS-98-08/WS98-08-029.pdf.
Vucetic and Obradovic (2005) Vucetic, S., and Z. Obradovic, 2005: Collaborative filtering using a Regression-Based approach. Knowl. Inf. Syst., 7 (1), 1–22, URL https://doi.org/10.1007/s10115-003-0123-8.
Wang et al. (2018) Wang, S., H. Tian, X. Zhu, and Z. Wu, 2018: Explainable matrix factorization with constraints on neighborhood in the latent space. Data Mining and Big Data, Lecture notes in computer science, Springer International Publishing, Cham, 102–113, URL http://dx.doi.org/10.1007/978-3-319-93803-5˙10.
Wang and Zhang (2013) Wang, Y., and Y. Zhang, 2013: Nonnegative matrix factorization: A comprehensive review. IEEE Trans. Knowl. Data Eng., 25 (6), 1336–1353, URL https://doi.org/10.1109/TKDE.2012.51.
Willmott and Matsuura (2005) Willmott, C., and K. Matsuura, 2005: Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance. Climate Research, 30, 79, doi:10.3354/cr030079, URL https://doi.org/10.3354/cr030079.
Xu and Tian (2015) Xu, D., and Y. Tian, 2015: A comprehensive survey of clustering algorithms. Annals of Data Science, 2 (2), 165–193, URL https://doi.org/10.1007/s40745-015-0040-1.
Yu et al. (2019) Yu, Y., X. Si, C. Hu, and J. Zhang, 2019: A review of recurrent neural networks: LSTM cells and network architectures. Neural Compution, 31 (7), 1235–1270, URL https://doi.org/10.1162/neco˙a˙01199.
Zhang (2013) Zhang, L., 2013: The definition of novelty in recommendation system. J. Eng. Sci. Technol. Rev., 6 (3), 141–145, URL http://dx.doi.org/10.25103/jestr.063.25.
Zhang et al. (2018) Zhang, L., T. Luo, F. Zhang, and Y. Wu, 2018: A recommendation model based on deep neural network. IEEE Access, 6, 9454–9463, URL https://doi.org/10.1109/ACCESS.2018.2789866.
Zhang et al. (2019) Zhang, S., L. Yao, A. Sun, and Y. Tay, 2019: Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv., 52 (1), 1–38, URL https://doi.org/10.1145/3285029.
Zheng et al. (2018) Zheng, S., C. Ding, and F. Nie, 2018: Regularized singular value decomposition and application to recommender system. ArXiv, URL http://arxiv.org/abs/1804.05090, 1804.05090.