An excerpt below from the paper at ACM Recysys ‘22 by Lien Michiels & Robin Verachtert, awarded as the Best Demo:

Over the past decade, recommender systems have become a staple of online user experiences across industries, from media to tourism to e-ommerce. At the same time, the domain of recommender systems’ research has evolved from a focus on rating prediction tasks, e.g. the Netflix 1 Million Dollars Prize, to top-N recommendation. Today, the state-of-the-art in top-N recommendation is advancing at a very fast pace. More and more, researchers are encouraged to share their code to enable reproducibility and increase transparency [4]. However, this code is often of low quality and/or is shared without instructions on how to set up your environment to allow reproduction of the study [1, 4]. With the RecPack library we aim to aid researchers in this important effort.

RecPack is an experimentation toolkit for top-N recommendation using implicit feedback data written in Python, with a familiar interface and clear documentation. Its goal is to support researchers who advance the state-of-the-art in top-N recommendation to write reproducible and reusable experiments. RecPack comes with a range of different datasets, recommendation scenarios, state-of-the-art baselines and metrics. Wherever possible, RecPack sets sensible defaults. For example, hyperparameters of all recommendation algorithms included in RecPack are initialized to the best performing settings found in the original experiments. 

The design of RecPack is heavily inspired by the interface of scikit-learn, a popular Python package for classification, regression and clustering tasks. Data scientists who are familiar with scikit-learn will already have an intuitive understanding of how to work with RecPack. On top of this, RecPack was developed with a production mindset: All contributions are rigorously reviewed and tested. The RecPack maintainers strive to maintain a test coverage of more than ninety percent at all times.

Using RecPack, you can:

> Quickly implement new algorithms by using one of RecPack’s abstract algorithm base classes. These base classes represent popular recommendation paradigms such as matrix factorization and deep learning.

> Compare your algorithms against state-of-the-art baselines in several different recommendation scenarios using a variety of different performance metrics.

> Tune hyperparameters of both baselines and your own implementations with minimal data leakage.

‍

In recent years, many other Python packages for top-N recommendation have been released [e.g. 2, 3, 5]. However, these focus more on the purpose of ‘benchmarking’, i.e. quick and accurate comparisons of state-of-the-art recommendation algorithms published recently, and consequently provide access through the use of a configuration language or command-line interface. RecPack on the other hand wishes to support researchers with the development of their own algorithms through repeated refinement, experimentation and bug-fixing in e.g. a Notebook environment. 

A typical experimentation pipeline for top-N recommendation is shown in the Figure below. RecPack provides a dedicated module to support each step. For the detailed documentation on each module and the entire process, check out the documentation that can be found below.

Conclusion

Froomle is incredibly proud of the hard work and dedication put forward by our PhD researchers to improve the capabilities of not just our own Data Scientists, but for the entire Recsys community. We are incredibly excited to see how Recpack is used in the future and to see the impact it will inevitably have.

‍

👉 Take me to the full paper!

‍

References mentioned

[1] 2021. But is the code (re)usable? Nature Computational Science 1, 7 (01 Jul 2021), 449–449. https://doi.org/10.1038/s43588-021-00109-9

[2] Vito Walter Anelli, Alejandro Bellogin, Antonio Ferrara, Daniele Malitesta, Felice Antonio Merra, Claudio Pomo, Francesco M. Donini, and Tommaso Di Noia. 2021. Elliot: a Comprehensive and RigorousFramework for Reproducible Recommender Systems Evaluation. In SIGIR ’21: Proceedings of the44rdInternationalACMSIGIRConferenceonResearchandDevelopmentinInformationRetrieval. http://sisinflab.poliba.it/publications/2021/ ABFMMPDD21

[3] Zhu Sun, Di Yu, Hui Fang, Jie Yang, Xinghua Qu, Jie Zhang, and Cong Geng. 2020. Are We Evaluating Rigorously? Benchmarking Recommendation for Reproducible Evaluation and Fair Comparison. In Proceedings of the 14th ACM Conference on Recommender Systems.

[4] Ana Trisovic, Matthew K. Lau, Thomas Pasquier, and Mercè Crosas. 2022. A large-scale study on research code quality and execution. Scientific Data 9, 1 (21 Feb 2022), 60. https://doi.org/10.1038/s41597-022-01143-6 

[5] Wayne Xin Zhao, Junhua Chen, Pengfei Wang, Qi Gu, and Ji-Rong Wen. 2020. Revisiting Alternative Experimental Settings for Evaluating Top-N Item Recommendation Algorithms. In CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020. 2329–2332.