I’m happy to share two papers on conversational recommender systems that were presented at SIGIR and ICTIR this summer. As always, the code and data are publicly available!
- A Standardized Re-evaluation of Conversational Recommender Systems on the ReDial Dataset (SIGIR reproducibility paper, with I. Kostric) — This work re-evaluates seven prominent CRS methods on ReDial under standardized conditions. We find that nearly half of the reported accuracy comes from “repetition shortcuts,” that performance gains often stem from stronger language model backbones rather than architectural innovations, and that user-centric utility metrics frequently contradict recall-based evaluation. [resources]
- RecQuest: Towards Estimating User Domain Knowledge in Conversational Recommender Systems (ICTIR full paper, with I. Kostric and U. Gadiraju) — Conversational recommender systems often implicitly treat every user as an expert. This paper introduces the task of estimating user domain knowledge from dialogues, along with RecQuest, a game-with-a-purpose data collection protocol, and a dataset of 515 dialogues across five product domains. [resources]