Publications

User Simulation book published

I’m thrilled to announce that our book with ChengXiang Zhai “User Simulation for Evaluating Information Access Systems” has been published (a preprint is available on arXiv). This comprehensive monograph delves into the pivotal role of user simulation in assessing the effectiveness of information access systems, such as search engines, recommender systems, and conversational assistants. Addressing the intricate challenges of evaluating these systems, our book explores user simulation techniques that account for the diverse behaviours and preferences of users. It offers a detailed examination of general frameworks, models, and algorithms designed to simulate user interactions, and establishes connections with related fields, including machine learning, dialogue systems, user modeling, and economics. We also discuss future research directions that extend beyond the evaluation of information access systems and are expected to have broader impact on how to evaluate interactive intelligent systems in general.

User Simulation book draft available

I’m excited to share the draft of our book, co-authored by ChengXiang Zhai, User Simulation for Evaluating Information Access Systems: http://arxiv.org/abs/2306.08550

This book focuses on providing a thorough understanding of user simulation techniques designed specifically for evaluation purposes. We systematically review both general frameworks and specific models and algorithms for simulating user interactions with search engines, recommender systems, and conversational assistants.

We invite feedback on this first version. If you have suggestions, comments, pointers, etc. reach out to me in email!

SIGIR’22 contributions

The following papers got accepted at SIGIR’22:

  • “Analyzing and Simulating User Utterance Reformulation in Conversational Recommender Systems” — full paper with Shuo Zhang and Mu Chun Wang [PDF]
  • “On Natural Language User Profiles for Transparent and Scrutable Recommendation” — perspectives paper with Filip Radlinski, Fernando Diaz, Lucas Dixon, and Ben Wedin [PDF]
  • “Would You Ask it that Way? Measuring and Improving Question Naturalness for Knowledge Graph Question Answering” — resource paper with Trond Linjordet [PDF]

Additionally, together with Alessandro Piscopo, Oana Inel, Sanne Vrijenhoek, and Martijn Millecamp, I’ll be co-organizing a workshop on Measuring Quality of Explanations in Recommender Systems [PDF].

update May 31: pre-prints added

Best paper award at DESIRES’21

I was honoured to receive a Best Paper Award for my paper “Conversational AI from an Information Retrieval Perspective: Remaining Challenges and a Case for User Simulation” at the 2nd International Conference on Design of Experimental Search & Information REtrieval Systems (DESIRES’21), which took place last week. The paper as well as the presentation slides are available online.

SIGIR’21 preprints and resources

Thanks to a fruitful collaboration with colleagues at Google, Bloomberg, Radboud University, Shandong University, and the University of Amsterdam, and, of course, students at the University of Stavanger, I have the following papers to appear at SIGIR this year. All are around conversational and/or recommender systems and come with publicly released resources.