I’m thrilled to announce that our book with ChengXiang Zhai “User Simulation for Evaluating Information Access Systems” has been published (a preprint is available on arXiv). This comprehensive monograph delves into the pivotal role of user simulation in assessing the effectiveness of information access systems, such as search engines, recommender systems, and conversational assistants. Addressing the intricate challenges of evaluating these systems, our book explores user simulation techniques that account for the diverse behaviours and preferences of users. It offers a detailed examination of general frameworks, models, and algorithms designed to simulate user interactions, and establishes connections with related fields, including machine learning, dialogue systems, user modeling, and economics. We also discuss future research directions that extend beyond the evaluation of information access systems and are expected to have broader impact on how to evaluate interactive intelligent systems in general.
Year: 2024
Highlights from 2023
Another year has passed, and it’s now time for the usual annual highlights post. Here are some of the notable events and achievements from 2023:
- Completed the first draft of our book “User Simulation for Evaluating Information Access Systems” co-authored with ChengXiang Zhai, and published it on arXiv.
- Delivered a tutorial with ChengXiang Zhai on simulation-based evaluation at CIKM’23.
- Co-organized the second workshop on Measuring the Quality of Explanations in Recommender Systems, this time held at RecSys’23.
- Published several papers on conversational information access, including user simulation (WSDM’23), datasets (SIGIR’23a, SIGIR’23b, and CIKM’23), and reproducibility (ECIR’23).
- Contributed to research on recommender systems, focusing on the impact of explanations (CHI’23) and LLM-based recommendations (RecSys’23).
- Received the Best Resource Paper Award with Weronika Łajewska at CIKM’23.
- Welcomed a new faculty member, Petra Galuscakova, to the IAI group, marking another milestone in our growth.
Looking forward to what 2024 has in store!