I’m excited to share that, together with the esteemed ChengXiang Zhai, we’ll be presenting a tutorial on “User Simulation for Evaluating Information Access Systems” at the upcoming 32nd ACM International Conference on Information and Knowledge Management (CIKM ’23) in Birmingham, UK, in October. This half-day tutorial is based on our upcoming FnTIR book. We look forward to sharing our insights and also hope for feedback that could shape the final version of the book. Visit our companion website usersim.ai for more info.
User Simulation book draft available
I’m excited to share the draft of our book, co-authored by ChengXiang Zhai, User Simulation for Evaluating Information Access Systems: http://arxiv.org/abs/2306.08550
This book focuses on providing a thorough understanding of user simulation techniques designed specifically for evaluation purposes. We systematically review both general frameworks and specific models and algorithms for simulating user interactions with search engines, recommender systems, and conversational assistants.
We invite feedback on this first version. If you have suggestions, comments, pointers, etc. reach out to me in email!
Highlights from 2022
A selection of highlights from 2022:
- We organized ECIR’22 in Stavanger, Norway, marking it as the first major IR conference with a hybrid setup that allowed in-person travel after the pandemic. We wrote about our experiences in SIGIR Forum.
- Co-organized a workshop on Measuring the Quality of Explanations in Recommender Systems (SIGIR’22).
- Pioneered work on natural language user profiles for transparent and scrutable recommendation, published as a perspectives paper at SIGIR’22.
- Trond Linjordet successfully completed and defended his PhD thesis “A Critical Look at the Evaluation of Knowledge Graph Question Answering“.
- Two new PhD students started at IAI under my supervision: Nolwenn Bernard and Weronika Łajewska.
- Above all, I became a father this year, a personal milestone that far outshines any professional success. It has made me even more deliberate about where I spend my time, prompting me to say no more often, including accepting committee invitations and traveling to conferences.
Event reports
The June 2022 issue of SIGIR Forum reports on two events that I co-organized:
- The Personal Knowledge Graphs (PKG) workshop at the AKBC’21 conference [PDF]
- The 44th European Conference on Information Retrieval (ECIR’22) that was held in Stavanger, Norway in a hybrid format [PDF]
SIGIR’22 contributions
The following papers got accepted at SIGIR’22:
- “Analyzing and Simulating User Utterance Reformulation in Conversational Recommender Systems” — full paper with Shuo Zhang and Mu Chun Wang [PDF]
- “On Natural Language User Profiles for Transparent and Scrutable Recommendation” — perspectives paper with Filip Radlinski, Fernando Diaz, Lucas Dixon, and Ben Wedin [PDF]
- “Would You Ask it that Way? Measuring and Improving Question Naturalness for Knowledge Graph Question Answering” — resource paper with Trond Linjordet [PDF]
Additionally, together with Alessandro Piscopo, Oana Inel, Sanne Vrijenhoek, and Martijn Millecamp, I’ll be co-organizing a workshop on Measuring Quality of Explanations in Recommender Systems [PDF].
update May 31: pre-prints added