Conferences

EMNLP’26 papers

I’m excited to share two papers that will be presented at EMNLP 2026 in Budapest later this month, on user simulation and natural language user profiles.

  • “Act Like a 5th Grader” is Not Enough: Bounding Knowledge in LLM-Based User Simulators (Findings paper, with A. M. Bakken) — LLM-based simulators tend to be “superhuman”: even when prompted to act like a 5th grader, they ace reading comprehension tests. Using over 71k responses from 2,359 Norwegian primary-school students, we propose the Cognitively Bounded User Simulator (CBUS), which models restricted working memory through an episodic bottleneck and substantially reduces the gap to real student behavior.
  • A Survey on Natural Language User Profiles for Recommendation: Methods, Datasets, and Metrics (main conference paper, with M. Arustashvili) — This survey reviews the literature on the two coupled tasks of profile generation and profile-based recommendation, proposes a taxonomy for organizing evaluation metrics, and outlines open challenges, including the tension between transparency and optimization.

I’ll be attending the conference in person. If you’re there, please come say hello!

SIGIR’25 contributions

I’m happy to share that I’ll be attending SIGIR ’25, which is shaping up to be a busy and exciting event.

Accepted papers:

  • “Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluation” — perspectives paper with Don Metzler and Zhen Qin [PDF]
  • “GINGER: Grounded Information Nugget-Based Generation of Responses” — short paper with W. Łajewska [PDF]
  • “MultiConAD: A Unified Multilingual Conversational Dataset for Early Alzheimer’s Detection” — resource paper with Arezo Shakeri and Mina Farmanbar [PDF]

In addition to the papers, I’ll also be giving a tutorial, together with Nolwenn Bernard, Saber Zerhoudi, and ChengXiang Zhai, on “Theory and Toolkits for User Simulation in the Era of Generative AI: User Modeling, Synthetic Data Generation, and System Evaluation” [website]. The tutorial covers key simulation methodologies, with a particular focus on recent advancements leveraging LLMs. Crucially, we will also provide practical guidance, highlighting relevant toolkits, libraries, and datasets available to researchers and practitioners.

Finally, I’m co-organizing the Second SIGIR Workshop on Simulations for Information Access (Sim4IA 2025) together with Philipp Schaer, Christin Katharina Kreutz, Timo Breuer, and Andreas Konstantin Kruff [website]. The workshop features a keynote, invited tech talks, a panel discussion, and (micro) shared tasks for simulating interactions with a traditional search engine or a conversational assistant.

If you’re attending the conference, please come say hello, drop into the tutorial or workshop, or reach out ahead of time—I’d love to connect.

User Simulation tutorial at AAAI’24 and WWW’24

Together with ChengXiang Zhai, we will be giving our user simulation tutorial at AAAI’24 and WWW’24, customized to the respective audiences. The AAAI’24 edition adopts a broader perspective of user simulation for evaluating an interactive AI system and focuses more on simulation algorithms and techniques that are well connected with various sub-fields of AI, such as machine learning and agent-based systems. In contrast, the WWW’24 edition emphasizes more on applications of user simulation for evaluating Web information access systems, including click modeling and the application of simulation techniques in e-commerce.

Event reports

The June 2022 issue of SIGIR Forum reports on two events that I co-organized:

TREC 2010 summary

The 19th Text REtrieval Conference (TREC) took place at the “usual” time and place: Gaithersburg, MD, in the second half of November. Seven tracks ran in 2010: Blog, Chemical IR, Entity, Legal, Relevance Feedback, Session, and Web.
The Entity track was very popular both in terms of the number of participants and the number of posters presented. The proposed approaches displayed a great degree of diversity and made the presentations very interesting. I don’t want to repeat myself, so I refer to the posts on the Entity website for the conference summary and plans for 2011.
As to TREC 2011, the Chemical IR, Entity, Session, Legal, and Web tracks will continue. The Blog track will migrate to a new Microblog track and will investigate social search, especially search over Twitter data. Two more new tracks will be added: Crowdsourcing (as a means of evaluation) and Medical records (content-based access to the free text fields of medical records, e.g., find patients with disease X treated with Y). Finally, CMU is planning another Web crawl, successor to ClueWeb09; one idea is to have a smaller set of pages, but crawled regularly over a period of time.