Highlights from 2019

As a first post of 2020, it’s only fitting to start with a brief summary of highlights from 2019.

PhD position in Knowledge Graphs for Conversational AI

The University of Stavanger invites applications for a fully funded PhD position.

Knowledge graphs, organizing structured information about entities, and their attributes and relationships, are ubiquitous today. They have become powerful assets for a broad range of search, recommendation, and mining scenarios. Examples include enabling rich knowledge panels and direct answers in search result pages, supporting data exploration and visualization, and facilitating media monitoring and reputation management. This project focuses on the usage of knowledge graphs for conversational AI, in particular, for conversational search and recommendation tasks.

The recent success of deep learning techniques in different areas of natural language processing has enabled conversational AI systems to generate human-like responses. These systems, however, still have from little to no understanding of the actual meaning of the dialog. Knowledge graphs are needed to make human-machine interactions more grounded in knowledge. Knowledge graphs may also be utilized for personalized experiences.

This project involves various tasks around knowledge graphs for conversational information access, including the development of (i) models of interaction, (ii) algorithms for personalized search and recommendation, (iii) methods for multi-modal result presentation, and (iv) evaluation methodology of resources.


The candidate is expected to have a background in information retrieval, natural language processing, or machine learning.

See this page for the application instructions. Note that you’ll need to provide a cover letter expressing your interest in this specific project.

Application deadline: January 12, 2020.

This position has been re-announced. Application deadline: April 19, 2020.

ECIR’19 keynote

As the recipient of the 2018 Karen Spärck Jones Award, I was invited to give a keynote at the 41st European Conference on Information Retrieval (ECIR’19). Below are the slides of my presentation.

Highlights from 2018

This was another year when I was just too busy to blog. But, here are a few things from 2018 to be proud of (in no particular order).

Two journal papers on online evaluation

I am a co-author of two journal papers that appeared in the special issues of the Journal of Data and Information Quality on Reproducibility in IR.

The article entitled “OpenSearch: Lessons Learned from an Online Evaluation Campaign” by Jagerman et al. reports on our experience with TREC OpenSearch, an online evaluation campaign that enabled researchers to evaluate their experimental retrieval methods using real users of a live website. TREC OpenSearch focused on the task of ad hoc document retrieval within the academic search domain. We describe our experimental platform, which is based on the living labs methodology, and report on the experimental results obtained. We also share our experiences, challenges, and the lessons learned from running this track in 2016 and 2017.

The article entitled “Evaluation-as-a-Service for the Computational Sciences: Overview and Outlook” by Hopfgartner et al. discusses the Evaluation-as-a-Service paradigm, where data sets are not provided for download, but can be accessed via application programming interfaces (APIs), virtual machines (VMs), or other possibilities to ship executables. We summarize and compare current approaches, consolidate the experiences of these approaches, and outline next steps toward sustainable research infrastructures.