Year: 2018

Two journal papers on online evaluation

I am a co-author of two journal papers that appeared in the special issues of the Journal of Data and Information Quality on Reproducibility in IR.

The article entitled “OpenSearch: Lessons Learned from an Online Evaluation Campaign” by Jagerman et al. reports on our experience with TREC OpenSearch, an online evaluation campaign that enabled researchers to evaluate their experimental retrieval methods using real users of a live website. TREC OpenSearch focused on the task of ad hoc document retrieval within the academic search domain. We describe our experimental platform, which is based on the living labs methodology, and report on the experimental results obtained. We also share our experiences, challenges, and the lessons learned from running this track in 2016 and 2017.

The article entitled “Evaluation-as-a-Service for the Computational Sciences: Overview and Outlook” by Hopfgartner et al. discusses the Evaluation-as-a-Service paradigm, where data sets are not provided for download, but can be accessed via application programming interfaces (APIs), virtual machines (VMs), or other possibilities to ship executables. We summarize and compare current approaches, consolidate the experiences of these approaches, and outline next steps toward sustainable research infrastructures.

Entity-Oriented Search book

Entity-Oriented SearchI am pleased to announce that my Entity-Oriented Search book is now available online.

This open access book covers all facets of entity-oriented search—where “search” can be interpreted in the broadest sense of information access—from a unified point of view, and provides a coherent and comprehensive overview of the state of the art. It represents the first synthesis of research in this broad and rapidly developing area. Selected topics are discussed in-depth, the goal being to establish fundamental techniques and methods as a basis for future research and development. Additional topics are treated at a survey level only, containing numerous pointers to the relevant literature. A roadmap for future research, based on open issues and challenges identified along the way, rounds out the book.

PhD position in Conversational AI (re-opened)

The position that has been advertised before has not been filled and is re-opened. See this page for the application instructions (remember to specify topic #9 Conversational AI for information access and retrieval as your preference). Application deadline: September 30, 2018.

Joining Google (for a year)

Today, I’ve started by sabbatical at Google, London, UK, where I’ll be working as a Staff Visiting Faculty Researcher on conversational recommendations. I still maintain a 20% position at the University of Stavanger to continue supervising my PhD students.

Table generation and retrieval

Tables are powerful and versatile tools for organizing and presenting data. Tables may be viewed as complex information objects, which summarize existing information in a structured form. Therefore, for many information needs, returning tables as search results may be more helpful to users than serving a ranked list of items (documents or entities). We have a line of work, with Shuo Zhang, centered around utilizing (relational) tables as the unit of retrieval (published at WWW’18 and SIGIR’18). I presented our research at this interesting intersection of entity retrieval and data search in my keynote at the DATA:SEARCH’18 workshop at SIGIR’18 (slides are here).