overview

EOS workshop summary

The First International Workshop on Entity-Oriented Search (EOS) was held on July 28, 2011 in Beijing, China, in conjunction with SIGIR 2011. The objective for the workshop was to bring together academic researchers and industry practitioners working on entity-oriented search to discuss tasks and challenges, and to uncover the next frontiers for academic research on the topic. The workshop program accommodated two invited talks, eleven refereed papers divided into three technical paper sessions, and a group discussion.

You can find the workshop proceedings here. A summary report, to appear in the December issue of SIGIR Forum, is already available online.

TREC Entity 2010 overview

The TREC Entity 2010 overview paper is now available online. We will soon start the discussion about the 2011 edition on the track’s mailing list.

TREC Enterprise 2008 overview

The overview paper of the TREC 2008 Enterprise track is -finally- available. While I was not an organizer of the track, I helped out with finishing the paper; the track organizers generously awarded my contribution with a first authorship. The document still needs to undergo the NIST approval process, but I am allowed to distribute it as “draft”.
[Dowload PDF|BibTex].

Despite having my name on the overview paper, I am still wearing a participant’s hat. So the first questions that comes to mind is: How did we do? (We is team ISLA, consisting of Maarten de Rijke and me.) To cut the story short — we won! Of course, TREC (according to some people) is not a competition. I am not going to take a side on that matter (at least not in this post), so let me translate the simple “we won” statement from ordinary to scientific language: our run showed the best performance among all submissions for the expert finding task of the TREC 2008 Enterprise track. Actually, we achieved both first and second place for all metrics and for all three different versions of the official qrels (they differ in how assessor agreement was handled). Our best run employed a combination of three models: a proximity-based candidate model, a document-based model, and a Web-based variation of the candidate model; our second best run is the same, but without the Web-based component. See the details in our paper [Download PDF|BibTex].
Needless to say, I am very content with these results. Seeing that my investments into research on expert finding has resulted in the state-of-the-art feels just great.

Future challenges in expertise retrieval

This was the title of the workshop I organized at SIGIR 2008 in July. The main objective of the workshop was to bring people from different research communities together, to discuss recent advances in expertise retrieval, and to define a research roadmap for the next years.
I think (and I hope I’m not alone with this) that the workshop was a success, with many interesting papers and lively discussions. If you’re interested in expert finding but missed it, now is your chance to find out what themes were discussed; check out the workshop summary that was recently published in the December 2008 issue of SIGIR Forum.