Two evaluation campaigns related to entity/expert search

The CLEF 2010 labs will feature two evaluation campaigns that are potentially of interest to people working in the area of entity/people/expert search.

The third WePS Evaluation Workshop (WePS3) focuses on two tasks related to web entity search:

The Cross-lingual Expert Search (CriES) workshop addresses the problem of multi-lingual expert search in social media environments. The workshop also includes a pilot challenge, which is very much like the expert finding task at the TREC Enterprise track: given a document collection and a query topic, return a ranked list of experts, who are likely to be experts on the topic. However, the document collection is a multilingual social environment (Yahoo! Answers) and topics come in 4 different languages (English, German, French, Spanish).

February 12, 2010 | Filed Under CfP, entity search, expert search, resources

2 Comments 

Comments

2 Responses to “Two evaluation campaigns related to entity/expert search”

  1. Gianluca Demartini on February 16th, 2010 9:42 am

    Hi,

    I have found also quite interesting the initial attempt to evaluate entity search on RDF data organized at the Semantic Search workshop (see http://km.aifb.uni-karlsruhe.de/ws/semsearch10/). The dataset is a billion RDF triples crawled on the web and a set of queries from search engine logs. Relevance judgments will be created with crowdsourcing.

    Gianluca

  2. Krisztian Balog on February 22nd, 2010 5:53 pm

    Thanks, Gianluca!

    We are also planning to do something similar at the TREC Entity track this year (see http://ilps.science.uva.nl/trec-entity/2010/02/conference-summary-and-plans-for-2010/).

    Of course, we first have to see how participants feel about the proposed semantic entity search task.

Leave a Reply