The CLEF 2010 labs will feature two evaluation campaigns that are potentially of interest to people working in the area of entity/people/expert search.
The third WePS Evaluation Workshop (WePS3) focuses on two tasks related to web entity search:
- Task 1: Clustering and Attribute Extraction for Web People Search.
Given a set of web search results for a person name, the task is to cluster the pages according to the different people sharing the name and extract certain biographical attributes for each person. [details] - Task 2: Name ambiguity resolution for Online Reputation Management.
Given a set of Twitter entries containing an (ambiguous) company name, and given the home page of the company, the task is to discriminate entries that do not refer to the company. Entries will be given in two languages: English and Spanish. [details]
The Cross-lingual Expert Search (CriES) workshop addresses the problem of multi-lingual expert search in social media environments. The workshop also includes a pilot challenge, which is very much like the expert finding task at the TREC Enterprise track: given a document collection and a query topic, return a ranked list of experts, who are likely to be experts on the topic. However, the document collection is a multilingual social environment (Yahoo! Answers) and topics come in 4 different languages (English, German, French, Spanish).
Hi,
I have found also quite interesting the initial attempt to evaluate entity search on RDF data organized at the Semantic Search workshop (see http://km.aifb.uni-karlsruhe.de/ws/semsearch10/). The dataset is a billion RDF triples crawled on the web and a set of queries from search engine logs. Relevance judgments will be created with crowdsourcing.
Gianluca
Thanks, Gianluca!
We are also planning to do something similar at the TREC Entity track this year (see http://ilps.science.uva.nl/trec-entity/2010/02/conference-summary-and-plans-for-2010/).
Of course, we first have to see how participants feel about the proposed semantic entity search task.