This year has started quite intensely, with a research grant proposal deadline already on week 1. But, I’d like to take a moment and look back on 2009, before rushing on to the next deadline (which is SIGIR 2010, in less than a week away).
First, I’d like to make an honorable mention of four of my colleagues/co-authors who have defended their PhD and became doctors in 2009. They are (in temporal order):
They all did a great job, congrats!
A significant milestone of 2009 was the launch of the TREC Entity track. The overall aim of this new track is to perform entity-related search on Web data. The track defines entities as “typed search results” or “things”, represented by their homepages on the web. In other words, our working definition of an entity is “something with a homepage”, and searching for entities thus corresponds to ranking these homepages. (As a sidenote: I am well aware that this definition of an entitiy is far from perfect, yet, the URL of a homepage is the best entity identifier we could come up with so far.)
The first year of the track investigated the problem of related entity finding:
Given an input entity, by its name and homepage, the type of the target entity, as well as the nature of their relation, described in free text, find related entities that are of target type, standing in the required relation to the input entity.
This task can be seen as a mixture of Question Answering (specifically, the QA list task) and homepage finding. In the first year, we limited the track’s scope to searches for instances of the organizations, people, and product entity types.
Thirteen groups participated and submitted a total of 41 runs; this demonstrates clear interest, and, I think, is quite decent for the first edition of the track (beating well established tracks, like the Blog track, in terms of the number of participating teams).
The track continues in 2010, where the main task will (again) be related entity finding, with moderate changes and more topics. We are also planning to feature another subtask (more details will follow on the track’s mailing list).
Another important development was the release of the EARS toolkit. EARS stands for Entity and Association Retrieval System, and is an open source implementation of entity-topic association finding models, used so far mostly in the context of expertise retrieval, but also for other tasks, for example blog distillation. While currently the functionality of EARS is limited to two baseline models (“Model 1” and “Model 2”), a number of additions are coming in future releases, throughout 2010. Most notably, proximity-based variations of existing models, and methods for finding entity-entity relations (i.e., addressing the related entity finding task defined above).
A yearly evaluation would not be complete without mentioning citation counts. Some say, citation is to publication as price is to stock. So, I do it this time in a NASDAQ-stlye. Total citation count has doubled (from 214 to 433) and my H-index has also increased, from 8 to 11. The top performing papers are shown below.
+/- denotes the position change in the relative ordering of my papers, according to the number of citations.
According to these numbers, our SIGIR 2006 paper is (still) a massive leader, and keeps on following the rich-getting-richer trend. A more interesting observation is that our expert profiling work seems to have gained impact and attention in the past year, as citation counts for the two profiling papers have almost tripled in 2009. This is good news, especially in the light of some ongoing work we are performing in this area.
That’s it for now (longest post ever). I wish a successful 2010 to everybody (and good luck to those with a SIGIR deadline)!
This picture gives you a bit of an idea how my week in the run-up for SIGIR deadline looked like (I also have a nice collection of energy drink cans at the office). But, that is finally over, and I managed to get in my submissions on time. That means I’m done with the work hard part, now it’s time for some serious relaxing and recharging of batteries.
