CLEF 2010

Welcome and Opening

Keynote Talk
Chair: Maarten de Rijke, University of Amsterdam, The Netherlands

IR Between Science and Engineering, and the Role of Experimentation
Norbert Fuhr, University of Duisburg-Essen, Germany

Coffee Break

Resources, Tools, and Methods
Chair: Johannes Leveling, Dublin City University, Ireland

A Dictionary- and Corpus-Independent Statistical Lemmatizer for Information Retrieval in Low Resource Languages
Aki Loponen, University of Tampere, Finland
Kalervo Järvelin, University of Tampere, Finland

A New Approach for Cross-Language Plagiarism Analysis
Rafael Corezola Pereira, Universidade Federal do Rio Grande do Sul (UFRGS), Brazil
Viviane P. Moreira, Universidade Federal do Rio Grande do Sul (UFRGS), Brazil
Renata Galante, Universidade Federal do Rio Grande do Sul (UFRGS), Brazil

Creating a Persian-English Comparable Corpus
Homa Baradaran Hashemi, University of Tehran, Iran
Azadeh Shakery, University of Tehran, Iran
Heshaam Faili, University of Tehran, Iran

Lunch

Panel: A PROMISE for Experimental Evaluation
Chair: Jussi Karlgren, Swedish Institute of Computer Science, Sweden

Panelists: Nicola Ferro, University of Padua, Italy
Allan Hanbury, Information Retrieval Facility (IRF), Austria
Maarten de Rijke, University of Amsterdam, The Netherlands
Giuseppe Santucci, Sapienza University of Rome, Italy

Experimental Collections and Datasets (1)

Chair : Jaques Savoy, University of Neuchâtel, Switzerland

Validating Query Simulators: An Experiment Using Commercial Searches and Purchases
Bouke Huurnink, University of Amsterdam, The Netherlands
Katja Hofmann, University of Amsterdam, The Netherlands
Maarten de Rijke, University of Amsterdam, The Netherlands
Marc Bron, University of Amsterdam, The Netherlands

Using Parallel corpora for Multilingual(Multi-Document) Summarisation Evaluation
Marco Turchi, European Commission - Joint Research Centre (JRC), Italy
Josef Steinberger, European Commission - Joint Research Centre (JRC), Italy
Ralf Steinberger, European Commission - Joint Research Centre (JRC), Italy
Mijail Kabadjov, European Commission - Joint Research Centre (JRC), Italy

Coffee Break

Labs Overviews

Chair: Henning Müller, University of Applied Sciences Western Switzerland, Switzerland

ImageCLEF
RespubliQA
WePS
CLEF-IP
PAN

Reception

Experimental Collections and Datasets (2)

Chair: Theodore Kalamboukis, Athen University of Economics and Business, Greece

MapReduce for information retrieval evaluation: “Let's quickly test this on 12 TB of data”
Djoerd Hiemstra, University of Twente, The Netherlands
Claudia Hauff, University of Twente, The Netherlands

Which Log for which Information? Gathering Multilinguality Data from Different Log File Types
Maria Gäde, Humboldt-Universität zu Berlin, Germany
Vivien Petras, Humboldt-Universität zu Berlin, Germany
Juliane Stiller, Humboldt-Universität zu Berlin, Germany

Other Evaluation Initiatives

Chair: Nicola Ferro, University of Padua, Italy

TREC, Ellen Voorhees, NIST, USA
NTCIR, Noriko Kando, NII, Japan
INEX, Jaap Kamps, University of Amsterdam, The Netherlands
ROMIP, Pavel Braslavski, Yandex, Russia
FIRE, Dipasree Pal, Indian Statistical Institute, India

Coffee Break

Keynote Talk

Chair: Alan Smeaton, Dublin City University, Ireland

Retrieval Evaluation in Practice
Ricardo Baeza-Yates, Yahoo! Research, Spain

Evaluation Methodologies and Metrics (1)

Chair: Christa Womser-Hacker, University of Hildesheim, Germany

Examining the Robustness of Evaluation Metrics for Patent Retrieval with Incomplete Relevance Judgements
Walid Magdy, Dublin City University, Ireland
Gareth Jones, Dublin City University, Ireland

On the Evaluation of Entity Profiles
Maarten de Rijke, University of Amsterdam, The Netherlands
Krisztian Balog, University of Amsterdam, The Netherlands
Toine Bogers, Royal School of Library & Information Science, Denmark
Antal van den Bosch, Tilburg University, The Netherlands

Lunch

Panel: The four Ladies of Experimental Evaluation

Chair: Donna Harman, NIST, USA

Panelists:
Noriko Kando, NII, Japan
Mounia Lalmas, University of Glasgow, UK
Carol Peters, ISTI CNR, Italy

Evaluation Methodologies and Metrics (2)

Chair: Ray Larson, University of California, Berkeley, USA

Evaluating Information Extraction
Andrea Esuli, ISTI-CNR, Italy
Fabrizio Sebastiani, , ISTI-CNR, Italy

Tie-breaking Bias: Effect of an Uncontrolled Parameter on Information Retrieval Evaluation
Guillaume Cabanac, Université de Toulouse, France
Gilles Hubert, Université de Toulouse, France
Mohand Boughanem, Université de Toulouse, France
Claude Chrisment, Université de Toulouse, France

Automated Component-Level Evaluation: Present and Future
Allan Hanbury, Information Retrieval Facility, Austria
Henning Müller, University of Applied Sciences Western Switzerland, Switzerland

Logistics of Social Events

Walk around and Social Dinner

Labs Kick-off

Parallel Lab Sessions

ImageCLEF
CLEF-IP
PAN

Coffee Break

Parallel Lab Sessions

ImageCLEF
CLEF-IP
PAN

Lunch

Parallel Lab Sessions

ImageCLEF
CLEF-IP
RespubliQA

Coffee Break

Parallel Lab Sessions

ImageCLEF
CLEF-IP
RespubliQA

Poster Session

Reports out from the Labs

Chair: Martin Braschler, Zurich University of Applied Sciences, Switzerland

CLEF-IP
PAN
RespubliQA

Parallel Lab Sessions

ImageCLEF
WePS
LogCLEF

Coffee Break

Parallel Lab Sessions

ImageCLEF
WePS
LogCLEF

Lunch

Parallel Lab Sessions

ImageCLEF
WePS
CriES

Coffee Break

Parallel Lab Sessions

ImageCLEF
WePS
CriES

Reports out from the Labs

Chair: Emanuele Pianta, CELCT, Italy

ImageCLEF
WePS
LogCLEF
CriES

Closing