logo

CLEF 2010

logo

Conference on Multilingual and Multimodal Information Access Evaluation
20-23 September 2010, Padua, Italy

 

Download Flyer


Program at a Glance


Conference

 

9:00-9:30

9:30-10:00

10:00-11:00

11:00-11:30

11:30-13:00

13:00-14:30

14:30-15:30

15:30-16:30

16:30-17:00

17:00-18:30

19:00-*

Monday
20 September

Registration

Welcome and Opening

Keynote Talk

  • Keynote Talk
    IR Between Science and Engineering, and the Role of Experimentation
    Norbert Fuhr, University of Duisburg-Essen, Germany

Coffee Break

Full Papers Session

  • Resources, Tools, and Methods
    3 Full Papers, 30 mins each

Lunch

Panel

  • Panel: A PROMISE for Experimental Evaluation
    Chair: Jussi Karlgren, Swedish Institute of Computer Science, Sweden

Full Papers Session

  • Experimental Collections and Datasets (1)
    2 full papers, 30 mins each

Coffee Break

Labs Overview

  • Labs Overviews
    5 labs, 20 mins each
  • ImageCLEF RespubliQA WePS CLEF-IP PAN

Reception

Tuesday
 21 September

Registration

Short Papers Session

  • Experimental Collections and Datasets
    2 short papers, 15 mins each

Other Evaluation Initiatives

  • Other Evaluation Initiatives
    TREC
    NTCIR
    FIRE
    INEX
    ROMIP

Coffee Break

Keynote Talk

  • Keynote Talk
    Retrieval Evaluation in Practice

    Ricardo Baeza-Yates, Yahoo! Research, Spain

Short Papers Session

  • Evaluation Methodologies and Metrics
    2 short papers, 15 mins each

Lunch

Panel

  • Panel: The four Ladies of Experimental Evaluation
    Chair: Donna Harman, NIST, USA
  • Noriko Kando, NII, Japan Mounia Lalmas, University of Glasgow, UK Carol Peters, ISTI CNR, Italy

Full Papers Session

  • Evaluation Methodologies and Metrics (2)
    3 full papers, 30 mins each

Walk around and Social dinner


Labs

 

9:00-9:30

9:30-11:00

11:00-11:30

11:30-13:00

13:00-14:30

14:30-16:00

16:00-16:30

16:30-18:00

18:00-*

Wednesday
22 September

Labs Kick-off

Parallel Lab Sessions

  • Parallel Lab Sessions
    ImageCLEF

    CLEF-IP

    PAN

Coffee Break

Parallel Lab Sessions

  • Parallel Lab Sessions
    ImageCLEF

    CLEF-IP

    PAN

Lunch

Parallel Lab Sessions

  • Parallel Lab Sessions
    ImageCLEF

    CLEF-IP

    ResPubliQA

Coffee Break

Parallel Lab Sessions

  • Parallel Lab Sessions
    ImageCLEF

    CLEF-IP

    ResPubliQA

Poster Session with Reception

Thursday
23 September

Reports out from the Labs

  • Reports out from the Labs
    CLEF-IP

    PAN

    RespubliQA

Parallel Lab Sessions

  • Parallel Lab Sessions
    ImageCLEF

    WePS

    LogCLEF

Coffee Break

Parallel Lab Sessions

  • Parallel Lab Sessions
    ImageCLEF

    WePS

    LogCLEF

Lunch

Parallel Lab Sessions

  • Parallel Lab Sessions
    ImageCLEF

    WePS

    CriES

Coffee Break

Parallel Labs Sessions

  • Parallel Lab Sessions
    ImageCLEF

    WePS

    CriES

Reports out from the Labs

  • Reports out from the Labs
    ImageCLEF

    WePS

    LogCLEF

    CriES

Closing



Monday, 20 September 2010

Monday, 20 September 2010, Aula Magna, Palazzo del Bo

 

09:30–10:00

Welcome and Opening

 

 

10:00–11:00

Keynote Talk
Chair: Maarten de Rijke, University of Amsterdam, The Netherlands

 

IR Between Science and Engineering, and the Role of Experimentation
Norbert Fuhr, University of Duisburg-Essen, Germany

 

 

11:00–11:30

Coffee Break

 

 

11:30:–13:00

Resources, Tools, and Methods
Chair: Johannes Leveling, Dublin City University, Ireland

 

A Dictionary- and Corpus-Independent Statistical Lemmatizer for Information Retrieval in Low Resource Languages
Aki Loponen, University of Tampere, Finland
Kalervo Järvelin, University of Tampere, Finland

 

A New Approach for Cross-Language Plagiarism Analysis
Rafael Corezola Pereira, Universidade Federal do Rio Grande do Sul (UFRGS), Brazil
Viviane P. Moreira, Universidade Federal do Rio Grande do Sul (UFRGS), Brazil
Renata Galante, Universidade Federal do Rio Grande do Sul (UFRGS), Brazil

 

Creating a Persian-English Comparable Corpus
Homa Baradaran Hashemi, University of Tehran, Iran
Azadeh Shakery, University of Tehran, Iran
Heshaam Faili, University of Tehran, Iran

 

 

13:00–14:30

Lunch

 

 

14:30–15:30

Panel: A PROMISE for Experimental Evaluation
Chair: Jussi Karlgren, Swedish Institute of Computer Science, Sweden

 

Panelists: Nicola Ferro, University of Padua, Italy
Allan Hanbury, Information Retrieval Facility (IRF), Austria
Maarten de Rijke, University of Amsterdam, The Netherlands
Giuseppe Santucci, Sapienza University of Rome, Italy

 

 

15:30–16:30

Experimental Collections and Datasets (1)

Chair : Jaques Savoy, University of Neuchâtel, Switzerland

 

Validating Query Simulators: An Experiment Using Commercial Searches and Purchases
Bouke Huurnink, University of Amsterdam, The Netherlands
Katja Hofmann, University of Amsterdam, The Netherlands
Maarten de Rijke, University of Amsterdam, The Netherlands
Marc Bron, University of Amsterdam, The Netherlands

 

Using Parallel corpora for Multilingual(Multi-Document) Summarisation Evaluation
Marco Turchi, European Commission - Joint Research Centre (JRC), Italy
Josef Steinberger, European Commission - Joint Research Centre (JRC), Italy
Ralf Steinberger, European Commission - Joint Research Centre (JRC), Italy
Mijail Kabadjov, European Commission - Joint Research Centre (JRC), Italy

 

 

16:30–17:00

Coffee Break

 

 

17:00–18:30

Labs Overviews

Chair: Henning Müller, University of Applied Sciences Western Switzerland, Switzerland

 

ImageCLEF
RespubliQA
WePS
CLEF-IP
PAN

 

 

19:00–21:00

Reception

Tuesday, 21 September 2010

Tuesday, 21 September 2010, Aula Magna, Palazzo del Bo

 

09:30–10:00

Experimental Collections and Datasets (2)

Chair: Theodore Kalamboukis, Athen University of Economics and Business, Greece

 

MapReduce for information retrieval evaluation: “Let's quickly test this on 12 TB of data”
Djoerd Hiemstra, University of Twente, The Netherlands
Claudia Hauff, University of Twente, The Netherlands

 

Which Log for which Information? Gathering Multilinguality Data from Different Log File Types
Maria Gäde, Humboldt-Universität zu Berlin, Germany
Vivien Petras, Humboldt-Universität zu Berlin, Germany
Juliane Stiller, Humboldt-Universität zu Berlin, Germany

 

 

10:00–11:00

Other Evaluation Initiatives

Chair: Nicola Ferro, University of Padua, Italy

 

TREC, Ellen Voorhees, NIST, USA
NTCIR, Noriko Kando, NII, Japan
INEX, Jaap Kamps, University of Amsterdam, The Netherlands
ROMIP, Pavel Braslavski, Yandex, Russia
FIRE, Dipasree Pal, Indian Statistical Institute, India

 

 

11:00–11:30

Coffee Break

 

 

11:30:–12:30

Keynote Talk

Chair: Alan Smeaton, Dublin City University, Ireland

 

Retrieval Evaluation in Practice
Ricardo Baeza-Yates, Yahoo! Research, Spain

 

 

12:30–13:00

Evaluation Methodologies and Metrics (1)

Chair: Christa Womser-Hacker, University of Hildesheim, Germany

 

Examining the Robustness of Evaluation Metrics for Patent Retrieval with Incomplete Relevance Judgements
Walid Magdy, Dublin City University, Ireland
Gareth Jones, Dublin City University, Ireland

 

On the Evaluation of Entity Profiles
Maarten de Rijke, University of Amsterdam, The Netherlands
Krisztian Balog, University of Amsterdam, The Netherlands
Toine Bogers, Royal School of Library & Information Science, Denmark
Antal van den Bosch, Tilburg University, The Netherlands

 

 

13:00–14:30

Lunch

 

 

14:30–15:30

Panel: The four Ladies of Experimental Evaluation

Chair: Donna Harman, NIST, USA

 

Panelists:
Noriko Kando, NII, Japan
Mounia Lalmas, University of Glasgow, UK
Carol Peters, ISTI CNR, Italy

 

 

15:30–17:00

Evaluation Methodologies and Metrics (2)

Chair: Ray Larson, University of California, Berkeley, USA

 

Evaluating Information Extraction
Andrea Esuli, ISTI-CNR, Italy
Fabrizio Sebastiani, , ISTI-CNR, Italy

 

Tie-breaking Bias: Effect of an Uncontrolled Parameter on Information Retrieval Evaluation
Guillaume Cabanac, Université de Toulouse, France
Gilles Hubert, Université de Toulouse, France
Mohand Boughanem, Université de Toulouse, France
Claude Chrisment, Université de Toulouse, France

 

Automated Component-Level Evaluation: Present and Future
Allan Hanbury, Information Retrieval Facility, Austria
Henning Müller, University of Applied Sciences Western Switzerland, Switzerland

 

 

17:00–17:30

Logistics of Social Events

 

 

18:00–23:00

Walk around and Social Dinner

Wednesday, 22 September 2010

Wednesday, 22 September 2010, Faculty of Statistical Sciences

 

09:00–09:30

Labs Kick-off

 

 

09:30–11:00

Parallel Lab Sessions

 

ImageCLEF
CLEF-IP
PAN

 

 

11:00–11:30

Coffee Break

 

 

11:30:–13:00

Parallel Lab Sessions

 

ImageCLEF
CLEF-IP
PAN

 

 

13:00–14:30

Lunch

 

 

14:30:–16:00

Parallel Lab Sessions

 

ImageCLEF
CLEF-IP
RespubliQA

 

 

16:00–16:30

Coffee Break

 

 

16:30:–18:00

Parallel Lab Sessions

 

ImageCLEF
CLEF-IP
RespubliQA

 

 

18:00–20:00

Poster Session

Thursday, 23 September 2010

Thursday, 23 September 2010, Faculty of Statistical Sciences

09:00–09:30

Reports out from the Labs

Chair: Martin Braschler, Zurich University of Applied Sciences, Switzerland

 

CLEF-IP
PAN
RespubliQA

 

 

09:30–11:00

Parallel Lab Sessions

 

ImageCLEF
WePS
LogCLEF

 

 

11:00–11:30

Coffee Break

 

 

11:30:–13:00

Parallel Lab Sessions

 

ImageCLEF
WePS
LogCLEF

 

 

13:00–14:30

Lunch

 

 

14:30:–16:00

Parallel Lab Sessions

 

ImageCLEF
WePS
CriES

 

 

16:00–16:30

Coffee Break

 

 

16:30:–18:00

Parallel Lab Sessions

 

ImageCLEF
WePS
CriES

 

 

18:00–18:40

Reports out from the Labs

Chair: Emanuele Pianta, CELCT, Italy

 

ImageCLEF
WePS
LogCLEF
CriES

 

 

18:40–19:00

Closing