Panels


  • The Four Ladies of Experimental Evaluation
    Chair: Donna Harman, NIST, USA

    • Panelists:
      Noriko Kando, NII, Japan
      Mounia Lalmas, University of Glasgow, UK
      Carol Peters, ISTI CNR, Italy

    Abstract:
    The goal of the panel is to present some of the main lessons that we have learned in well over a decade of experimental evaluation and to promote discussion with respect to what the future objectives in this field should be.

    Slides

  • A PROMISE for Experimental Evaluation
    Chair: Jussi Karlgren, Swedish Institute of Computer Science, Sweden

    • Panelists:
      Nicola Ferro, University of Padua, Italy
      Allan Hanbury, Information Retrieval Facility (IRF), Austria
      Maarten de Rijke, University of Amsterdam, The Netherlands
      Giuseppe Santucci, Sapienza University of Rome, Italy

    Abstract:
    Participative Research labOratory for Multimedia and Multilingual Information Systems Evaluation (PROMISE) is a Network of Excellence, starting in conjunction with this first independent CLEF 2010 conference, and designed to support and develop the evaluation of multilingual and multimedia information access systems, largely through the activities taking place in Cross-Language Evaluation Forum (CLEF) today, and taking it forward in important new ways.

    PROMISE is coordinated by the University of Padua, and comprises 10 partners: the Swedish Institute for Computer Science, the University of Amsterdam, Sapienza University of Rome, University of Applied Sciences of Western Switzerland, the Information Retrieval Facility, the Zurich University of Applied Sciences, the Humboldt University of Berlin, the Evaluation and Language Resources Distribution Agency, and the Centre for the Evaluation of Language Communication Technologies.

    The single most important step forward for multilingual and multimedia information access which PROMISE will work towards is to provide an open evaluation infrastructure in order to support automation and collaboration in the evaluation process.