Revision of ImageCLEF 2009 Robot Vision Task from Tue, 11/10/2009 - 19:45

Introduction

ImageCLEF hosted for the first time a Robot Vision task in 2009. The task addressed the problem of topological localization of a mobile robot using visual information. Specifically, participants were asked to determine the topological location of a robot based on images acquired with a perspective camera mounted on a robot platform. See below for a more detailed description of the task.

RobotVision@ImageCLEF'09 is now over. The final results were released and are presented below.

Description of the Task

Participants are given training data consisting of an image sequence recorded in a five room subsection of an indoor environment under fixed illumination conditions and at a given time. The challenge is to build a system able to answer the question 'where are you?' (I'm in the kitchen, in the corridor, etc.) when presented with a test sequence containing images acquired in the previously observed part of the environment or in additional rooms that were not imaged in the training sequence. The test images were acquired 6-20 months later after the training sequence, possibly under different illumination settings. The system should assign each test image to one of the rooms that were present in the training sequence or indicate that the image comes from a room that was not included during training. Moreover, the system can refrain from making a decision (e.g. in the case of lack of confidence).

The algorithm must be able to provide information about the location of the robot separately for each test image (obligatory task) (e.g. when only some of the images from the test sequences are available or the sequences are scrambled). This corresponds to the problem of global topological localization. However, results can also be reported for the case when the algorithm is allowed to exploit continuity of the sequences and rely on the test images acquired before the classified image (optional task). The reported results will be compared separately and winners will be announced for both types of problems.

Examples of images from the IDOL2 database showing the interiors of the rooms, variations observed over time and caused by activity in the environment as well as introduced by changing illumination.

The competition starts with the release of annotated training and validation data. Training and validation will be performed on a subset of the publically available IDOL2 Database. The database contains image sequences acquired in a five room subsection of an office environment, under three different illumination settings and over a time frame of 6 months. The test image sequences will be released two months later (see the schedule below). The test sequences were acquired in the same environment, 20 months after the training data, and contain additional rooms that were not imaged previously.

Detailed information about the task is available for registered participants at http://cogvis.nada.kth.se/robotclef09/
Contact Andrzej Pronobis in order to obtain an account required to access the page.

Participation and Results

In 2009, a new record of 85 research groups registered for the seven sub tasks of ImageCLEF. Of these 85, 19 registered to the Robot Vision task. 7 of the registered groups submitted at least one run:

• Multimedia Information Retrieval Group, University of Glasgow, United Kingdom
• Idiap Research Institute, Martigny, Switzerland
• Faculty of Computer Science, The Alexandru Ioan Cuza University (UAIC), Iaşi, Romania
• Computer Vision & Image Understanding Department (CVIU), Institute for Infocomm Research, Singapore
• Laboratoire des Sciences de l'Information et des Systèmes (LSIS), France
• Intelligent Systems and Data Mining Group (SIMD), University of Castilla-La Mancha, Albacete, Spain
• Multimedia Information Modeling and Retrieval Group (MRIM), Laboratoire d'Informatique de Grenoble, France

A total of 27 runs were submitted, with 18 runs submitted to the obligatory task and 9 runs submitted to the optional task. In order to encourage participation, there was no limit to the number of runs that each group could submit.

The results for each run submitted to the obligatory and optional tasks are presented below.

Schedule

15.1.2009: registration opens for all CLEF task
31.3.2009: training and validation data and task release
31.5.2009: test data release
15.6.2009: submission of runs
15.7.2009: release of results
15.8.2009: submission of working notes papers
30.9-2.10.2009: CLEF workshop in Corfu, Greece.

Organizers

Barbara Caputo, Idiap Research Institute, Martigny, Switzerland, bcaputo@idiap.ch
Andrzej Pronobis, Centre for Autonomous Systems, KTH, Stockholm, Sweden, pronobis@csc.kth.se
Patric Jensfelt, Centre for Autonomous Systems, KTH, Stockholm, Sweden, patric@csc.kth.se