Revision of ImageCLEF Large Scale Visual Concept Detection and Annotation Task from Wed, 02/04/2009 - 13:17

Outline

Following the tradition from 2008, ImageCLEF will offer a visual concept detection and annotation task in 2009 again.
This year, the focus lies on the extension of the task concerning the amount of data available and the amount of concepts to be annotated. In 2008 there were about 1800 images for training and 1000 images for testing available. This year the training and test set will consist of several thousand images respectively.

All images can have multiple annotations. Most of the visual concepts refer to a holistic impression of the image.

Categories for the visual concepts can be for example:

• Abstract Categories (Landscape, Family&Friends, Partylife …)
• Seasons
• Time of Day
• Persons (no, single, big groups)
• Quality (blurred, underexposed …)
• Representation (portrait, macro image, canvas …)

Altogether we think about providing ca. 50 concepts. The number will be fixed until the topic and training data release. The visual concepts are provided in a small ontology. Participants may use the hierarchical order of the concepts and the relations between concepts for solving the annotation task.

This task poses two main challenges:
1) Can image classifiers scale to the large amount of concepts and data?
2) Can an ontology (hierarchy and relations) help in large scale annotations?

Data

For the large scale visual concept detection and annotation task, we are going to use the MIR Flickr 25.000 image dataset that provides altogether 25.000 images for training and test.

Tentative Schedule

01.2.2009: registration opens for all CLEF tasks
15.3.2009: training data and concepts release
01.4.2009: test data release
01.6.2009: submission of runs
01.8.2009: release of results
15.8.2009: submission of working notes papers
30.9-2.10.2009: CLEF workshop in Corfu, Greece.

Organizers

Stefanie Nowak, Fraunhofer Institute for Digital Media Technology, Ilmenau, Germany, Stefanie.Nowak[at]idmt.fraunhofer.de
Peter Dunker, Fraunhofer Institute for Digital Media Technology, Ilmenau, Germany, Peter.Dunker[at]idmt.fraunhofer.de
Mark Huiskes, Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands, mark.huiskes[at]liacs.nl