ImageCLEF - Image Retrieval in CLEF

In 2008, ImageCLEF will offer a Visual Concept Detection Task continuing the Object Annotation Task of ImageCLEF 2006 and the Object Retrieval Task of ImageCLEF 2007. In contrast to its predecessors, the Visual Concept Detection Task of ImageCLEF 2008 will be strongly interacting with the ImageCLEF 2008 Photographic Retrieval Task.

Outline

The Visual Concept Detection Task has the objective to identify visual concepts that would help in solving the photographic retrieval task in ImageCLEF 2008.

Therefore we will publish a training database of approximately 1,800 images which are classified according to the concept hierarchy described in the following section along with their classification. Only these data may be used to train concept detection/annotation techniques.

At a later stage, we will publish a test database of 1,000 images. For each of these images participating groups are required to determine the presence/absence of the concepts.

At a later stage, the usefullness of the detected topics for the retrieval procedure will be evaluated. Groups who are interested in this will be required to detect the defined concepts in the 20,000 images which are used in the photo retrieval task. If you are interested in this, please contact the organizers as we are currently not yet sure how this can be evaluated.

Hierarchy

The images are labelled according to this class hierarchy:

Data

The visual concept detection task will work on a subset of the extended IAPR TC-12 database which is also used for the Photo Retrieval Task of ImageCLEF 2008.

You need to be registered to access these data.

Username and password to access these data is provided upon registration for ImageCLEF 2008.

Training data is now available.

• Training Data
• Training Data Annotation

Format of the annotation

27-27700.jpg 0 1 0 0 1 0 0 1 1 1 0 0 1 0 0 0 0
27-27704.jpg 0 1 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0
27-27705.jpg 0 1 0 0 1 0 0 1 0 1 0 0 1 0 0 1 0
27-27706.jpg 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
27-27709.jpg 0 1 0 1 0 0 1 0 0 0 0 0 1 0 0 1 0
27-27712.jpg 0 1 0 0 0 0 0 1 0 1 0 0 1 0 1 0 0
...

The first column gives the filename of the image and the succeeding columns denote presence/absence of the concepts in this order:
0 indoor
1 outdoor
2 person
3 day
4 night
5 water
6 road or pathway
7 vegetation
8 tree
9 mountains
10 beach
11 buildings
12 sky
13 sunny
14 partly cloudy
15 overcast
16 animal

Evaluation

We will use equal error rate to evaluate the performance of the individual runs.

Submission format is equal to the annotation format of the training data, except that you are expected to give some confidence scores for each concept to be present or absent.

That is, you have to submit a file containing the same number of columns, but each number can be an arbitrary floating point number, where higher numbers denote higher security regarding the presence of a particular concept.

Schedule

We are going to release the test data as soon as possible.

Organisers

Thomas Deselaers, RWTH Aachen University, Aachen, Germany
Allan Hanbury, Vienna University of Technology Institute of Computer Aided Automation, Pattern Recognition and Image Processing Group, Vienna, Austria