Revision of ImageCLEF 2009 medical retrieval task from Wed, 05/13/2009 - 13:24

Overview

The medical retrieval task of ImageCLEF2009 will use a similar database to 2008 but with a larger number of images.
There will be two types of tasks in 2009, the retrieval of similar images for a precise information need and the retrieval of similar cases (test task). Retrieval of cases is more complex to evaluate and thus we will only provide with a small number of test cases to plan an eventually larger task for 2010.

The data set used contains all images from articles published in Radiology and Radiographics including the text of the captions and a link to the html of the full text articles. Over 70,000 images are currently available in this way.

Schedule:

31.1.2009: inscription for CLEF (done)
1.4.2009: release of the data set (done)
25.4.2009-1.5.: release of the topics (done)
7.6.2009: Submission deadline for runs
15.7.2009: end of the relevance judgment process
20.7.2009: distribution of results to the participants
15.8.2009: submission deadline for working notes papers
16.9.2009: pre-CLEF workshop on visual information retrieval evaluation
17.-19.9.2009: CLEF workshop

Data Download

Our database distribution includes an xml file with the image id, the captions of the images, the titles of the journal articles in which the image had appeared and the PubMed ID of the journal article. In addition, a compressed file containing the over 70,000 images will be provided.

The data is now available for download at http://skynet.ohsu.edu/iclef09_data .
The data includes the data for the five test topics for case-based retrieval.
Please login using the user id and password provided during registration.

Topics

30.4.2009:
25 ad-hoc topic are now available at http://skynet.ohsu.edu/iclef09_data/Topics
Please login using the user id and password provided during registration.
The topics are provided in xml format, with descriptions in English, French and German. 2-4 sample images (in jpeg format) are also provided for each image.

11.5.2009:
5 case-based topics have been made available at ttp://skynet.ohsu.edu/iclef09_data/Topics/caseBasedTopics/
The topics are provided in xml format, with captions only in English as this is a test case. 4-5 images per topic are available for the visual retrieval part.
Results in this case are not images but entire research articles. The topics are numbered 26-30 but results need to be submitted in separate files from the image-based topics.

Data Submission

Please ensure that your submissions are compliant with the trec_eval format prior to submission.
We will reject any runs that do not meet the required format.
Also, please note that each group is allowed a maximum of 10 runs. The qrels will be distributed among the participants, so
further runs can be evaluated for the working notes papers by the participants.
Do not hesitate to ask if you have questions regarding the trec_eval format.

At the time of submission, the following information about each run will be requested. Please let us know if you would like clarifications on how to classify your runs.

1. What was used for the retrieval: Image, text or mixed (both)
2. Was other training data used?
3. Run type: Automatic, Manual, Interactive
4. Query Language

Results