ImageCLEFtuberculosis

Welcome to the 2nd edition of the Tuberculosis Task!

Motivation

About 130 years after the discovery of Mycobacterium tuberculosis, the disease remains a persistent threat and a leading cause of death worldwide.

The greatest problem that can happen to a patient with tuberculosis (TB) is that the organisms become resistant to two or more of the standard drugs. In contrast to drug sensitive (DS) tuberculosis, its multi-drug resistant (MDR) form is much more difficult and expensive to recover from. Thus, early detection of the drug resistance (DR) status is of great importance for effective treatment. The most commonly used methods of DR detection are either expensive or take too much time (up to several months). Therefore, there is a need for quick and at the same time cheap methods of DR detection. One of the possible approaches for this task is based on Computed Tomography (CT) image analysis. Another challenging task is automatic detection of TB types (TBT) using CT volumes. In this subtask, five types of tuberculosis are considered: Infiltrative, Focal, Tuberculoma, Miliary and Fibro-cavernous. Lung lesions have different appearance, size and pattern depending on the TB type.

Differences compared to 2017:

• Both training and test datasets for MDR recognition task (subtask #1) are extended by means of adding several cases with extensively drug-resistant tuberculosis (XDR TB), which is a rare and more severe subtype of MDR TB.
• In case of TB type detection (subtask #2) the datasets are extended by adding new CT scans of the same patients involved in 2017, and also by introducing CT images of a few new patients.
• A new task (subtask #3) compared to 2017 is introduced which is dedicated to scoring of severity of TB cases based on chest CT images.

News

• 22.03.2018: Test set is released.

Participant registration

Please refer to the general ImageCLEF registration instructions

Schedule

• 08.11.2017: registration opens for all ImageCLEF tasks (until 27.04.2018)
• 22.01.2018: development data release starts
• 22.03.2018: test data release starts
• 01.05.2018: deadline for submitting the participants runs
• 15.05.2018: release of the processed results by the task organizers
• 31.05.2018: deadline for submission of working notes papers by the participants
• 15.06.2018: notification of acceptance of the working notes papers
• 29.06.2018: camera ready working notes papers
• 10-14.09.2018: CLEF 2018, Avignon, France

Subtasks Overview

The ImageCLEFtuberculosis task 2018 includes three independent subtasks.

Subtask #1: MDR detection

The goal of this subtask is to assess the probability of a TB patient having resistant form of tuberculosis based on the analysis of chest CT scan.

Subtask #2: TBT classification

The goal of this subtask is to automatically categorize each TB case into one of the following five types: (1) Infiltrative, (2) Focal, (3) Tuberculoma, (4) Miliary, (5) Fibro-cavernous.

Subtask #3: Severity scoring

This subtask is aimed at assessing TB severity score based on chest CT image. The Severity score is a cumulative score of severity of TB case assigned by a medical doctor. Originally, the score varied from 1 ("critical/very bad") to 5 ("very good"). In this subtask, the score value is simplified so that values 1, 2 and 3 correspond to "high severity" class, and values 4 and 5 correspond to "low severity". In the process of scoring, the medical doctors considered many factors like pattern of lesions, results of microbiological tests, duration of treatment, patient's age and some other. The goal of this subtask is to distinguish "low severity" from "high severity" based on the CT image, only.

Data collection

Subtask #1: MDR detection

For subtask #1, a dataset of 3D CT images is used along with a set of clinically relevant metadata. The dataset includes only HIV-negative patients with no relapses and having one of the two forms of tuberculosis: drug sensitive (DS) or multi-drug resistant (MDR). The MDR class includes patients with extensively drug-resistant (XDR) tuberculosis.

Subtask #2: TBT classification

The dataset used in subtask #2 includes chest CT scans of TB patients along with the TB type. Some patients include more than one scan. All scans belonging to the same patient present the same TB type.

Subtask #3: Severity scoring

The dataset for subtask #3 includes chest CT scans of TB patients along with the corresponding severity score (1 to 5) and the severity level designated as "low" and "high".

For all subtasks we provide 3D CT images with an image size per slice of 512*512 pixels and number of slices varying from about 50 to 400. All the CT images are stored in NIFTI file format with .nii.gz file extension (g-zipped .nii files). This file format stores raw voxel intensities in Hounsfield units (HU) as well the corresponding image metadata such as image dimensions, voxel size in physical units, slice thickness, etc. A freely-available tool called "VV" can be used for viewing image files. Currently, there are various tools available for reading and writing NIFTI files. Among them there are load_nii and save_nii functions for Matlab and Niftilib library for C, Java, Matlab and Python.

Moreover, for all patients in both subtasks we provide automatic extracted masks of the lungs. This material can be downloaded together with the patients CT images. The details of this segmentation can be found here.
In case the participants use these masks in their experiments, please refer to the section "Citations" at the end of this page to find the appropriate citation for this lung segmentation technique.

Remarks on the automatic lung segmentation:

The segmentations were manually analysed based on statistics on number of lungs found and size ratio of the lungs. Only those segmentations with anomalies on these statistics were visualized. The code used to segment the patients was improved considering the cases wrong segmented. After all improvements, there are still 24 patients (20 from TBT task and 4 from the MDR task) that could not be properly labelled due to the size and/or damage of one lung. In these cases, the mask only contains the label "1". Moreover, 8 patients were segmented fusing the above mentioned method and a registration-based segmentation.

Submission instructions

Please note that each group is allowed a maximum of 10 runs per subtask.

Subtask #1: MDR detection

Submit a plain text file named with the prefix MDR (e.g. MDRfree-text.txt) with the following format:

• <Patient-ID>,<Probability of MDR>

e.g.:

• MDR_TST_001,0.1
• MDR_TST_002,1
• MDR_TST_003,0.56
• MDR_TST_004,0.02

Please use a score between 0 and 1 to indicate the probability of the patient having MDR.

You need to respect the following constraints:

• Patient-IDs must be part of the predefined Patient-IDs
• All patient-IDs must be present in the runfiles
• Only use numbers between 0 and 1 for the score. Use the dot (.) as a decimal point (no commas accepted)

Subtask #2: TBT classification

Submit a plain text file named with the prefix TBT (e.g. TBTfree-text.txt) with the following format:

• <Patient-ID>,<TB-Type>

e.g.:

• TBT_TST_501,1
• TBT_TST_502,3
• TBT_TST_503,5
• TBT_TST_504,4
• TBT_TST_505,2

Please use the following Codes for the TB types:

You need to respect the following constraints:

• Patient-IDs are obtained as follows:
Image-IDs: {TBT_TST_001_01, TBT_TST_001_02, TBT_TST_001_03} --> Patient-ID: TBT_TST_001
Image-IDs: {TBT_TST_002_01} --> Patient-ID: TBT_TST_002
• Patient-IDs must be part of the predefined Patient-IDs
• All patient-IDs must be present in the runfiles
• Only use the defined codes for the various TB types
• Only use one TB type per patient

Subtask #3: Severity scoring

Submit a plain text file named with the prefix SVR (e.g. SVRfree-text.txt) with the following format:

• <Patient-ID>,<Severity score>,<Probability of "HIGH" severity>

e.g.:

• SVR_TST_001,1,0.93
• SVR_TST_002,3,0.54
• SVR_TST_003,5,0.1
• SVR_TST_004,4,0.245
• SVR_TST_005,2,0.7

Please use an integer value between 1 and 5 to indicate the severity score.
Please use a score between 0 and 1 to indicate the probability of the patient having "HIGH" severity (it corresponds to severity scores 1 to 3).

You need to respect the following constraints:

• Patient-IDs must be part of the predefined Patient-IDs
• All patient-IDs must be present in the runfiles
• Only use one integer value from 0 to 5 for the severity score
• Only use numbers between 0 and 1 for the probability. Use the dot (.) as a decimal point (no commas accepted)

Evaluation methodology

Subtask #1: MDR detection

The results will be evaluated using ROC-curves produced from the probabilities provided by the participants.

Subtask #2: TBT classification

The results will be evaluated using unweighted Cohen’s Kappa (sample Matlab code).

Subtask #3: Severity scoring

The results will be evaluated considering this subtask as a binary classification problem and as a regression problem. The classification problem will be evaluated using ROC-curves produced from the probabilities provided by the participants. For the regression problem, mean square error will be used.

Results

Subtask #1: MDR detection

Subtask #2: TBT classification

Subtask #3: Severity scoring

Citations

• When referring to the ImageCLEFtuberculosis 2018 task general goals, general results, etc. please cite the following publication which will be published by September 2018:
  
  • Yashin Dicente Cid, Vitali Liauchuk, Vassili Kovalev, Henning Müller, Overview of ImageCLEFtuberculosis 2018 - Detecting multi-drug resistance, classifying tuberculosis type, and assessing severity score, CLEF working notes, CEUR, 2018.
  • BibTex:
    @Inproceedings{ImageCLEFTBoverview2018,
    author = {Dicente Cid, Yashin and Liauchuk, Vitali and Kovalev, Vassili and and M\"uller, Henning},
    title = {Overview of {ImageCLEFtuberculosis} 2018 - Detecting multi-drug resistance, classifying tuberculosis type, and assessing severity score},
    booktitle = {CLEF2018 Working Notes},
    series = {{CEUR} Workshop Proceedings},
    year = {2018},
    volume = {},
    publisher = {CEUR-WS.org $<$http://ceur-ws.org$>$},
    pages = {},
    month = {September 10-14},
    address = {Avignon, France}
    }
• When referring to the ImageCLEF 2018 task general goals, general results, etc. please cite the following publication which will be published by September 2018:
  
  • Bogdan Ionescu, Henning Müller, Mauricio Villegas, Alba García Seco de Herrera, Carsten Eickhoff, Vincent Andrearczyk, Yashin Dicente Cid, Vitali Liauchuk, Vassili Kovalev, Sadid A. Hasan, Yuan Ling, Oladimeji Farri, Joey Liu¡, Matthew Lungren, Duc-Tien Dang-Nguyen, Luca Piras, Michael Riegler, Liting Zhou, Mathias Lux, Cathal Gurrin, Overview of ImageCLEF 2018: Challenges, Datasets and Evaluation. In: Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Ninth International Conference of the CLEF Association (CLEF 2018), Avignon, France, LNCS Lecture Notes in Computer Science, Springer (September 10-14 2018)
  • BibTex:
    @inproceedings{ImageCLEF18,
    author = {Bogdan Ionescu and Henning M\"uller and Mauricio Villegas and Alba Garc\'ia Seco de Herrera and Carsten Eickhoff and Vincent Andrearczyk and Yashin Dicente Cid and Vitali Liauchuk and Vassili Kovalev and Sadid A. Hasan and Yuan Ling and Oladimeji Farri and Joey Liu and Matthew Lungren and Duc-Tien Dang-Nguyen and Luca Piras and Michael Riegler and Liting Zhou and Mathias Lux and Cathal Gurrin},
    title = {{Overview of ImageCLEF 2018}: Challenges, Datasets and Evaluation},
    booktitle = {Experimental IR Meets Multilinguality, Multimodality, and Interaction},
    series = {Proceedings of the Ninth International Conference of the CLEF Association (CLEF 2018)},
    year = {2018},
    volume = {},
    publisher = {{LNCS} Lecture Notes in Computer Science, Springer},
    pages = {},
    month = {September 10-14},
    address = {Avignon, France}
    }
• When using the provided mask of the lungs , please cite the following publication:
  • Yashin Dicente Cid, Oscar A. Jiménez-del-Toro, Adrien Depeursinge, and Henning Müller, Efficient and fully automatic segmentation of the lungs in CT volumes. In: Goksel, O., et al. (eds.) Proceedings of the VISCERAL Challenge at ISBI. No. 1390 in CEUR Workshop Proceedings (Apr 2015)
  • BibTex:

    @inproceedings{DJD2015,

    Title = {Efficient and fully automatic segmentation of the lungs in CT volumes},
    Booktitle = {Proceedings of the {VISCERAL} Anatomy Grand Challenge at the 2015 {IEEE ISBI}},
    Author = {Dicente Cid, Yashin and Jim{\'{e}}nez del Toro, Oscar Alfonso and Depeursinge, Adrien and M{\"{u}}ller, Henning},
    Editor = {Goksel, Orcun and Jim{\'{e}}nez del Toro, Oscar Alfonso and Foncubierta-Rodr{\'{\i}}guez, Antonio and M{\"{u}}ller, Henning},
    Keywords = {CAD, lung segmentation, visceral-project},
    Month = may,
    Series = {CEUR Workshop Proceedings},
    Year = {2015},
    Pages = {31-35},
    Publisher = {CEUR-WS},
    Location = {New York, USA}
    }

Organizers

• Vassili Kovalev <vassili.kovalev(at)gmail.com>, Institute for Informatics, Minsk, Belarus
• Henning Müller <henning.mueller(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland
• Vitali Liauchuk <vitali.liauchuk(at)gmail.com>, Institute for Informatics, Minsk, Belarus
• Yashin Dicente Cid <yashin.dicente(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland

Acknowledgements