You are here

ImageCLEFlifelog

Welcome to the 3rd edition of the Lifelog Task!

Motivation

An increasingly wide range of personal devices, such as smartphones, video cameras as well as wearable devices that allow capturing pictures, videos, and audio clips for every moment of our lives are becoming available. Considering the huge volume of data created, there is a need for systems that can automatically analyse the data in order to categorize, summarize and also query to retrieve the information the user may need.

Despite the increasing number of successful related workshops and panels ( JCDL 2015 , iConf 2016 , ACM MM 2016 , ACM MM 2017 , ICMR 2018 ) lifelogging has seldom been the subject of a rigorous comparative benchmarking exercise as, for example, the new lifelog evaluation task at NTCIR-13 or the last editions of the ImageCLEFlifelog task. In this edition of this task we aim to bring the attention of lifelogging to an as wide as possible audience and to promote research into some of the key challenges of the coming years.

News

  • 26.11.2018: Development dataset released here.

Schedule

  • 05.11.2018: Registration opens
  • 26.11.2018: Development data release
  • 18.03.2019: Test data release
  • 26.04.2019: Registration closes
  • 01.05.2019 : Deadline for submission of runs by the participants 11:59:59 PM GMT.
  • 17.05.2019: Release of processed results by the task organizers.
  • 24.05.2019: Deadline for submission of working notes papers by the participants
  • 14.06.2019: Notification of acceptance of the working notes papers.
  • 28.06.2019: Camera ready working notes papers.
  • 09.-12.09.2019: CLEF 2019, Lugano, Switzerland

Task description

Dataset

The task will be split into two related subtasks using a completely new rich multimodal dataset which consists of 42 days of data from two lifeloggers, namely: images (1,500-2,500 per day from wearable cameras), visual concepts (automatically extracted visual concepts with varying rates of accuracy), semantic content (semantic locations, semantic activities) based on sensor readings (via the Moves App) on mobile devices, biometrics information (heart rate, galvanic skin response, calorie burn, steps, continual blood glucose, etc.), music listening history, computer usage (frequency of typed words via the keyboard and information consumed on the computer via ASR of on-screen activity on a per-minute basis).

SubTask 1:Solve my life puzzle (Puzzle)

Given a period of time, e.g., "From 13 August to 16 August" or "Every Saturday", the participants should analyse the lifelog data and provide a summarisation based on the selected concepts (provided by the task organizers) of Activities of Daily Living (ADL) and the environmental settings / contexts in which these activities take place.

SubTask 2: Lifelog moment retrieval (LMRT)

This sub-task follows the success of the LMRT sub-task in ImageCLEFlifelog2018 with some minor adjustments. The participants have to retrieve a number of specific predefined activities in a lifelogger's life. For example, they should return the relevant moments for the query “Find the moment(s) when I was shopping”. Particular attention should be paid to the diversification of the selected moments with respect to the target scenario. The ground truth for this subtask was created using manual annotation.

The ground truth for this subtask was created using manual annotation.

Submitting a working notes paper to CLEF

Upon the completion of the task, participating teams are expected to present their systems in a working note paper, regardless their results. You should keep in mind that the main goal of the lab is not to win the benchmark but compare techniques based on the same data, so everyone can learn from the results. Authors are invited to submit using the LNCS proceedings format.

The CLEF 2019 working notes will be published in the CEUR-WS.org proceedings, facilitating the indexing by DBLP. According to the CEUR-WS policies, a light review of the working notes will be conducted by the task organizers to ensure quality.

Working notes will have to be submitted before 24th May 2019 11:59 pm - midnight - Central European Summer Time, through the EasyChair submission system. The working notes papers are technical reports written in English and describing the participating systems and the conducted experiments. To avoid redundancy, the papers should *not* include a detailed description of the actual task, data set and experimentation protocol. Instead of this, the papers are required to cite both the general ImageCLEF overview paper and the corresponding lifelog task overview paper, and to present the official results returned by the organizers. Bibtex references will be available soon. A general structure for the paper should provide at a minimum the following information:

  1. Title
  2. Authors
  3. Affiliations
  4. Email addresses of all authors
  5. The body of the text. This should contain information on:
    • tasks performed
    • main objectives of experiments
    • approach(es) used and progress beyond state-of-the-art
    • resources employed
    • results obtained
    • analysis of the results
    • perspectives for future work

The paper should not exceed 12 pages, and further instructions on how to write and submit your working notes will be available soon on this page :

Recommended Reading

[1] Duc-Tien Dang-Nguyen, Luca Piras, Michael Riegler, Liting Zhou, Mathias Lux, and Cathal Gurrin, "Overview of ImageCLEFlifelog 2018: Daily Living Understanding and Lifelog Moment Retrieval", CLEF2018 Working Notes, Avignon, France, 2018.

[2] Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Duc-Tien Dang-Nguyen, Michael Riegler, Luca Piras, "Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge", ICMR '18 International Conference on Multimedia Retrieval
Yokohama, Japan - June 11 - 14, 2018.

[3] Duc-Tien Dang-Nguyen, Luca Piras, Michael Riegler, Giulia Boato, Liting Zhou, Cathal Gurrin, "Overview of ImageCLEFlifelog 2017: Lifelog Retrieval and Summarization", CLEF2017 Working Notes, Dublin, Ireland, 2017, vol 1866.

[4] Cathal Gurrin, Xavier Giro-i-Nieto, Petia Radeva, Mariella Dimiccoli, Håvard Johansen, Hideo Joho, Vivek K Singh, "LTA 2016: The First Workshop on Lifelogging Tools and Applications", ACM Multimedia, Amsterdam, The Netherlands, 2016.

[5] Cathal Gurrin, Xavier Giro-i-Nieto, Petia Radeva, Mariella Dimiccoli, Duc Tien Dang Nguyen, Hideo Joho, "LTA 2017: The Second Workshop on Lifelogging Tools and Applications", ACM Multimedia, Mountain View, CA USA, 2017.

[6] Cathal Gurrin, Hideo Joho, Frank Hopfgartner, Liting Zhou, Rami Albatal, "Overview of NTCIR-12 Lifelog Task", Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies, Tokyo, Japan, 2016.

[7] Duc-Tien Dang-Nguyen, Luca Piras, Giorgio Giacinto, Giulia Boato, Francesco GB De Natale, "Multimodal Retrieval with Diversification and Relevance Feedback for Tourist Attraction Images", ACM Transactions on Multimedia Computing, Communications, and Applications, vol 13, n° 4, 2017.

[8] Duc-Tien Dang-Nguyen, Luca Piras, Giorgio Giacinto, Giulia Boato, Francesco GB De Natale, "A hybrid approach for retrieving diverse social images of landmarks", IEEE International Conference on Multimedia and Expo (ICME), Turin, Italy, 2015.

[9] Working notes of the 2015 MediaEval Retrieving Diverse Social Images task, CEUR-WS.org, Vol. 1436, ISSN: 1613-0073.

[10] B. Ionescu, A.L. Gînscă, B. Boteanu, M. Lupu, A. Popescu,H. Müller, Div150Multi: A Social Image Retrieval Result Diversification Dataset with Multi-topic Queries”, ACM MMSys, Klagenfurt, Austria, 2016.

Helpful tools and resources

Eyeaware lifelogging framework;

OpenCV – Open Source Computer Vision;

LIRE: Lucence Image Retrieval;

trec_eval scoring software;

ImageCLEF - Image Retrieval in CLEF;

Weka Data Mining Software;

Nvidia DIGITS;

Caffee deep learning framework;

Creative Commons.

Citations

  • When referring to the ImageCLEFlifelog 2019 task general goals, general results, etc. please cite the following publication:
    • Duc-Tien Dang-Nguyen, Luca Piras, Michael Riegler, Minh-Triet Tran, Mathias Lux, and Cathal Gurrin. 2019. Overview of ImageCLEFlifelog 2019: Solve my life puzzle and Lifelog Moment Retrieval. In CLEF2019 Working Notes (CEUR Workshop Proceedings). CEUR-WS.org <http://ceur-ws.org>, Lugano, Switzerland.
    • BibTex:

      @Inproceedings{LifeLogTask19_CLEF,
        author = {Duc-Tien Dang-Nguyen and Luca Piras and Michael Riegler and Minh-Triet Tran and Mathias Lux and Cathal Gurrin},
        title = {{Overview of ImageCLEFlifelog 2019: Solve my life puzzle and Lifelog Moment Retrieval}},
        booktitle = {CLEF2019 Working Notes},
        series = {{CEUR} Workshop Proceedings},
        year = {2019},
        volume = {},
        publisher = {CEUR-WS.org $<$http://ceur-ws.org$>$},
        pages = {},
        month = {September 09-12},
        address = {Lugano, Switzerland},
        }

      Organizers

      • Duc-Tien Dang-Nguyen <ductien.dangnguyen(at)uib.no>, University of Bergen, Norway
      • Luca Piras <luca.piras(at)diee.unica.it>, University of Cagliari, Cagliari, Italy
      • Michael Riegler <michael(at)simula.no>, University of Oslo, Norway
      • Minh-Triet Tran <tmtriet(at)hcmus.edu.vn>, University of Science, Ho Chi Minh City, Vietnam
      • Mathias Lux <mlux(at)itec.aau.at>, Klagenfurt University, Austria
      • Cathal Gurrin <cgurrin(at)computing.dcu.ie>, Dublin City University, Ireland