ImageCLEFrecommending

Motivation

In recent years cultural heritage organisations have made considerable efforts to digitise their collections, and this trend is expected to continue due to organisational goals and national cultural policies. Thus media archives have not only exponentially increased in size, but now hold contents in various modalities (video, image, text). Even when structured metadata is available it is still difficult to discover the contents of media archives and allow users to navigate multiperspectivity in media collections. Content-based recommendation systems can help but there is limited understanding how well these perform and how relevant they are for the final end users. Moreover, the system used so far have not addressed the new user requirements of more transparency and explainability of the algorithms used.

Lessons learned:

To be added soon.

News

To be added soon.

• The submission formats for the recommending items and the recommending editorials sub-tasks are available. Posted 04 April 2023
• The training/development datasets are available on GitHub. Posted 06 March 2023

Task Description

The task targets a key infrastructure for researchers and heritage professionals: Europeana. With over 53 million records, the single search bar that served as the main access point was identified as a bottleneck by many users. Thus, the strategy has gradually shifted towards exploration of the available collections based on themes. Now users can explore over 60 curated digital exhibitions, countless galleries and blog posts. While there is a system in place to recommend individual items given a query item, the recommendations for editorials are done at the moment only manually. For instance when a new blog is created, the author would manually provide a list of related galleries, blogs or exhibitions that have been already published.

The task requires participants to devise recommendation methods and systems, apply them in the supplied data set gathered from Europeana and provide a series of recommendations for items and editorials. The task is thus divided into two sub-tasks:

• given a list of items provide a list of recommended items;
• given an editorial (Europeana blog or gallery) provide a list of recommended editorials.

<\ul>

Data

For the task a new dataset based on Europeana items and editorials will be provided to the participants. The individual items in the dataset will include a wealth of metadata based on the Europeana Data Model (EDM) schema. Editorials will be either Galleries (containing a title, optional description and list of items which make it up), or blog posts (containing a title, text in English and a number of items). It should be noted that although all data items follow EDM the quality of the metadata is not perfect, with some data fields being potentially somewhat ambiguous, or at least used sometimes in a creative way by the original data providers (especially with some overlap sometimes in what ends up in "format", "medium", "type" and "subject".

Evaluation methodology

Performance will be evaluated on the basis of the recommendations that are provided computing Mean Average Precision at X (Map@X) compared to the ground truth. Moreover, the systems competing in this task that can provide an explanation for the results provided will be preferred.

Participant registration

Please refer to the general ImageCLEF registration instructions

Recommendation Task Schedule

• 14.11.2022: registration opens for all ImageCLEF tasks
• 17.01.2023: training/development data release starts
• 26.04.2023: test data release starts
• 10.05.2023: deadline for submitting the participants runs
• 17.05.2023: release of the processed results by the task organizers
• 05.06.2023: deadline for submission of working notes papers by the participants
• 23.06.2023: notification of acceptance of the working notes papers
• 07.07.2023: camera ready working notes papers
• 18-21.09.2023: CLEF 2023, Thessaloniki, Greece

Submission Instructions

Instructions and format of the submissions for both recommendations sub-tasks are available at the following Github link.

Results

To be added soon.

CEUR Working Notes

To be added soon.

Citations

To be added soon.

Contact

Organizers:

• Alexandru Stan, <as(at)in-two.com>, IN2 Digital Innovations, Germany
• George Ioannidis, <gi(at)in-two.com>, IN2 Digital Innovations, Germany
• Bogdan Ionescu <bogdan.ionescu(at)upb.ro>, Politehnica University of Bucharest, Romania
• Hugo Manguinhas, <hugo.manguinhas(at)europeana.eu>, Europeana Foundation, Netherlands

Acknowledgments

CUHE has indirectly received funding from the European Union's Horizon 2020 research and innovation action programme, via the AI4Media Open Call #1 issued and executed under the AI4Media project (Grant Agreement no. 951911).