ImageCLEFtoPicto

Welcome to the 4th edition of the ToPicto Task!

Motivation

Several genetic diseases, such as Rett syndrome, can result in language impairment, thereby interfering with the development of language skills such as speaking, listening, reading, and writing. Both language production and comprehension are impaired. Language impairment may also arise from incidents such as a car accident or a stroke, leading to aphasia — a partial or complete loss of the ability to express oneself or understand written and spoken language. In these particular cases, Augmentative and Alternative Communication (AAC) can be implemented. AAC involves the use of pictograms to help individuals accurately convey their messages [1].

In AAC, Pictograms refer to an image representing a more or less concrete concept. It can be a single word, a named entity, or a polylexical expression among others (see the example with pictograms taken from ARASAAC, a collection featuring over 25,000 pictograms freely available under a Creative Commons CC-BY-NC-SA license).

Using pictograms as a communication aid has proven effective in visualizing syntax, manipulating words, and facilitating language access [2,3]. Moreover, the use of AAC has a positive social impact for individuals with language impairment. The “Croix-Rouge” (French Red Cross) has identified a reduction in stress, an improvement in autonomy and health, and greater serenity and enjoyment in daily life [4]. However, not everyone has prior knowledge about AAC and pictograms. Yet, in a situation where a “verbal” person aims to communicate with an AAC user, a tool that converts the two modalities — speech and text — into a sequence of pictograms is essential. By providing a relevant and comprehensible sequence of pictograms for the impaired person, the communication between the two parties can be initiated.

The goal of ToPicto is to bring together linguists, computer scientists, and translators to develop new translation methods to translate either speech or text into a corresponding sequence of pictograms.

Preliminary Schedule

• TBD: Registration opens for all ImageCLEF tasks
• TBD: Registration closes for all ImageCLEF tasks
• TBD: Development dataset released
• TBD: Test dataset released
• TBD: Deadline for submitting participant runs
• TBD: Release of the processed results by the task organizers
• TBD: Submission of participant papers [CEUR-WS]
• TBD: Notification of acceptance
• TBD: CLEF 2027, TBD

Task Description

ImageCLEFToPicto 2027 consists of two substaks:

Subtask 1: TBD

Subtask 2: TBD

Data

Information will be added soon.

Evaluation Methodology

Information will be added soon.

Visualization of the output

The target (tgt) or the hypothesis (hyp) provided by your model for each utterance should be a sequence of pictogram terms (tokens). Each token should correspond to a pictogram from ARASAAC. To visualize the output with the pictogram images, we developed a platform available here: https://huggingface.co/spaces/ToPicto/Visualize-Pictograms

How to use it:

1. Write a sequence of pictogram terms.
2. The platform will display the corresponding pictogram images.

Example:

• id: common_voice_fr_24203862
• src: common_voice_fr_24203862.wav
• tgt/hyp: quatre concert coordonner à ville et à new_york

Participant registration

Please refer to the general ImageCLEF registration instructions

Results

CEUR Working Notes

Citations

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[1] Romski, M., & Sevcik, R. A. (2005). Augmentative communication and early intervention: Myths and realities. Infants & Young Chitdren, 18(3), 174-185.
[2] Cataix-Nègre, É. (2017). Communiquer autrement: Accompagner les personnes avec des troubles de la parole ou du langage : les communications alternatives. De Boeck Supérieur.
[3] Beukelman, D.R. and Mirenda, P. (2013). Augmentative and Alternative Communication: Supporting Children and Adults with Complex Communication Needs. Paul H. Brookes Pub.
[4] Communication alternative améliorée (CAA) : la Croix-Rouge française dévoile sa première étude d’impact social ! (2021). Croix-Rouge. Retrieved June 28, 2023, from https://www.croix-rouge.fr/actualite/communication-alternative-amelioree-caa-la-croix-rouge-francaise-devoile-sa-premiere-etude-d-impact-social-2513

Contact

• Maja Hjuler — <maja-jonck.hjuler(at)univ-grenoble-alpes.fr>, Université Grenoble Alpes, LIG, France
• Diandra Fabre — <diandra.fabre(at)univ-grenoble-alpes.fr>, Université Grenoble Alpes, LIG, France
• Benjamin Lecouteux — <benjamin.lecouteux(at)univ-grenoble-alpes.fr>, Université Grenoble Alpes, LIG, France
• Didier Schwab — <didier.schwab(at)univ-grenoble-alpes.fr>, Université Grenoble Alpes, LIG, France

Acknowledgments