You are here

Home › LifeCLEF 2024

FungiCLEF 2024

SnakeCLEF

Tentative Timeline

  • December 2023: Registration opens for all LifeCLEF challenges Registration is free of charge
  • 13 March 2024: Competition Start
  • 24 May 2024: Competition Deadline
  • 31 May 2024: Deadline for submission of working note papers by participants [CEUR-WS proceedings]
  • 21 June 2024: Notification of acceptance of working note papers [CEUR-WS proceedings]
  • 8 July 2024: Camera-ready deadline for working note papers.
  • 9-12 Sept 2024: CLEF 2024 Grenoble - France

All deadlines are at 11:59 PM CET on a corresponding day unless otherwise noted. The competition organizers reserve the right to update the contest timeline if they deem it necessary.

Motivation

Automatic recognition of fungi species assists mycologists, citizen scientists, and nature enthusiasts identify wild species. Its availability supports the collection of valuable biodiversity data. In practice, species identification typically does not depend solely on the visual observation of the specimen but also on other information available to the observer - such as habitat, substrate, location and time. Thanks to rich metadata, precise annotations, and baselines available to all competitors, the challenge provides a benchmark for image recognition with the use of additional information. Moreover, the toxicity of a mushroom can be crucial for a mushroom picker's decision. We will explore the decision process within the competition beyond the commonly assumed 0/1 cost function.

Task Description

Given the set of real fungi species observations and corresponding metadata, the goal of the task is to create a classification model that returns a ranked list of predicted species for each observation (multiple photographs of the same individual + geographical location).

The classification model must fit limits for memory footprint and a prediction time limit (120 minutes) within a given HuggingFace server instance (Nvidia T4 small 4vCPU, 15GB RAM, 16GB VRAM).

Note: Since the test set contains multiple out-of-the-scope classes. The solution has to handle such classes.

Participation requirements

Publication Track

All registered participants are encouraged to submit a working-note paper to peer-reviewed LifeCLEF proceedings (CEUR-WS) after the competition ends.
This paper must provide sufficient information to reproduce the final submitted runs.

Only participants who submitted a working-note paper will be part of the officially published ranking used for scientific communication.

The results of the campaign appear in the working notes proceedings published by CEUR Workshop Proceedings (CEUR-WS.org).
Selected contributions among the participants will be invited for publication in the Springer Lecture Notes in Computer Science (LNCS) the following year.

Context

This competition is held jointly as part of:

Data

The challenge dataset is primarily based on the Danish Fungi 2020 dataset, which contains 295,938 training images belonging to 1,604 species observed mostly in Denmark. All training samples passed an expert validation process, guaranteeing high-quality labels. Rich observation metadata about habitat, substrate, time, location, EXIF, etc., are provided.

The validation set contains 30,131 observations with 60,832 images and 2,713 species, covering the whole year and including observations collected across all substrate and habitat types.

Using additional data or metadata is permitted!

Image Data Metadata

Evaluation process

As last year, we will calculate four custom metrics and macro averaged F1 Score and Accuracy. All "unusual metrics" are explained on the competition website. The code is provided on GitHub.

The FungiCLEF 2024 recognition challenge is running with several metrics representing different decision problems, where the goal is to minimize the average empirical loss \( L \) for decisions \(q(x)\) over observations \(x\) and true labels \(y\), given a cost function \(W (y, q(x)) \).
\[ L = \dfrac{1}{n} \sum_{i=1}^n W (k_i, q(x_i)) \]

1. Standard Classification with "unknown" category. The first metric is the standard classification error, i.e. the average error of the predicted class.
All species not represented in the training set should correctly be classified as an "unknown" category.

The decision function is simple: for each observation is simply represented by an identity matrix, i.e.
\[ W_1(y,q(x))) = \left\{
\begin{matrix}
0 & \text{ if } q(x) = y \\
1 & \text{ otherwise}
\end{matrix}
\right.
\]

2. Cost for confusing edible species for poisonous and vice versa.

Let us have a function \(d\) that indicates dangerous (poisonoius) species as \(d(y)=1\) if species \( y \) is poisonois, and \(d(y)=0\) otherwise.
Let us denote \(c_\text{PSC}\) the cost for poisonous species confusion (if a poisonous observation was misclassified as edible) and \(c_\text{ESC}\) the cost for edible species confusion (if an edible observation was misclassified as poisonous).
\[ W_2(y,q(x))) = \left\{
\begin{matrix}
0 & \text{if } d(y) = d(q(y)) \hfill \\
c_\text{PSC} & \text{if } d(y) = 1 \text{ and } d(q(y)) = 0 \\
c_\text{ESC} & \text{otherwise} \hfill
\end{matrix}
\right.
\]
For the benchmark, we set \(c_\text{ESC} = 1\) and \(c_\text{PSC}=100\).

3. A user-focused loss composes of both the classification error and the poisonous/edible confusion.

\[ L_3 = \sum_i W_1 (k_i, q(x_i)) + W_2 (k_i, q(x_i)) \]

Other resources

For more FungiCLEF-related info, please refer to overview papers from previous editions. Besides, you can check out other competitions from the CLEF-LifeCLEF and CVPR-FGVC workshops.

Organizers

Credits

ZCU-FAV      CTU in Prague      University of Copenhagen      PiVa AI     Inria

Acknowledgement

TAČRMAMBO

ImageCLEF - Cross-language image retrieval evaluations