You are here

ImageCLEF 2011: Wikipedia image retrieval results

Introduction

The results for the Wikipedia image retrieval task at ImageCLEF 2011 are computed with the trec_eval tool (version 9.0). Please use this tool in combination with the sources provided below.

The relevance assessments (qrels) for the Wikipedia image retrieval 2011 topics can be found here.
Results
The evaluation results for all submitted runs in the wikipedia image retrieval task are listed below ranked by MAP.

  group runid Modality FB/QE Annotation language Topic language MAP P10 P20 Rprec bpref
1  XRCE XRCE_CEA_MULTI_RUN_SFL_AX_viscon Mixed FBQE EN+FR+DE  EN+FR+DE  0.3880 0.6320 0.5100 0.4162 0.3847
2  XRCE XRCE_CEA_MULTI_RUN_AX_ENFRDE_FV_SFL Mixed   EN+FR+DE    0.3869 0.6240 0.5030 0.4174 0.3834
3  XRCE XRCE_CEA_MULTI_RUN_SPLAX_ENFRDE_FV_SFL Mixed FBQE EN+FR+DE  EN+FR+DE  0.3848 0.6200 0.4990 0.4174 0.3829
4  XRCE XRCE_CEA_MULTI_RUN_XTI_nn10_AX_ENFRDE Mixed FBQE EN+FR+DE  EN+FR+DE  0.3560 0.5340 0.4710 0.3835 0.3536
5  XRCE XRCE_MULTI_RUN_AX_FV_SFL Mixed QE EN+FR+DE  EN+FR+DE  0.3557 0.5940 0.4870 0.4051 0.3633
6  XRCE XRCE_MULTI_RUN_SPLAX_FV_SFL Mixed FBQE EN+FR+DE  EN+FR+DE  0.3556 0.5780 0.4840 0.4006 0.3624
7  XRCE XRCE_CEA_MULTI_RUN_SFLAX_VISCON Mixed FBQE EN+FR+DE  EN+FR+DE  0.3471 0.5740 0.4450 0.3756 0.3398
8  UNED UNED-UV18_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.3405 0.5420 0.4500 0.3752 0.3378
9  UNED UNED-UV20_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.3367 0.5460 0.4410 0.3673 0.3314
10  UNED UNED-UV19_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.3233 0.5400 0.4230 0.3586 0.3217
11  XRCE XRCE_CEA_TXT_RUN_SPLAX_ENFRDE Textual FBQE EN+FR+DE  EN+FR+DE  0.3141 0.5160 0.4270 0.3504 0.3107
12  XRCE XRCE_CEA_TXT_RUN_AX_ENFRDE Textual QE EN+FR+DE  EN+FR+DE  0.3130 0.5300 0.4250 0.3560 0.3119
13  CEA LIST CEA_XRCE_RUN_ENFRDE_FV_SFL Mixed QE EN+FR+DE  EN+FR+DE  0.3075 0.5420 0.4210 0.3486 0.3149
14  UNED UNED-UV8_EN+FR+DE_YES_EN+FR+DE_AUTO_NOFB_TXT_TITLE Textual NOFB EN+FR+DE  EN+FR+DE  0.3044 0.5060 0.4040 0.3435 0.3012
15  UNED UNED-UV14_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.3006 0.5200 0.4030 0.3379 0.2983
16  UNED UNED-UV17_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.3006 0.4960 0.3960 0.3376 0.2996
17  UNED UNED-UV16_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.2980 0.5000 0.4030 0.3338 0.2954
18  DUTH DUTH_QDSum_w-60-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2886 0.4860 0.3870 0.3401 0.2905
19  DUTH DUTH_QDDuth_w-50-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2871 0.4620 0.3870 0.3309 0.2885
20  UNED UNED-UV15_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.2869 0.5040 0.4060 0.3306 0.2909
21  UNED UNED-UV13_EN+FR+DE_YES_EN+FR+DE_AUTO_NOFB_TXTIMG_TITLE Mixed   EN+FR+DE    0.2869 0.5040 0.4060 0.3306 0.2909
22  DUTH DUTH_QDSum_w-40-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2866 0.5120 0.4190 0.3525 0.3014
23 UNTESU UNTESU_BLRF Textual FB EN  EN  0.2866 0.4220 0.3650 0.3276 0.2821
24  DUTH DUTH_QDDuthFusion_w-50-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2862 0.4700 0.3830 0.3335 0.2844
25  DUTH DUTH_DuthFusion_w-70-PerIndexLang-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2844 0.4480 0.3870 0.3205 0.2784
26  DUTH DUTH_DuthFusion_w-70-PerIndexLang-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2840 0.4580 0.3990 0.3168 0.2775
27  CEA LIST CEA_XRCE_RUN_EN_FV_SFL Mixed QE EN EN  0.2839 0.5060 0.4060 0.3241 0.2907
28 UNTESU UNTESU_SSA150Wrf Textual FBQE EN  EN  0.2820 0.4200 0.3610 0.3190 0.2679
29  DUTH DUTH_ComSum_w-70-PerIndexLang-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2818 0.4840 0.3990 0.3495 0.2945
30  DUTH DUTH_QDSum_w-70ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2810 0.4560 0.3730 0.3373 0.2821
31  DUTH DUTH_DuthFusion_w-60-PerIndexLang-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2804 0.4740 0.4040 0.3359 0.2853
32 UNTESU UNTESU_SSA50Wrf Textual FBQE EN  EN  0.2794 0.4240 0.3630 0.3107 0.2647
33  DUTH DUTH_DuthFusion_w-60-PerIndexLang-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2786 0.4640 0.4110 0.3254 0.2815
34  XRCE XRCE_TXT_RUN_AX Textual QE EN+FR+DE  EN+FR+DE  0.2780 0.4700 0.3860 0.3250 0.2873
35  XRCE XRCE_TXT_RUN_SPLAX Textual FBQE EN+FR+DE  EN+FR+DE  0.2769 0.4640 0.3820 0.3235 0.2833
36  UNED UNED-UV9_EN+FR+DE_YES_EN+FR+DE_AUTO_NOFB_TXT_TITLE Textual NOFB EN+FR+DE  EN+FR+DE  0.2758 0.4520 0.3550 0.3154 0.2771
37  DUTH DUTH_DuthFusion_w-80-PerIndexLang-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2751 0.4360 0.3730 0.3155 0.2677
38  DUTH DUTH_ComSum_w-80-PerIndexLang-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2730 0.4660 0.3800 0.3365 0.2797
39  DUTH DUTH_ComSum_w-80-PerModality-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2717 0.4380 0.3740 0.3271 0.2728
40  CEA LIST CEA_viscon_1.07 Mixed QE EN+FR+DE  EN+FR+DE  0.2703 0.4800 0.3690 0.3061 0.2678
41  DUTH DUTH_ComSum_w-80-PerModality-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2673 0.4440 0.3670 0.3275 0.2716
42  DUTH DUTH_DuthFusion_w-70-PerModality-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2644 0.4220 0.3590 0.3135 0.2604
43  DUTH DUTH_QDSum_w-40ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2613 0.4940 0.4000 0.3321 0.2828
44  DUTH DUTH_QDDuth_w-30-None Mixed NOFB EN+FR+DE  EN+FR+DE  0.2605 0.4840 0.4090 0.3188 0.2768
45  UNED UNED-UV5_EN_YES_EN_AUTO_NOFB_TXT_TITLE Textual NOFB EN  EN  0.2601 0.4560 0.3670 0.3014 0.2600
46  DUTH DUTH_ComSum_w-70-PerIndexLang-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2592 0.4720 0.3910 0.3349 0.2821
47  CEA LIST enfrde_all Textual QE EN+FR+DE  EN+FR+DE  0.2591 0.4660 0.3630 0.2942 0.2543
48 UNTESU UNTESU_SSA150W Textual QE EN  EN  0.2577 0.4060 0.3510 0.2835 0.2401
49  DUTH DUTH_QDDuthFusion_w-30-ProfilePerQuery Mixed NOFB EN+FR+DE  EN+FR+DE  0.2559 0.4780 0.4000 0.3173 0.2732
50  UNED UNED-UV7_EN_YES_EN_AUTO_NOFB_TXT_TITLE Textual   EN    0.2515 0.4460 0.3660 0.2997 0.2541
51  UNED UNED-UV4_EN+FR+DE_YES_EN+FR+DE_AUTO_NOFB_TXT_TITLE Textual NOFB EN+FR+DE  EN+FR+DE  0.2489 0.3800 0.3290 0.2913 0.2450
52  CEA LIST en_cosine_trec_query_all_wikis Textual QE EN  EN  0.2476 0.4560 0.3500 0.2910 0.2463
53  DEMIR mixed_2cedd_fcth_run Mixed NOFB EN+FR+DE  EN+FR+DE  0.2432 0.4520 0.3420 0.3001 0.2564
54  DEMIR mixed_3_2cedd_run Mixed NOFB EN+FR+DE  EN+FR+DE  0.2426 0.4640 0.3460 0.2944 0.2573
55  UNED UNED-UV10_EN_YES_EN_AUTO_NOFB_TXT_TITLE Textual NOFB EN  EN  0.2403 0.4520 0.3510 0.2908 0.2458
56  DEMIR mixed_fcth_run Mixed NOFB EN+FR+DE  EN+FR+DE  0.2401 0.4340 0.3370 0.3018 0.2554
57  DEMIR mixed_cedd_run Mixed NOFB EN+FR+DE  EN+FR+DE  0.2394 0.4500 0.3390 0.2976 0.2529
58  DEMIR IFB2c1.0 Textual NOFB EN+FR+DE  EN+FR+DE  0.2369 0.4180 0.3320 0.2929 0.2476
59  CEA LIST CEA_XRCE_RUN_FR_FV_SFL Mixed QE FR  FR  0.2355 0.4520 0.3530 0.2850 0.2525
60 UNTESU 2011_SSA50_FB Textual FBQE EN  EN  0.2327 0.3160 0.2860 0.2543 0.2113
61  ReDCAD redcad02tx Textual NOFB EN+FR+DE  EN+FR+DE  0.2306 0.3700 0.3060 0.2862 0.2326
62 UNTESU UNTESU_SSA150rf Textual FBQE EN  EN  0.2292 0.3120 0.2810 0.2476 0.2050
63  UNED UNED-UV3_EN_YES_EN_AUTO_NOFB_TXT_TITLE Textual NOFB EN  EN  0.2243 0.3980 0.3270 0.2702 0.2287
64  DBISForMaT DBISTXT_DEENFR_COMBINED_SW Mixed NOFB EN  EN  0.2195 0.4180 0.3630 0.2827 0.2366
65 SZTAKI bpacad_txtjen_colimg Mixed FBQE EN+FR+DE  EN+FR+DE  0.2167 0.4700 0.3690 0.2762 0.2334
66 UNTESU 2011_SSA50 Textual QE EN  EN  0.2143 0.3260 0.2900 0.2438 0.2027
67  CEA LIST CEA_XRCE_RUN_DE_FV_SFL Mixed QE DE  DE  0.2138 0.4300 0.3190 0.2620 0.2326
68 SZTAKI bp_text_3 Textual NOFB EN+FR+DE  EN+FR+DE  0.2136 0.4380 0.3480 0.2683 0.2274
69 SZTAKI bp_text_13 Textual   EN+FR+DE    0.2105 0.4440 0.3430 0.2772 0.2305
70 SZTAKI bp_text_9 Textual   EN+FR+DE  EN+FR+DE  0.2103 0.4440 0.3420 0.2773 0.2303
71  DBISForMaT DBIS2011.Q10mod.IR.AVG Mixed NOFB EN  EN  0.2096 0.4080 0.3300 0.2749 0.2337
72  DBISForMaT DBISQ10mod.all.IR.AVG_MBProVariant Mixed NOFB EN  EN  0.2096 0.4080 0.3300 0.2749 0.2337
73  DBISForMaT DBISQ10mod.all.IR.AVG.SW Mixed NOFB EN  EN  0.2096 0.4080 0.3300 0.2749 0.2337
74  SINAI SINAI-90_cdlemur_10_cdlucene_6 Textual FB EN  EN  0.2068 0.4020 0.3380 0.2587 0.2189
75  UNED UNED-UV2_EN_YES_EN_AUTO_NOFB_TXT_TITLE Textual NOFB EN  EN  0.2056 0.3700 0.2900 0.2518 0.2097
76  SINAI SINAI-caption_description_lemur_okapi_fb10_5 Textual FB EN  EN  0.2052 0.4020 0.3410 0.2588 0.2184
77  DBISForMaT DBISTXT_EN_OW_NOSW Textual NOFB EN  EN  0.2043 0.3960 0.3120 0.2610 0.2173
78  DBISForMaT DBISTXT_EN_OW_SW Textual NOFB EN  EN  0.2043 0.3960 0.3120 0.2610 0.2173
79  DBISForMaT DBISQ10mod.all.IR.AVG.combine.SW Mixed NOFB EN  EN  0.1961 0.3820 0.3060 0.2573 0.2198
80 SZTAKI bp_text_10 Textual   EN  EN  0.1957 0.4240 0.3420 0.2692 0.2223
81  DBISForMaT DBISTXT_DEENFR_SW Textual   EN+FR+DE  EN+FR+DE  0.1956 0.3700 0.2970 0.2451 0.2091
82 SZTAKI bp_text_2 Textual NOFB EN+FR+DE  EN+FR+DE  0.1951 0.4120 0.3250 0.2534 0.2118
83  DBISForMaT DBISTXT_EN_SW Textual NOFB EN  EN  0.1945 0.3740 0.2950 0.2470 0.2041
84  DBISForMaT DBISTXT_DEENFR_NOSW Textual   EN+FR+DE  EN+FR+DE  0.1944 0.3680 0.2940 0.2459 0.2071
85  DBISForMaT DBISTXT_EN_NOSW Textual   EN    0.1925 0.3740 0.2930 0.2462 0.2016
86 SZTAKI bp_text_12 Textual   EN+FR+DE  EN+FR+DE  0.1904 0.4120 0.3330 0.2595 0.2137
87 SZTAKI bp_text_8 Textual   EN+FR+DE  EN+FR+DE  0.1901 0.4100 0.3310 0.2578 0.2135
88  ReDCAD redcad02md Textual NOFB EN+FR+DE  EN+FR+DE  0.1896 0.3400 0.2920 0.2386 0.2016
89 SZTAKI bp_text_1 Textual NOFB EN  EN  0.1818 0.4000 0.3200 0.2507 0.2063
90  CEA LIST fr_cosine_trec_query_all_wikis Textual QE FR  FR  0.1785 0.3460 0.2680 0.2296 0.1895
91 SZTAKI bp_text_7 Textual   EN  EN  0.1778 0.3860 0.3240 0.2499 0.2034
92  SINAI SINAI-caption_lemur_okapi_fb10_2 Textual FB EN  EN  0.1732 0.3820 0.3280 0.2349 0.1937
93  UNED UNED-UV1_EN_YES_EN_AUTO_NOFB_TXT_TITLE Textual NOFB EN  EN  0.1727 0.3040 0.2380 0.2140 0.1786
94  UAIC2011 uaic3lucene&alllang&cedd Mixed NOFB EN+FR+DE  EN+FR+DE  0.1665 0.4080 0.3090 0.2313 0.1934
95  SINAI SINAI-80caption_20description_lucene_4 Textual NOFB DE EN  0.1618 0.3620 0.2920 0.2160 0.1748
96  CEA LIST de_cosine_trec_query_all_wikis Textual QE German DE  0.1586 0.2920 0.2410 0.2157 0.1760
97  ReDCAD redcad01md Textual NOFB EN  EN  0.1578 0.3140 0.2420 0.2052 0.1730
98  UNED UNED-UV6_FR_YES_FR_AUTO_NOFB_TXT_TITLE Textual NOFB FR  FR  0.1561 0.3580 0.2600 0.2111 0.1813
99  SINAI SINAI-caption_lucene_1 Textual FB EN  EN  0.1496 0.3680 0.2850 0.2046 0.1634
100  ReDCAD redcad01tx Textual NOFB EN  EN  0.1335 0.3160 0.2760 0.2181 0.1735
101  UAIC2011 uaic4lucene&svm&alllang&cedd Mixed NOFB EN+FR+DE  EN+FR+DE  0.1259 0.3340 0.2430 0.1817 0.1516
102  SINAI SINAI-caption_description_lucene_3 Textual FB EN  EN  0.1200 0.2540 0.1950 0.1637 0.1256
103  UAIC2011 uaic6lucene&alllang&qe&cedd Mixed QE EN+FR+DE  EN+FR+DE  0.1099 0.2840 0.2250 0.1552 0.1305
104  UAIC2011 uaic4lucene&svm&en&cedd Mixed NOFB EN  EN  0.0955 0.2800 0.2100 0.1587 0.1236
105  UAIC2011 uaic2lucene&cedd Mixed   EN    0.0946 0.2800 0.2110 0.1629 0.1249
106  UAIC2011 uaic5lucene&en&qe&cedd Mixed QE EN  EN 0.0705 0.2380 0.1820 0.1158 0.0936
107  UNED UNED-UV11_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.0553 0.1180 0.1030 0.0816 0.0631
108  UNED UNED-UV12_EN+FR+DE_YES_EN+FR+DE_AUTO_FB_TXTIMG_TITLE Mixed FB EN+FR+DE  EN+FR+DE  0.0516 0.0880 0.0950 0.0802 0.0579
109  DEMIR cedd_min_run Visual       0.0044 0.0340 0.0280 0.0138 0.0115
110  DBISForMaT DBIS2011.Q10mod_noIndri Visual NOFB     0.0034 0.0200 0.0210 0.0114 0.0094