Report

To publish about the dataset, the contest or the results, please cite us:

@inproceedings{Artaud2018,
  author    = {Artaud, Chloé and
               Sidère, Nicolas and
               Doucet, Antoine and
               Ogier, Jean-Marc and
               Poulain d'Andecy, Vincent},
  title     = {Find it! Fraud Detection Contest Report},
  booktitle = {24th International Conference on Pattern Recognition, {ICPR} 2018,
               Beijing, China, August 20-24, 2018},
  year      = {2018},
}

Few statistics:

36 registrations
11 countries
Different affiliations :
- 66% Academic
- 19% Industry
- 14% Other : Police…
5 participants for 1st task (classification) and only 2 for the 2nd (localization)

Human baseline

We asked 5 humans to check each document of the test dataset of the first task. They don’t well detect the forged document!

Task 1 Results

The Verdoliva’s team and the Zampoglou’s team sent us results obtained with the first task training dataset and too with both the task 1 and task 2 training dataset. The non-balanced results of Zampoglou’s team used the same settings as the 2 tasks dataset.

	Precision	Recall	F-Measure
Fabre	0.364	0.933	0.523
Cruz	0.857	0.4	0.545
Clausner	0.882	0.5	0.638
Zampoglou T1 non-balanced	0.964	0.9	0.931
Verdoliva T1	0.906	0.967	0.935
Zampoglou T1 balanced	1.0	0.9	0.947

Verdoliva T1+T2	0.935	0.967	0.951
Zampoglou T1+T2	1.0	1.0	1.0

Human5	0.45	0.33	0.38
Human4	0.55	0.37	0.44
Human3	0.69	0.37	0.48
Human2	0.64	0.47	0.54
Human1	0.75	0.5	0.6

Task 2 results

In order to evaluate the second task we computed:

the average of the Jaccard Indexes of all documents
the average of the non-zero Jaccard Indexes (where one fraud was at least a little well located)

	Average	Standard deviation
Clausner	0,091	0,222
Verdoliva	0,426	0,261
Clausner without 0	0,287	0,315
Verdoliva without 0	0,461	0,24

ICPR contest session presentation

You can see more details on the participants approaches and the results in the presentation of the Contest Report here: ICPRFindit.