Leaderboard

The leaderboard displays the top teams/participants ranked according to their best valid score on the rehearsal competition conducted every week. Additionally, the scores per domain are given as bar plots; and the Top 3 Plot shows the distribution of points obtained by the top 3 entries across the domains. A more detailed explanation about how the ranks are calculated, is given below.

It is important to note that we do not verify these results. Therefore, the leaderboard suggests the performance of different algorithms but is not a final arbiter for winning the competition prize. The final competition will also consider hidden instances and hidden domains. Furthermore, all the competing submissions will be run on a standard machine therefore creating a "level playing field".

To submit an entry to the Leaderboard, please follow the relevant Leaderboard Rules and Leaderboard Submission details.


Algorithm
Author/Team
Score
Date
Affiliation
1
PHunter4 Fan Xue 204.28 07/06/11
Hong Kong Polytechnic U., Hong Kong
2
ISEA2 Jiri Kubalik 177.28 25/05/11 Czech Technical University, Czech Rep.
3
HAHA1 Andreas Lehrbaum 166.78 28/04/11 Vienna University of Technology, Austria
4
TW4 Hsiao Ping-Che 130.20 07/06/11
National Taiwan University, Taiwan
5
basic-test Franco Mascia 125.67 28/05/11 Université Libre de Bruxelles, Belgium
6
ADHS1 Mustafa Misir 120.08 06/05/11 KaHo Sint-Lieven, Belgium
7
SALS He Jiang 100.33 07/06/11
Dalian University of Technology, China
8
ML Mathieu Larose 88.67 07/06/11
Université de Montréal, Canada
9
HAEA Jonatan Gomez 78.07 07/06/11
Univ. Nacional de Colombia, Colombia
10 SEA2 David Meignan 69.00 19/04/11 Polytechnique Montréal, Canada
11
AVEG_Nep Tommaso Urli 63.17 07/06/11
University of Udine, Italy
12
MCHH Kent McClymont
58.50 07/06/11
University of Exeter, UK
13
XCJ
Kamran Shafi 46.28
07/06/11
University of New South Wales, Australia
14
FPA1 CS-PUT 43.15 04/05/11 Poznan University of Technology, Poland
15
SelfSearch Jawad Elomari 36.00
07/06/11 Warwick University, UK
16
GISS0 Karim A. Touma 34.53 07/04/11 University of Santiago de Chile, Chile
17
Ant-Q Imen Khamassi
18.00 07/06/11
University of Tunisia, Tunisia
Notes:
  • Leaderboard submissions are now closed (08 June 2011).
  • The last leaderboard update (09 June 2011), contains a single entry per participant/team (the most recent submission).
  • As a reference, the current top 3 leaderboard entries:  PHunter4, ISEA2 and  HAHA1, score above 340.00 points when competing (separately) against the 8 ASAP default hyper-heuristics using the ScoreCalculator.java or MultipleScoreCalculator.java programs.
  • The ASAP default hyper-heuristics are no longer included in the leaderboard since 19/05/2011.
  • The maximum possible cross-domain leaderboard score for an algorithm is 400.00 (10 points for each of the 40 instances), which indicates that there is still room for improvement. 


Scores per Domain

In order to give more information on the algorithms' performance, the plots below illustrate the the scores for the 17 leaderboard entries on each domain. The maximum possible score per domain is 100.00 (10 points for each of the 10 training instances).

Max-SATmax-sat scores

Bin Packingbin-packing scores

Personnel Scheduling

personnel sched

Flow Shop

flowshop
[Top]

The Top 3 Plot

This plot shows the total score, and the points obtained per domain, for the 3 top Leadearboard entries.

top 3
[Top]

How the Ranks are Calculated?

The leaderboard ranks are based on a rehearsal competition conducted every week among all the submitted entries.

The rehearsal competition considers the complete set of results submitted. That is, the 40 numbers corresponding to the best objective values obtained on the 40 competition training instances (10 per domain). These values are then used for calculating the scores using the Formula 1 point system (see the source code in MultipleScoreCalculator.java). There are 10 instances (events) per domain. The maximum score is, therefore, 100 per domain, and 400 for the total leaderboard score.
[Top]

Last Updated: 09 June 2011, by Gabriela Ochoa.