The leaderboard displays the top teams/participants ranked according to their best valid score on the rehearsal competition conducted every week. Additionally, the scores per domain are given as bar plots; and the Top 3 Plot shows the distribution of points obtained by the top 3 entries across the domains. A more detailed explanation about how the ranks are calculated, is given below.

It is important to note that we do not verify these results. Therefore, the leaderboard suggests the performance of different algorithms but is not a final arbiter for winning the competition prize. The final competition will also consider hidden instances and hidden domains. Furthermore, all the competing submissions will be run on a standard machine therefore creating a "level playing field".

To submit an entry to the Leaderboard, please follow the relevant Leaderboard Rules and Leaderboard Submission details.

PHunter4 Fan Xue 204.28 07/06/11
Hong Kong Polytechnic U., Hong Kong
ISEA2 Jiri Kubalik 177.28 25/05/11 Czech Technical University, Czech Rep.
HAHA1 Andreas Lehrbaum 166.78 28/04/11 Vienna University of Technology, Austria
TW4 Hsiao Ping-Che 130.20 07/06/11
National Taiwan University, Taiwan
basic-test Franco Mascia 125.67 28/05/11 Université Libre de Bruxelles, Belgium
ADHS1 Mustafa Misir 120.08 06/05/11 KaHo Sint-Lieven, Belgium
SALS He Jiang 100.33 07/06/11
Dalian University of Technology, China
ML Mathieu Larose 88.67 07/06/11
Université de Montréal, Canada
HAEA Jonatan Gomez 78.07 07/06/11
Univ. Nacional de Colombia, Colombia
10 SEA2 David Meignan 69.00 19/04/11 Polytechnique Montréal, Canada
AVEG_Nep Tommaso Urli 63.17 07/06/11
University of Udine, Italy
MCHH Kent McClymont
58.50 07/06/11
University of Exeter, UK
Kamran Shafi 46.28
University of New South Wales, Australia
FPA1 CS-PUT 43.15 04/05/11 Poznan University of Technology, Poland
SelfSearch Jawad Elomari 36.00
07/06/11 Warwick University, UK
GISS0 Karim A. Touma 34.53 07/04/11 University of Santiago de Chile, Chile
Ant-Q Imen Khamassi
18.00 07/06/11
University of Tunisia, Tunisia
  • Leaderboard submissions are now closed (08 June 2011).
  • The last leaderboard update (09 June 2011), contains a single entry per participant/team (the most recent submission).
  • As a reference, the current top 3 leaderboard entries:  PHunter4, ISEA2 and  HAHA1, score above 340.00 points when competing (separately) against the 8 ASAP default hyper-heuristics using the ScoreCalculator.java or MultipleScoreCalculator.java programs.
  • The ASAP default hyper-heuristics are no longer included in the leaderboard since 19/05/2011.
  • The maximum possible cross-domain leaderboard score for an algorithm is 400.00 (10 points for each of the 40 instances), which indicates that there is still room for improvement. 

Scores per Domain

In order to give more information on the algorithms' performance, the plots below illustrate the the scores for the 17 leaderboard entries on each domain. The maximum possible score per domain is 100.00 (10 points for each of the 10 training instances).

Max-SATmax-sat scores

Bin Packingbin-packing scores

Personnel Scheduling

personnel sched

Flow Shop


The Top 3 Plot

This plot shows the total score, and the points obtained per domain, for the 3 top Leadearboard entries.

top 3

How the Ranks are Calculated?

The leaderboard ranks are based on a rehearsal competition conducted every week among all the submitted entries.

The rehearsal competition considers the complete set of results submitted. That is, the 40 numbers corresponding to the best objective values obtained on the 40 competition training instances (10 per domain). These values are then used for calculating the scores using the Formula 1 point system (see the source code in MultipleScoreCalculator.java). There are 10 instances (events) per domain. The maximum score is, therefore, 100 per domain, and 400 for the total leaderboard score.

Last Updated: 09 June 2011, by Gabriela Ochoa.