The following benchmarks are selected to be used on the first day of the competition.
HPC Challenge (HPCC) will be used to score the benchmark portion of the competition. A team may execute HPCC as many times as desired during the setup and benchmarking phase, but the HPCC run submitted for scoring will define the hardware baseline for the rest of the competition. In other words, after submitting this benchmark, the same system configuration should be used for the rest of the competition.
The rules described in the Rules section of HPCC web page on code modification does apply.
High Performance LINPACK (HPL)
The teams will compete on High Performance LINPACK (HPL) benchmark for the ‘High LINPACK’ award for the team submitting the highest HPL score. Additional, independent HPL runs (outside the submitted HPCC run) may be considered for the “Highest LINPACK” award if they are performed with exactly the same hardware powered on as used for HPCC run submitted for scoring. While eligible for the Highest LINPACK award, independent HPL runs will NOT count toward the team’s overall score. The HPL run must be submitted on the first day of the competition.
The teams may use any HPL binary.
• The teams need to declare which binary they going to run (by June 10) and provide the binary info + NVIDIA contact (or anyone else) that provided them the binary.
• Due to the Open MPI issue #3003, we advise all student teams to avoid using Open MPI versions between 1.10.3 to 1.10.6 due to the timer bug. This bug can potentially cause HPL to show the calculated results better than the theoretical peak.
HPCG stands for High Performance Conjugate Gradient. It is a self-contained benchmark that generates and solves a synthetic 3D sparse linear system using a local symmetric Gauss-Seidel preconditioned conjugate gradient method. HPCG is a software package that performs a fixed number of symmetric Gauss-Seidel preconditioned conjugate gradient iterations using double precision (64 bit) floating point values. Integer arrays have global and local scope (global indices are unique across the entire distributed memory system, local indices are unique within a memory image). Reference implementation is written in C++ with MPI and OpenMP support. HPCG is being used on the first day of the competition. 30 minutes is the minimum time needed for the official run.
The teams may use any HPCG binary.
Notes: The teams need to declare which binary they going to run (by June 10) and provide the binary info + NVIDIA contact (or anyone else) that provided them the binary.
The following benchmarks are selected to be used on the second and third day of the competition.
Grid is a C++ library, Developed by Peter Boyle (U. of Edinburgh) et al, for Lattice Quantum Chromodynamics (Lattice QCD) calculations that is available on github and is designed to exploit parallelism at all levels:
• SIMD (vector instructions)
• OpenMP (shared memory parallelism)
• MPI (distributed memory parallelism through message passing)
For the Competition we will use ISC-freeze-2 tag version (on the github)
• You will need to use C_LIME library for GRID (http://usqcd-software.github.io/downloads/c-lime/lime-1.3.2.tar.gz)
• MPI_THREAD_MULTIPLE must be supported in the MPI layer.
The Time to solution results will be used as a reference from the log file, here is an example:
Grid : Message : 602642 ms : Computation time is 523.439286 seconds
Grid : Message : 602642 ms : I/O time is 69.615096 seconds
Grid : Message : 602642 ms : Time to solution is 593.054382 seconds
Grid : Message : 602642 ms : Done
Nektar++ is a spectral/hp element framework designed to support the construction of efficient high-performance scalable solvers for a wide range of partial differential equations (PDE). Although primarily driven by application-based research, it has been designed as a platform to support the development of novel numerical techniques in the area of high-order finite element methods.
The Total Computation Time results will be used as a reference from the log file, here is an example:
Total Computation Time = 2475s
L 2 error (variable u) : 0.287467
L inf error (variable u) : 1.93685
L 2 error (variable v) : 0.400866
L inf error (variable v) : 1.96964
HPC Secret Application
A Secret application will be announced on the day of the competition.
TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs with a single API.
TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google’s Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well.
For more information about TensorFlow, please visit: https://www.tensorflow.org/
The Deep Learning AI task will be an image recognition model training, using TensorFlow. The teams will be asked to demonstrate maximum accuracy within a fixed training time (to be announced at the competition), using ImageNet 2012, under the following directions:
1. Framework: TensorFlow (any general available version) or TensorFlow over RDMA. The RDMA code can be found at:
2. Model: VGG16
3. Benchmark: TensorFlow training benchmark found at https://www.tensorflow.org/performance/benchmarks, based on ImageNet 2012 dataset found at http://www.image-net.org/
The following rules will apply:
• The teams have the freedom to choose a distributed or non-distributed version of TensorFlow and in addition, to optimize the method.
• The teams should submit their code if they have made any changes.
1. The teams need to download the imagenet 2012, and have it on the servers. We will have a backup disk for teams that won’t be able to have it ready on the servers on the competition day. Please try your best to have the dataset preload to simplify logistics during the competition day, in case it is really not possible, email us to prepare.
2. The teams need to perform pre-processing on the images (change the image filenames)
3. Download tensorflow (framework, benchmarks, models)
4. Note, in case you are using python2, see a fix here:
5. You will need to use the benchmark, and print the accuracy. For example:
python tf_cnn_benchmarks.py –model=vgg16 –print_training_accuracy=true ….
The following awards will be given:
The highest score received for the LINPACK benchmark under the power budget. Results of LINPACK must be turned in at the end of the first day.
To be given to the team which receives the most unique votes from ISC participants during the SCC.
Vote here for your favorate team.
1st, 2nd and 3rd Place Overall Winners
There will be 3 overall winner awards given to the teams that are determined by the scoring of the below. The scoring for the overall winners will be calculated using the scores from HPCC, the chosen applications, and the interview by the SCC board.
The breakdown of the scores:
• 10% for HPCC performance
• 10% HPCG
• 15% Grid
• 15% Nektar++
• 15% Secret Application
• 25% TensorFlow
• 10% for interview by the representatives of the SCC board