Benchmarking

Benchmarks

The following benchmarks are selected to be used on the first day of the competition.

HPC Challenge

HPC Challenge (HPCC) will be used to score the benchmark portion of the competition. A team may execute HPCC as many times as desired during the setup and benchmarking phase, but the HPCC run submitted for scoring will define the hardware baseline for the rest of the competition. http://icl.cs.utk.edu/hpcc/

High Performance LINPACK (HPL)

The competition will feature a “High LINPACK” award for the team submitting the highest HPL score. Additional, independent HPL runs (outside the submitted HPCC run) may be considered for the “Highest LINPACK” award if they are performed with exactly the same hardware powered on as used for HPCC run submitted for scoring. While eligible for the Highest LINPACK award, independent HPL runs will NOT count toward the team’s overall score. The HPL run must be submitted on the first day of the competition.

HPCG

HPCG stands for High Performance Conjugate Gradient. It is a self-contained benchmark that generates and solves a synthetic 3D sparse linear system using a local symmetric Gauss-Seidel preconditioned conjugate gradient method. HPCG is a software package that performs a fixed number of symmetric Gauss-Seidel preconditioned conjugate gradient iterations using double precision (64 bit) floating point values. Integer arrays have global and local scope (global indices are unique across the entire distributed memory system, local indices are unique within a memory image). Reference implementation is written in C++ with MPI and OpenMP support. Please visit: http://www.hpcg-benchmark.org/

Applications

The following applications are selected to be used on the second day and third day of the competition.

FEniCS

FEniCS is a popular computing platform for partial differential equations (PDE). FEniCS enables users to quickly translate scientific models into efficient finite element code. With the high-level Python and C++ interfaces to FEniCS, it is easy to get started, but FEniCS offers also powerful capabilities for more experienced programmers. FEniCS runs on a multitude of platforms ranging from laptops to high-performance clusters. For more info, please visit: https://fenicsproject.org

Application used for the Coding Challenge

A software application (to be announced on a later date in March).

TensorFlow

TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.

TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google’s Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well. For more info, please visit: https://www.tensorflow.org/

Secret Application(s)

Secret application(s) will be announced on the day of the competition.

Performance Efficiency

Please note that the performance efficiency might be considered for one of the section in the SCC. Please be familiar with the calculation of the Performance Efficiency.

To achieve the maximum performance efficiency, we can calculate using the formula below:

Max [ efficiency performance ] = max [ performance x performance/Theoretical Peak (in FLOPs)]

Max [ efficiency performance ] = max [ performance * efficiency ]

Awards – Five awards will be given

Highest LINPACK

The highest score received for the LINPACK benchmark under the power budget. Results of LINPACK must be turned in at the end of the first day.

Fan Favorite

To be given to the team which receives the most unique votes from ISC participants during the SCC.

1st, 2nd and 3rd Place Overall Winners

There will be 3 overall winner awards given to the teams that are determined by the scoring of the below. The scoring for the overall winners will be calculated using the scores from HPCC, the chosen applications, and the interview by the SCC board.

The breakdown of the scores:

  • 10% for HPCC performance. (The submission for HPCC will need to be a single run with all the tests in HPCC)
  • 80% for applications runs. (The score for each successfully ran input data will be determined by the relative score to the highest scorer).
  • 10% for interview by the representatives of the SCC board.