Q: Does the hardware we use have to be commercially available at the time the competition start? We can’t find a rule about that, but there was such a rule at the SC last year.
A: We actually do not have a rule for using hardware that has to be commercially available at the time the competition starts.
Q: I have also been asked by two of my students who will require a visa for travelling to ISC if the HPCAC would be able to provide an invitation letter for them.
A: They would be able to get invitation letters at the time when they are registering for the conference. There will be an option to generate an invitation letter in PDF which they would be able to bring a copy for visa. If there’s issues, please let us know and we can work with ISC Events to generate one after you are registered.
Q: Members of my team will need to apply for visas in order to travel to Germany. How can we receive the invitation letters to apply for visa?
A: In order to receive an invitation letter, you will first need to register for the conference. In the registration process to the ISC conference, there will be a checkbox on the ISC registration web page which allows you to generate an invitation letter. You may use the invitation letter to apply for a visa.
Q: How do the SCC participants access the competition area during the time for build-up?
A: For exhibition hall access on Saturday, you and the SCC teams will have to pick up generic build-up badges from Gate 1 or Gate 8 of the exhibition hall when you arrive. Also, the Full Conference Passes can be picked up at the registration counter starting 7:30 AM on Sunday (more info). With the “E” for exhibitor the passes will grant you and the teams exhibition hall access during build-up and dismantle times as well as from 8:00am on Tuesday and Wednesday (2 hours before the official opening of the exhibition).
Q: When the shipment to the SCC booths will be available for the SCC teams?
A: The booths will be ready to move-in by Saturday, 12pm. When student team arrives, they can ask Schenker to deliver shipments to the team’s SCC booth
Q: What’s your definition of “Reboot”? Are things like restarting part of the services software counts? Is restarting the kernel itself with things like “kexec”, which does not involve running the POST phase, allowed?
A: Restarting system services are allowed. Restarting kernel with kexec, which doesn’t involve POST, is allowed.
Q: How are the different HPCC Benchmarks graded? How is the optimized and naive run graded?
A: HPCC is comprised of different sections. Each sections within in HPCC would be graded. You will submit your best run for the grading. The HPCC run will need to be a complete run from start to finish. The HPL part cannot be substitute with the results from a run generated from a different executable.
Q: It is our understanding that the HPC Challenge benchmarks are designed with a CPU only cluster in mind. Can we use optimised binaries for a GPU cluster instead? For example in the HPL, DGEMM and FFT cases. If not, can we change the code in any way?
A: Yes, you may optimize the portions within HPCC, such as using CUDA functions if you are using GPU. However, you would need to provide the changes in the source or executable, as a way for us to to reproduce the result on a similar platform as a way to verify it. HPCC run will need to be a standalone and complete run with the same output as a typical HPCC run, which means the run cannot be done on separate runs from different executables, like using a separate executable to run for HPL to complement the HPCC run. Other HPCC rules, described in the “Rules” section in the HPCC web page on code modification does apply. For details, please visit the Rules section: http://icl.cs.utk.edu/hpcc/overview/index.html
Q: How are the different benchmarks of the HPCC weighted? Will the geometric mean be the relevant result or is every weight important independently?
A: There are subtests within HPCC. Some, but not all, of the results from the subtests will be used to determine the HPCC score. Each of the scores from those subtests will be weighed equally.
Q: When using specialized computation hardware like graphics cards or Xeon Phi Processors, how do we have to determine memory usage for HPCC benchmarks? For example given all calculation is done only by graphics cards, do we need to take system RAM into consideration choosing N in the HPL benchmark or is just the size of memory from the computation device (graphics card/phi processor) relevant?
A: You can select the N value that is more advantageous for your hardware. You can use memory from RAM, or other computation device.
Q: HPCG is mentioned as a separate benchmark, but it is not mentioned in the awards section. Is it considered as an application (80% for the overall award) or an extension of the HPC Challenge (10% for the overall award)?
A: HPCG would be considered as in overall score (within the 80%)
Q: I would like to ask what role the Graph500 benchmark will play in the ISC17 competition awards, as it is not mentioned explicitly anywhere in the awards section
A: We will not be having Graph500 for this year’s competition, and we have updated the benchmark page accordingly.
Q: Is the Graph500 benchmark not a part of the competition anymore?
A: You are correct for Graph500, it will not used for this year’s competition.
Q: Are we allowed to run jobs overnight using a scheduler or should all computation cease at the close of the competition day?
A: The competition stops at the end of the day every day. The results for all applications are due end of the day, so no jobs needed to run overnight.
Q: How this experience on Baidu machines is related to the competition? I mean, during the competition all challenges still need to be run on the local cluster, right? So, this Baidu option is just for “training”, correct?
A: Baidu is helping us running the competition by giving access to the Baidu Cloud environment, in case teams would like to have an alternate platform for the teams to try and learn about running the TensorFlow application. Just want to clarify because it seems this point was not clear.You are also correct that all teams will use the local cluster for the competition.
Q: Could you please tell me what the application of TensorFlow needs us to do Traffic prediction,or anything of others? And what’s the dataset of this application?
A: The details for the TensorFlow application was described in an email delivered on May 15, 2017. If there are any further question about it, please let us know by submitting an inquiry in the web form below.
Q: We could design/choose a neural network best for CAPTCHA recognition by ourselves before the competition
A: Yes, correct
Q: We will be given a new training set during the competition for training on our competition cluster
A: Yes, correct. You should expect new training set that some properties of the images will be different from the ones you get now.
Q: We provide the necessary output from the competition for evaluation
A: Yes, correct. As mentioned in the introduction; teams are to provide their code, md5 & JSON files and output of the training of their best compiled model. Judges will verify all submitted requirements and run your model against the unseen sample data
Q: Is there version requirements for the package we need to use?
A: You can use any versions of TensorFlow or Keras or other libraries.
Q: Do we have to use Keras? Or we can also just use native TensorFlow without Keras API? Can we use third party dependency? For example, OpenCV for preprocessing on images?
A: Teams need to use Keras as shown in the example.
Q: Can we use a pre-trained weights to fine tune from? Or we have to train from scratch?
A: Teams will be given an image set to train from (scratch) at the competition. Teams are required to train based on that.
Q: Can we know what is the magnitude of the training data we will be provided for training?
A: We plan to give out training sets are up to 20K images.
Q: Do the images in one training set contain the same number of characters?
A: Yes, the images in each training set contain the same number of characters.
Q: Could you provide more information regarding the content of the CAPTCHAs that we will receive at the competition? Will they be just 4 numerical characters as well or will it increase to things like 6 alphanumerical characters? This information is crucial to the implementation of the neural network
A: You should expect the contents of the CAPTCHA that will be given at the competition different from the ones you are given right now for testing. You should expect, and be ready to make changes on your implementation for the CAPTCHA that will be given during the competition.
Q: Is the CAPTCHA challenge an additional challenge for the competition besides the one regarding the Tensorflow application? Or is it merely given as an example of what we can expect as a challenge for the Tensorflow application
A: Solving for CAPTCHA is part of the Tensorflow application challenge
Q: There were no performance restrictions mentioned for the CAPTCHA challenge. Is there a time limit for training the model or some other restriction?
A: We will deliver training images to the teams on the day of the challenge, you will have the time during application run for that day during the competition to submit results
Q: I am assuming the power restrictions still apply for the CAPTCHA challenge?
Q: Can we use pre-trained layers/models for the CAPTCHA challenge, adding our own layers to them in order to train the model? For example use the VGG-16 pre-trained model available for Keras and add our own layers to train for this specific application.
A: No, pre-trained models will not be allowed, you must solve with the training data provided.
Q: Can we use the Keras API for the TensorFlow challenge as well?
A: Yes, Keras is part of the requirement to solve for the challenge.
Q: Could you please tell me when the third application will be announced?
A: The third application which is a coding challenge, based on MiniDFT, it has been already announced. Please see the “Benchmarking” section for details
Q: How complex is this year’s coding challenge? Do we need to implement a full program or just parts of it?
A: Coding Challenge is on MiniDFT. Please check the Benchmarking section for details
Q: Where can I find documentation regarding the input files? Would like to reduce runtime of the provided small.in file for automatic CI testing of the code changes, we’ll make. Current unoptimized runtime is over 5 min on a cluster node and a CI node have only limited hardware resources. Those runs are just used to automatically test the correctness of our code not for benchmarking.
A: the “small.in” test is already the minimum possible. In fact it does a single SCF iteration (see “electron_maxstep = 1”).
Q: Is the report submitted by June 16th public shared?
A: The report is not going to be public
Q: Which version is evaluation ‘B’ based on? The version by June 16th or the final one at the end of the competition?
A: The “final version”, the one you will use during the competition. It can be exactly the same of the one made public or you can do further modification / tuning during the competition itself. The judges will inspect as well the last-minute changes
Q:Could we use a faster version in the reproducibility test? If so, which version is evaluation ‘C’ based on?
A: The “pe-23.LOCAL.in” reference tests that is going to use for the reproducibility test runs for 10 minutes on 256 MPI (SandyBridge cores). We cannot make it faster than that, the test has to be reasonable. However the expectation is that you will be able to show reproducibility on that result in the minimum time possible since you had the opportunity before the competition to run it multiple times
Q: A quick follow-up question: as you have stated, teams are allowed to work on the code/continue committing into their repository during the competition (i.e. past the June 16 5pm UTC deadline). Given that, will they be allowed to update their report/presentation? In other words, if some further improvement gets done between June 16 and “the end of MiniDFT day at competition”, will teams have a chance to update their report to reflect this improvement, or June 16 is one and only shot for the report?
A: Yes, you are correct. If some further improvements get done between June 16 and “the end of MiniDFT day at competition” (it will be on Tuesday) then you and your team have a chance to update the report and also the code in the repository to reflect additional improvements. As part of the competition, there is a reproducible test to perform. You have to submit a complete run made by using the input case provided (pe-23.LOCAL.in) prior to the competition by Friday June 16th 5:00pm UTC. Then we will check that you can reproduce that wall time (more or less, we will not be terribly strict). This means you will need to use the same version of the code you submitted in the repository on Friday June 16th , prior to any further optimizations. It is very important you submit the code you used to generate that score because we want to see that code running on the cluster. Then for the unknown input case provided during the competition you can use the latest (still public) version of your code. It is also mentioned in the README.ISC17-SCC file. Please let us know if this is not clear.
Q: I would like to ask if there is any measure that prevents teams from copying each other’s code once all code goes public? For example, I could run the pe-23.LOCAL.in during the competition with my own executable (to reproduce). But then I find out some optimization done by other teams and incorporated that into my new executable which is used for the unknown input.
A: It is actually allowed for teams to look at other teams work after the code is made public (there is nothing to prevent that, and it is allowed). The idea is for the teams to take inspirations from other teams (if each team decides that other team’s implementation are better!).
During the competition we will give teams a new input and we will check for additional code changes. The judge will determine the ability of the teams to study other teams’ work and react in a very short time. This will increase competition amongst participants to try on their best to improve their implementation until the very end!
The judges would like to first see each teams’ own best implementation on Friday, and review once again when they submit the code on the day of the competition. It is the reason we will evaluate code changes and reproducibility of pe-23.LOCAL.in using the version of the code submitted on Friday, June 16th. (As mentioned, the MiniDFT code will need to be made public by Friday June 16th 5:00pm UTC.)
Also please expect to document and cite the code/ideas from other teams, and any justifications in their final submission on the day of the competition. It helps the judges to assess how much teams borrow from other sources.
Q: What test case will be used for benchmarking the FEniCS application?
A: The actual test case to benchmark will be given at the start of the competition.
Q: Which features of Fenics will we need to run the Test Case given to us at the start of the competition. e.g. do we need OpenGL and VTK to be able to plot directly from fenics or can we outsource plotting (if it is needed at all)
A: We are evaluating the FEniCS test cases and would have an answer soon and update this response. You should expect to setup/install a visualization/plotting software for viewing and qualitatively inspect the output it is what is expected. You can use the visualization supported by FEniCS, or other plotting software, such as ParaView.
Q: What are the minimal requirements and version of FEniCS used?
A: The following was the minimum requirements for running FEniCS (version 2016.2.0)
MPI (possible to use a vendor optimized version)
HDF5 (Parallel, PETSc can download and build this)
Python pip (for installing the dijitso,ufl,fiat,ffc,instant components)
HYPRE(can be downloaded by petsc with –download-hypre=1)
blas && lapack (required by PETSc, possible to use a vendor optimized version)
python only for the installation of dolfin (versions 2 or 3)
Q: Could you provide a sample input for FEniCS similar to one given at the competition?
A: We are coming up with the examples/input data to be used for the competition at this point. However, you can try the examples in the demo directory to get familiar for now
Submit your question here:
To submit questions about the ISC 2017 Student Cluster Competition, please fill out the web form. Please bookmark this page for updates.
*Note all fields required.