Success Stories

Computation of the linear and nonlinear optical properties of graphenes and carbon nanotubes

succ1_1Dr. Manthos Papadopoulos’ group from the National Hellenic Research Foundation in cooperation with others scientists from France, Slovakia and Norway were awarded 200,000 CPU hours and 5,000 GPU hours upon Cy-Tera to perform a series of demanding and innovative calculations for the linear and non-linear optical properties of graphenes and carbon nanotubes so as to provide more accurate results than those currently available and at a lower computational cost. This study was important, as the systematic computational study of their molecular (hyper) polarizabilities could provide an insight on the structure-property relationship of nano-derivatives. These systems could compose the fundamental building blocks for the development of materials greatly needed by the photonic industry.

3D Simulations of Jittering-Jets in Core-Collapse Supernovae

succ2_1Professor Noam Soker and his PhD student were awarded 200,000 CPU hours on Cy-Tera to study the connection between jets and core-collapse supernovae explosions of massive stars using 3D gas dynamic numerical simulations to examine the physical properties and conditions for successful explosions.
This computational time was used to simulate the new and novel “Jets Driven Feedback Mechanism Model” which seeks to explain the reason as to why massive stars explode at the end of their life, something very important as new surveys reveal previously unknown types of supernovae which require new models to be correctly explained.




Simulations for swimming biologically inspired robots

Professor Ekatesucc3_1rinaris’ group were allocated 200,000 CPU hours and performed computational fluid dynamic techniques on time‐varying geometries, specifically for motions that reflect biological aquatic swimming. The group investigated the forces acting upon different types of arms which could be used upon an aquatic swimming robot. With nine publications prepared as a result of this allocation, their work will contribute to the development of future bio-inspired robotic devices.

Discontinuous Galerkin discretizations over two Production calls

H-adaptivity simulation for viscus flow around a sphere at Re=1000
H-adaptivity simulation for viscus flow around a sphere at Re=1000

After a successful 1st production call application, Professor Ekaterinaris’ group was also successful with their 2nd production call application. Using high order accurate discontinuous Galerkin discretization to model complex shocks dominated with hypersonic flows and the transport of the reactive species arising from the dissociation of oxygen and nitrogen, a refinement procedure was used to accurately represent the complex flow structures generated.
Such numerical simulations are important as they are expected to play a key enabling role in the design and evolution of concepts for the next-generation space vehicles. Results of this application can help achieve a better understanding of chemical dissociation and radiation processes for high-speed high enthalpy re-entry hypersonic flows where chemical reactions and non-equilibrium phenomena including flow transition and turbulence play a major role for thermal protection of space vehicles.


Pressure-velocity coupled high performance CFD modelling

After an unsuccessful 1st LinkSCEEM/Cy-Tera production call application where preparatory access was granted, Professor Gelfgat’s group from Israel were successful with their second production call application.
Modelling unavoidable instabilities in the course of transition to turbulence using lid-driven 3D cavities and convection in rectangular 3D boxes configurations, simulations were carried out to allow for experimental and computational results for the development of code based on multi-frontal direct sparse solvers. The code and results of this project can also be useful in various fields of Mechanical Engineering such as drag reduction, renewable energy problems and manufacturing of high-quality materials.7

Visualization of a three-dimensional convective flow in a laterally heated cubic cavity at Ra=103 (a) and Ra=107 (b).
Visualization of a three-dimensional convective flow in a laterally heated cubic cavity at Ra=103 (a) and Ra=107 (b).

An Environmental Science and Technology group from Cyprus progresses from preparatory access to production access

Density change profiles of the ice 1h B2 crystals upon superheating at 8.5-9.1% w/w
Density change profiles of the ice 1h B2 crystals upon superheating at 8.5-9.1% w/w

Dr. Daskalakis’ environmental science and technology group started using Cy-Tera under preparatory access, using their allocated time for code development and scalability testing.
In the second Production call for proposals, Daskalakis’ group applied and was successfully granted resources allocation under production access. The project in question investigated the direct role that pollutants have on atmospheric precipitation processes. More specifically, the project aimed to provide quantitative and qualitative information for processes involving the microphysics of pollutant enriched aerosols in the clouds, as well as their interactions with biological membranes for human and plant protection purposes.
Using Molecular Dynamics simulations to act as a simulation “microscope” the results of the project emphasize the need for the effects of certain environmental pollutants in terms of their relative concentration to be incorporated into existing weather prediction models as lack of such knowledge prevents one from evaluating and predicting a correct dynamical behaviour of clouds – something important to know given the rising levels of pollution in the atmosphere.

Synchrotron Radiation Data Analysis

Pradel A., Langer M., Maisey J., Geffard-Kuriyama D., Cloetens P., Janvier P. and Tafforeau P., 2009. Skull and brain of a 300 million-year-old chimaeroid fish revealed by synchrotron holotomography. PNAS, 106(13) :5224-5228.
Pradel A., Langer M., Maisey J., Geffard-Kuriyama D., Cloetens P., Janvier P. and Tafforeau P., 2009. Skull and brain of a 300 million-year-old chimaeroid fish revealed by synchrotron holotomography. PNAS, 106(13) :5224-5228.

Many discoveries in various fields like biology, chemistry, medicine, environmental science, geology, archaeology and physics are made by ‘seeing’ things using light. The range of the light used can be from infrared to X-rays and beyond. In recent decades, synchrotron light has become an essential tool on studying matter using radiation from infrared to hard X-rays.
The experiments on a synchrotron light source produce massive amounts of data of the order of petabytes, for example tomography beamlines. There is a need for online and offline data analysis of such large amounts of data, what requires high performance computing (HPC).
Lots of algorithms currently implemented on CPUs could profit from being ported to GPUs: the efforts made in synchrotron-related research encourage continuing the work in this promising field by porting typical X-ray data analysis programs to GPUs and exploring the possibility of enabling these programs to run efficiently on multicore CPUs.
The HPC experts at the ESRF chose CPU/GPU-based heterogeneous clusters for carrying out online and offline analysis. They selected a number of applications that need to be ported to such heterogeneous systems which are as follows:

Typical Image taken from a Powder Diffraction experiment using a 4Mpixel detector (2048 x 2048 pixels).
Typical Image taken from a Powder Diffraction experiment using a 4Mpixel detector (2048 x 2048 pixels).

– PyHST – PyHST (High Speed Tomography in python version): is the main volume reconstruction program used for a large number of tomography experiments at the ESRF. After porting the program to GPUs a speedup factor of about 40 has been achieved.
– BigDFT (Density Functional Theory) – a density functional theory program has shown a speedup factor up to seven running on GPUs. It has also been selected as one of the benchmark applications for the PRACE project
– PPM (Percolated Perpendicular Media) – a program for simulating the reflectivity of magnetic multilayer structures. It achieved a speedup ratio of more than 100 on GPUs.
– hadow3 – a ray tracing program. A part of this application, i.e. FRESNEL2D has been chosen for its promising acceleration performance. A typical computation can take days if not ported to GPUs. However, a speedup factor of approximately 37 was obtained on GPUs.
– XTLS – a program for simulating X-rays through nanofocusing optical elements called transfocators. This is similar to Shadow3 and simulates wave propagation through Fresnel lenses in 1D. A speed-up in excess of 95 was achieved on GPUs.
– PyFAI (python tool for fast azimuthal integration) – a program that performs azimuthal integration on huge stacks of powder diffraction images. The GPU version of PyFAI keeps pace with even modern high-throughput detectors.
The ESRF scientific software team produced a number of international (conference/journal) papers out of this valuable contribution to synchrotron radiation community.

A Library of Automatically Tuned Sparse Matrix Kernels for Grapsucc8_1hics Processing Units

A project that has progressed from Educational to Preparatory and then to Production access.
Professor Walid Abu-Sufah’s computational group from the University of Jordan started using LinkSCEEM resources through educational access provided to them through a cluster of PC’s which also possessed graphic cards. Educational Access was then given to them on Euclid which was a more structured hybrid CPU-GPU cluster upon which the group could test and run CUDA code.
The group later applied for a number of Preparatory Access projects where development, parallelisation and optimisation of their code was carried out. Coupled to this was the scalability testing required for the production access application completed by the group in the 3rd Production call.
The group was allocated twelve thousand GPU hours on Cy-Tera to design, implement, and test fundamental sparse matrix computations kernels for high-performance execution on graphics processing units.

High performance CFD modelling

succ9_1A project that has progressed from Preparatory Access to Production Access.
After the productive use of a preparatory access allocation, Israeli research scientists have had two successful production call applications accepted. Three hundred thousand CPU hours, spread over 2 projects, have been allocated and research will be carried out in the field of Computational Fluid Dynamics.
The success of these research scientists highlights the importance of preparatory access to computational projects. Computational time given in preparatory access provides projects with the time they require to parallelize and/or optimize their project codes. Furthermore, the time can be used to carry out the appropriate scalability tests and measurements required for production access applications were large computational resources can be awarded to projects.


Hadronic Structure

The strong interaction is one of the four fundamental forces of nature. It is key in the creation and evolution of the universe and is ever present in the formation and stability of nuclei, which form the atoms and molecules that constitute the visible matter in the universe.
Quantum Chromodynamics (QCD) is the part of the standard model of particle physics which describes the dynamics of the strong interaction. This project carried out an in depth study of the strong interaction using Lattice QCD – a computational method used to simulate QCD.
Using state of the art datasets produced by the European Twisted Mass Collaboration, Cypriot scientists used Cy-Tera to carry out a high-statistics calculation of several key baryonic observables whose calculation with increased accuracy could open the way for predictions in theories associated with new physics beyond the standard model.
During the first year of Cy-Tera, this project was allocated nearly 2 million CPU hours and 40 thousand GPU hours. The use of Cy-Tera and LinkSCEEM resources allowed for the project to produce results that resulted in four research publications and seven scientific presentations were given based on these findings.
The project was awarded a further 1.2 million CPU hours and 70 thousand GPU hours as it continued its research into a second year of computation.