The latest technological advances in computing have brought a significant change in the concept of new product design, production control, or autonomous systems. In the last few years, we have been witnessing the considerable transition to virtual prototyping and gradual pressure on integrating large part of the industrial sector in the fourth industrial revolution. The main objective of the flagship is to create a robust open source package ESPRESO applicable for a wide range of complex engineering simulations in areas such as mechanical engineering, civil engineering, biomechanics, and energy industry. The open source approach allows automatized simulation chains, based on HPC-as-a-service, such as automatized systems for shape or topological optimization which will be created on the top of the ESPRESO framework. For all the framework components, development of highly scalable methods allowing full utilization of the computational capacity of state-of-the-art supercomputers will be strictly enforced. This work enables to enlarge the IT4Innovations portfolio of services offered to industry and extend the IT4Innovations HPC demonstrators based on real applications.

Researcher:

  • Dr Tomáš Brzobohatý

Let supercomputers save you time and discover deeper possibilities for your product design

Within its research activities, IT4Innovations develops the open-source ESPRESO simulation package. It is designed to solve complex engineering problems, and during its development, emphasis is placed on optimal use of high-performance computing infrastructure (HPC). The package is distributed based on a free of charge open-source licence, which makes the application of automated simulation chains, such as computing templates customized to suit the specific product portfolio, significantly easier, and is useful for easy performance of multiple calculations without any licence fees or liabilities. Since the very beginning, each module within the software package is developed as a parallel code based on massively scalable numerical methods and algorithms. Thereby we are able to ensure optimal utilization of state-of-the-art supercomputers. The added value of the package is our implementation of a FETI domain decomposition-based massively parallel linear solver, which is able to solve problems of hundreds of billions of unknowns.

The package contains several logical units, which can be combined to reflect particular requirements for a given problem solution. These units include entry data preprocessing, computational mesh processing, numerical model construction, specific physical problem solution using a massively parallel sparse linear solver in combination with available nonlinear solvers, and final preparation of output data for visualization and online monitoring of achieved results.  Another key property of the package is its simple and flexible configuration interface, by means of which all modules can be controlled using single configuration. This interface can be extended easily, and its simple form allows it to be interconnected via a web portal with the HPC-as-a-Service system, which is being developed by the IT4Innovations Advanced Data Analysis and Simulation Lab. HPC-as-a-Service is a well-known term in the field of High-Performance Computing systems, meaning users can access an HPC infrastructure with no need to buy and administer their own physical servers or data centre infrastructure. Via this service, users have an opportunity to use HPC technologies without any prior investment into hardware. Moreover, this access further reduces the entry barrier for users and small and medium-sized enterprises who are interested in utilization of massively parallel computers, but often lack the necessary knowledge level. You can learn more about this service in the next issue of our Newsletter or on our website, where our HPC platform for scientific workflow flagship is to be presented.

ESPRESO – KEY PROPERTIES

  • a complex open-source finite element solver for HPC infrastructure,
  • interconnection with popular commercial as well as open-source tools,
  • scalable solvers for the most powerful supercomputers,
  • a simple interface for automated solutions

 

For more information about the ESPRESO library SEE.

ESPRESO – FAST AND SCALABLE INPUT

The I/O module of the ESPRESO package is designed so as to efficiently load database files containing unstructured computational meshes and perform high-quality domain decomposition. It is conceptualy developed as a massively parallel tool designated for the HPC environment, including high-performance storage systems.

In developing the I/O module, our motivation was to interconnect tools for creating complex engineering models such as ANSYS, HyperMesh, ANSA, ABAQUS, and open-source parallel solvers. This direct interconnection allows not only wider use of HPC technologies without any additional costs of commercial licences in the case of repetitive calculations or solving non-standard problems, but also use of robust commercial pre-processing tools for preparation of complex sets in cases where excessive demands on computational mesh quality are required.

The module can be used in two ways, either as a parallel convertor for various formats, or as a direct loader and pre-processor of massively parallel solvers of the ESPRESO package. In the latter case, the user can use a database file saved in his/her favourite format and use it for multiple calculations. Every calculation can be performed using a different number of compute nodes without further restrictions as our approach does not require running on a fixed number of MPI processors and domains as in the case of a parallel binary file.  This solution allows a different amount of computational resources to be used based on the needs or availability of computational resources.

Supported input formats:

  • ANSYS CDB
  • EnSight
  • VTK
  • XDMF
  • OpenFOAM
  • Abaqus

 

 

Parallel loading of unstructured meshes

  • direct input into the parallel solver without the need for conversion of a database file,
  • support for various formats containing unstructured meshes (support for a new format easily implementable based on user demands),
  • restart of a calculation using a different amount of allocated resources.

Domain decomposition

  • geometric decomposition based on the Hilbert space-filling curve,
  • connection to state-of-the-art graph decomposition libraries with significant improvement of its parallel scalability.

Asynchronnous output

  • overlapping of running calculations by storing solution results,
  • output to commonly used postprocessing formats such as EnSight and VTK,
  • monitoring of results on selected regions for statistic and optimization toolchain,
  • restart of solutions without any connection to a previous amount of computational resources.

ESPRESO – FEM LIBRARY

One of the main parts of our product is a complete FEM module for description of physical processes. Development of this module follows from IT4Innovations collaboration with industrial partners as well as implemented national and international projects. All parts of the FEM module contain massively scalable methods, which allow the computational capacity of the state-of-the-art supercomputers to be fully utilized. One of the complete modules of the FEM library is, for example, a module for heat transfer problems. This module is designed to solve complex engineering problems while covering all physical aspects such as convection, conduction, radiation, and advection, which are related to nonlinear and time-dependent problems. A simple configuration interface allows combining both steady state and time-dependent analyses.

The module for solving problems of structural mechanics includes nonlinear material models, contact problems, harmonic response analysis, implicit time schemes, and others. By combining mesh morphing using RBF (Radial Basis Function), internal scalable solvers, and modular configuration, an optimal design can be created in a short time.

ESPRESO – MASSIVELY PARALLEL SOLVER

The ESPRESO library contains several massively scalable solvers, which are based on domain decomposition methods developed by an in-house research group, allowing the computational capacity of the state-of-the-art supercomputers to be fully utilized thus being able to solve problems with hundreds of billions of unknows.

  • in-house massively parallel solver based on multilevel FETI domain decomposition methods allows solving problems with billions of unknowns,
  • version for massively parallel systems,
  • version offering utilization of NVIDIA GPU accelerators,
  • programming methods to solve contact problems,
  • tuning of the solver setting using evolutionary algorithms.

 

Successfully tested on large petascale systems:

  • Titan – Oak Ridge National Laboratory, USA
  • Piz Daint – CSCS, Switzerland
  • Salomon – IT4Innovations, Czech Republic

Available third-party solvers

In the case of highly heterogenous systems, parallel direct solvers or algebraic multigrid can be used to solve defined problems.

Principal authors of the ESPRESO library:

Lubomír Říha, Ondřej Meca, Tomáš Brzobohatý, Oldřich Vlach, and Alexandros Markopoulos.

For more information about the ESPRESO library SEE.