Fraunhofer-Gesellschaft

Publica

Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Python Workflows on HPC Systems

 
: Strassel, Dominik; Reusch, Philipp; Keuper, Janis

:

Institute of Electrical and Electronics Engineers -IEEE-; IEEE Computer Society; Association for Computing Machinery -ACM-:
IEEE/ACM 9th Workshop on Python for High-Performance and Scientific Computing, PyHPC 2020. Proceedings : Held in conjunction with SC 2020, The International Conference for High Performance Computing, Networking, Storage and Analysis, Virtual Conference, November 9-19, 2020
Piscataway, NJ: IEEE, 2020
ISBN: 978-0-7381-1087-5
ISBN: 978-0-7381-1086-8
S.32-40
Workshop on Python for High-Performance and Scientific Computing (PyHPC) <9, 2020, Online>
International Conference for High Performance Computing, Networking, Storage and Analysis (SC) <2020, Online>
Englisch
Konferenzbeitrag
Fraunhofer ITWM ()

Abstract
The recent successes and wide spread application of compute intensive machine learning and data analytics methods have been boosting the usage of the Python programming language on HPC systems. While Python provides many advantages for the users, it has not been designed with a focus on multi-user environments or parallel programming - making it quite challenging to maintain stable and secure Python workflows on a HPC system. In this paper, we analyze the key problems induced by the usage of Python on HPC clusters and sketch appropriate workarounds for efficiently maintaining multi-user Python software environments, securing and restricting resources of Python jobs and containing Python processes, while focusing on Deep Learning applications running on GPU clusters.

: http://publica.fraunhofer.de/dokumente/N-625025.html