10.5281/zenodo.853492
https://zenodo.org/records/853492
oai:zenodo.org:853492
Babuji, Yadu
Yadu
Babuji
University of Chicago
Brizius, Alison
Alison
Brizius
University of Chicago
Chard, Kyle
Kyle
Chard
University of Chicago
Foster, Ian
Ian
Foster
Argonne National Laboratory
Katz, Daniel S.
Daniel S.
Katz
University of Illinois Urbana-Champaign,
Wilde, Michael
Michael
Wilde
Argonne National Laboratory
Wozniak, Justin
Justin
Wozniak
Argonne National Laboratory
Introducing Parsl: A Python Parallel Scripting Library
Zenodo
2017
Parallel scripting
Parsl
Python
2017-08-30
Working paper
10.5281/zenodo.853491
Creative Commons Attribution 4.0 International
Researchers frequently rely on large-scale and domain-specific workflows to conduct their science. These workflows may integrate a variety of independent software functions and external applications. However, developing and executing such workflows can be difficult, requiring complex orchestration and management of applications and data as well as customization for specific execution environments. Parsl (Parallel Scripting Library), a Python library for programming and executing data-oriented workflows in parallel, addresses these problems. Developers simply annotate a Python script with Parsl directives; Parsl manages the execution of the script on clusters, clouds, grids, and other resources. Parsl orchestrates required data movement and manages the execution of Python functions and external applications in parallel. In this abstract we describe Parsl's architecture and highlight two domains in which it has been used.