pyccel/pyccel-benchmarks

Up-to-date performance benchmark results for Pyccel

Pyccel : write Python code, get Fortran speed

Pyccel stands for Python extension language using accelerators.

The aim of Pyccel is to provide a simple way to generate automatically, parallel low level code. The main uses would be:

1. Convert a Python code (or project) into a Fortran or C code.
2. Accelerate Python functions by converting them to Fortran or C functions.

Pyccel can be viewed as:

• Python-to-Fortran/C converter
• a compiler for a Domain Specific Language with Python syntax

Pyccel comes with a selection of extensions allowing you to convert calls to some specific Python packages to Fortran/C. The following packages will be (partially) covered:

• numpy
• scipy

Pyccel's acceleration capabilities lead to much faster code. Comparisons of Python vs Pyccel or other tools can be found in the benchmarks repository. The results for the devel branch currently show the following performance on Python 3.10:

If you are eager to try Pyccel out, we recommend reading our quick-start guide.

Citing Pyccel

If Pyccel has been significant in your research, and you would like to acknowledge the project in your academic publication, we would ask that you cite the following paper:

Bourne, Güçlü, Hadjout and Ratnani (2023). Pyccel: a Python-to-X transpiler for scientific high-performance computing. Journal of Open Source Software, 8(83), 4991, https://doi.org/10.21105/joss.04991

The associated bibtex can be found here.

Installation

Pyccel has a few system requirements to ensure that the system where it is installed is capable of compiling Fortran code. These requirements are detailed in the documentation. Once all requirements are satisfied, the simplest way to install Pyccel is using PyPI. Simply run:

python3 -m pip install --user pyccel

Alternative installation methods such as installing from source, or installing with a docker are described in the documentation.

Contributing

We welcome any and all contributions.

There are many ways to help with the Pyccel project which are more or less involved. A summary can be found in the documentation.

We can also be contacted via the Pyccel Discord Server.

User Documentation

• Quick-start Guide

• Installation

• Contributing

• C/Fortran Compilers

• Decorators

• Header Files

• Templates

• N-dimensional Arrays

• Function-pointers as arguments

• Const keyword

• Supported libraries/APIs

  • OpenMP
  • NumPy

Developer Documentation

• Overview
• How to solve an issue
• Review Process
• Development Conventions
• Tips and Tricks
• Scope
• Type Inference
• Array Ordering

pyccel-benchmarks's Issues

DaCe compiler

DaCe is a fast parallel programming framework that takes code in Python/NumPy and other programming languages, and maps it to high-performance CPU, GPU, and FPGA programs. It would be nice to add it to our benchmarks.

[JOSS] Reproducibility - benchmarks do not conform to community standards for dependency management

Regarding openjournals/joss-reviews#4991 - This repository is mentioned in the paper being reviewed under "Benchmarks." Upon cloning this repository and following the README.md to run the run_benchmark.py script, I receive errors about missing dependencies.

$ python run_benchmark.py --verbose --no_pythran --no_pyccel_f --no_pyccel_c
===========================================
    Ackermann
===========================================
-------------------
    python
-------------------
['python3', '-m', 'timeit', '-s', 'from ackermann_mod import ackermann;import sys; sys.setrecursionlimit(3000);', 'ackermann(3,8)']
1 loop, best of 5: 322 msec per loop

-------------------
    numba
-------------------
['python3', '-c', 'from numba_ackermann_mod import ackermann;import sys; sys.setrecursionlimit(3000);import time; t0 = time.process_time(); ackermann(3,8); t1 = time.process_time(); ackermann(3,8); t2 = time.process_time(); print(2*t1-t0-t2)']
Execution Error!
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/aaron/pyccel-benchmarks/tmp/ackermann/numba_ackermann_mod.py", line 8, in <module>
    from numba import njit
ModuleNotFoundError: No module named 'numba'

['python3', '-m', 'timeit', '-s', 'from numba_ackermann_mod import ackermann;import sys; sys.setrecursionlimit(3000);', 'ackermann(3,8)']
Execution Error!
Traceback (most recent call last):
  File "/usr/lib/python3.9/timeit.py", line 325, in main
    number, _ = t.autorange(callback)
  File "/usr/lib/python3.9/timeit.py", line 223, in autorange
    time_taken = self.timeit(number)
  File "/usr/lib/python3.9/timeit.py", line 177, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 3, in inner
    from numba_ackermann_mod import ackermann;import sys; sys.setrecursionlimit(3000);
  File "/home/aaron/pyccel-benchmarks/tmp/ackermann/./numba_ackermann_mod.py", line 8, in <module>
    from numba import njit
ModuleNotFoundError: No module named 'numba'

Ackermann                 | -                         | -
Ackermann (ms)            | 322.00                    | -                         | -
===========================================
    Bellman Ford
===========================================
-------------------
    python
-------------------
['python3', '-m', 'timeit', '-s', 'from bellman_ford_mod import bellman_ford_test;', 'bellman_ford_test()']
Execution Error!
Traceback (most recent call last):
  File "/usr/lib/python3.9/timeit.py", line 325, in main
    number, _ = t.autorange(callback)
  File "/usr/lib/python3.9/timeit.py", line 223, in autorange
    time_taken = self.timeit(number)
  File "/usr/lib/python3.9/timeit.py", line 177, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 6, in inner
    bellman_ford_test()
  File "/home/aaron/pyccel-benchmarks/tmp/bellman_ford_test/./bellman_ford_mod.py", line 54, in bellman_ford_test
    from numpy import array
ModuleNotFoundError: No module named 'numpy'

-------------------
    numba
-------------------
['python3', '-c', 'from numba_bellman_ford_mod import bellman_ford_test;import time; t0 = time.process_time(); bellman_ford_test(); t1 = time.process_time(); bellman_ford_test(); t2 = time.process_time(); print(2*t1-t0-t2)']
Execution Error!
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/aaron/pyccel-benchmarks/tmp/bellman_ford_test/numba_bellman_ford_mod.py", line 8, in <module>
    from numpy import array
ModuleNotFoundError: No module named 'numpy'

['python3', '-m', 'timeit', '-s', 'from numba_bellman_ford_mod import bellman_ford_test;', 'bellman_ford_test()']
Execution Error!
Traceback (most recent call last):
  File "/usr/lib/python3.9/timeit.py", line 325, in main
    number, _ = t.autorange(callback)
  File "/usr/lib/python3.9/timeit.py", line 223, in autorange
    time_taken = self.timeit(number)
  File "/usr/lib/python3.9/timeit.py", line 177, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 3, in inner
    from numba_bellman_ford_mod import bellman_ford_test;
  File "/home/aaron/pyccel-benchmarks/tmp/bellman_ford_test/./numba_bellman_ford_mod.py", line 8, in <module>
    from numpy import array
ModuleNotFoundError: No module named 'numpy'

Bellman Ford              | -                         | -
Traceback (most recent call last):
  File "/home/aaron/pyccel-benchmarks/run_benchmark.py", line 339, in <module>
    unit_index = round(sum(used_units)/len(used_units))
ZeroDivisionError: division by zero

Note: I explicitly disabled the Pythran and Pyccel benchmarks, or I get the error described in #2.

This repository should include a setup.cfg and setup.py file to conform to the setuptools standards for package distribution. While this repository is not a distributed package in itself, it is currently not possible to reproduce the results described in the paper, and the instructions in README.md are insufficient to run the benchmarks. Conforming to the setuptools standard will allow others to reproduce the benchmark with a single command to install dependencies.

Environment information:

$ python --version
Python 3.9.9

$ lsb_release -a
No LSB modules are available.
Distributor ID: Kali
Description:    Kali GNU/Linux Rolling
Release:        2020.4
Codename:       kali-rolling

$ uname -a
Linux AARON-XPS 4.19.104-microsoft-standard #1 SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 GNU/Linux

[JOSS] Flags used in tests are not documented

The flags used in the benchmarks are hardcoded, but they produce suboptimal results in some systems including those with avx512.

Consider adding this information to the documentation, including the how to change compiler (gcc version for example).

Development branch tests not running

The development branch tests are not currently running. This is because pip fails to install pyccel-benchmarks. After investigation it seems that the cause is this line:

pyccel-benchmarks/setup.py

Line 16 in da6860e

'python_version >= "3.7"',

The python_version key does not specify the minimum acceptable python version, rather it is used to choose different packages depending on the python version (e.g. 'threadpool >= 1.2.7; python_version < "3.2.0"', see https://stackoverflow.com/questions/21082091/install-requires-based-on-python-version). The misuse of python_version causes the failure.

[JOSS] Reproducibility - Update README.md to reflect new path of run_benchmarks.py

    Here and on line 11 of README.md, the path to `run_benchmarks.py` is not updated to reflect that `run_benchmarks.py` was moved to `benchmarks/`.

Originally posted by @aholmes in #5 (comment)

---

I can make a PR for this, but I'm not certain how or whether you want to reflect this in README.md. Thoughts?

[JOSS] Reproducibility - benchmarks do not run

Regarding openjournals/joss-reviews#4991 - This repository is mentioned in the paper being reviewed under "Benchmarks." Upon cloning this repository and following the README.md to run the run_benchmark.py script, I receive this error:

$ python run_benchmark.py --verbose
===========================================
    Ackermann
===========================================
-------------------
    python
-------------------
['python3', '-m', 'timeit', '-s', 'from ackermann_mod import ackermann;import sys; sys.setrecursionlimit(3000);', 'ackermann(3,8)']
1 loop, best of 5: 326 msec per loop

-------------------
    pythran
-------------------
['time', 'pythran', '-O3', '-march=native', '-mtune=native', '-mavx', '-ffast-math', 'ackermann_mod.py']
Traceback (most recent call last):
  File "/home/aaron/pyccel-benchmarks/run_benchmark.py", line 238, in <module>
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'time'

Environment information:

$ python --version
Python 3.9.9

$ lsb_release -a
No LSB modules are available.
Distributor ID: Kali
Description:    Kali GNU/Linux Rolling
Release:        2020.4
Codename:       kali-rolling

$ uname -a
Linux AARON-XPS 4.19.104-microsoft-standard #1 SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 GNU/Linux