pyccel/pyccel

Python extension language using accelerators

Pyccel : write Python code, get Fortran speed

Pyccel stands for Python extension language using accelerators.

The aim of Pyccel is to provide a simple way to generate automatically, parallel low level code. The main uses would be:

1. Convert a Python code (or project) into a Fortran or C code.
2. Accelerate Python functions by converting them to Fortran or C functions.

Pyccel can be viewed as:

• Python-to-Fortran/C converter
• a compiler for a Domain Specific Language with Python syntax

Pyccel comes with a selection of extensions allowing you to convert calls to some specific Python packages to Fortran/C. The following packages will be (partially) covered:

• numpy
• scipy

Pyccel's acceleration capabilities lead to much faster code. Comparisons of Python vs Pyccel or other tools can be found in the benchmarks repository. The results for the devel branch currently show the following performance on Python 3.10:

If you are eager to try Pyccel out, we recommend reading our quick-start guide.

Citing Pyccel

If Pyccel has been significant in your research, and you would like to acknowledge the project in your academic publication, we would ask that you cite the following paper:

Bourne, Güçlü, Hadjout and Ratnani (2023). Pyccel: a Python-to-X transpiler for scientific high-performance computing. Journal of Open Source Software, 8(83), 4991, https://doi.org/10.21105/joss.04991

The associated bibtex can be found here.

Installation

Pyccel has a few system requirements to ensure that the system where it is installed is capable of compiling Fortran code. These requirements are detailed in the documentation. Once all requirements are satisfied, the simplest way to install Pyccel is using PyPI. Simply run:

python3 -m pip install --user pyccel

Alternative installation methods such as installing from source, or installing with a docker are described in the documentation.

Contributing

We welcome any and all contributions.

There are many ways to help with the Pyccel project which are more or less involved. A summary can be found in the documentation.

We can also be contacted via the Pyccel Discord Server.

User Documentation

• Quick-start Guide

• Installation

• Contributing

• C/Fortran Compilers

• Decorators

• Header Files

• Templates

• N-dimensional Arrays

• Function-pointers as arguments

• Const keyword

• Supported libraries/APIs

  • OpenMP
  • NumPy

Developer Documentation

• Overview
• How to solve an issue
• Review Process
• Development Conventions
• Tips and Tricks
• Scope
• Type Inference
• Array Ordering

pyccel's Issues

Wrong translation of comm.Gather() function from mpi4py to Fortran

Pyccel generates code like

call mpi_gather(sendbuf, n0_sendbuf, MPI_INT, recvbuf, n0_recvbuf, MPI_INT, root, comm, ierr)

which is wrong, because according to the MPI 3.0 standard we have

recvcount: number of elements for any single receive (non-negative integer, significant only at root)

Therefore, given that all messages are the same size (equal to the send buffer), instead of n0_recvbuf we should pass n0_sendbuf again.

Error when using numpy's empty and a user defined function

The following is a MWE illustrating the problem. It was translated using epyccel to translate the module. The error does not seem to occur for other numpy functions such as cos

from pyccel.decorators  import external, pure

#==============================================================================
@pure
def get_0(  ):
    return 0

#==============================================================================
@external
@pure
def test():
    from numpy      import empty
    span  =  get_0(  )

    basis  = empty( 2, dtype=float )

The error returned is:

  File "test.py", line 4, in <module>
    tmp=epyccel(SEF)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/epyccel.py", line 1086, in epyccel
    return _epyccel_seq( inputs, **kwargs )
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/epyccel.py", line 1037, in _epyccel_seq
    return epyccel_module( inputs, **kwargs )
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/epyccel.py", line 989, in epyccel_module
    code = fcode(f2py_module, ast.parser)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/codegen/printing/fcode.py", line 2333, in fcode
    return FCodePrinter(parser, settings).doprint(expr, assign_to)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/codegen/printing/codeprinter.py", line 59, in doprint
    lines = self._print(expr).splitlines()
  File "/home/admlocal/.local/lib/python3.6/site-packages/sympy/printing/printer.py", line 287, in _print
    return getattr(self, printmethod)(expr, **kwargs)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/codegen/printing/fcode.py", line 244, in _print_Module
    '{sep}\n').format(body=body, sep=sep, f=self._print(i))
  File "/home/admlocal/.local/lib/python3.6/site-packages/sympy/printing/printer.py", line 287, in _print
    return getattr(self, printmethod)(expr, **kwargs)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/codegen/printing/fcode.py", line 1390, in _print_FunctionDef
    body_code = '\n'.join(self._print(i) for i in body)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/codegen/printing/fcode.py", line 1390, in <genexpr>
    body_code = '\n'.join(self._print(i) for i in body)
  File "/home/admlocal/.local/lib/python3.6/site-packages/sympy/printing/printer.py", line 287, in _print
    return getattr(self, printmethod)(expr, **kwargs)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/codegen/printing/fcode.py", line 1030, in _print_Assign
    func = self.get_function(name)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/codegen/printing/fcode.py", line 185, in get_function
    raise ValueError('function {} not found'.format(name))
ValueError: function f2py_test not found

Check the validity of variables and functions names before converting it to Fortran

In Python we can have names of variables and functions that starts with _ like _x or _func,
in that case we should raise an error or maybe change the names to a valid one.
also Python treats upper case variable names and lower case variable names as two different set of variables but Fortran sees them as one for example the variable with names x and X are two different variables in python but the same in Fortran, so we should either raise an error or change the names

Tanh from numpy not implemented

The following code fails with "NotImplementedError: TODO"

from pyccel.decorators import external

@external
def test_tanh():
    from numpy import tanh
    return tanh(2)

from pyccel.epyccel import epyccel

test2=epyccel(test_tanh)
test2()

Large constants are split in a way that does not appear to be allowed in fortran

If a constant is too large for the line (e.g. because the user wants to ensure machine precision) then it can be split in a way that is incompatible with fortran. E.g:

pi = &
3.141592653589793238462643383279502884197169399375105820974944592

which gives the error: Error: Unclassifiable statement at (1)

Enumerate does not work using pyccel

Although enumerate seems to work using epyccel. It does not work when using pyccel.

The following is an example:

from pyccel.decorators import types

@types('double[:]')
def print_all( vals ):
    for i,v in enumerate(vals):
        print(i,v)

from numpy import array

v = array([1.0, 2.0, 3.0, 4.0])

print_all(v)

Pi is printed in single precision

Pi when imported from scipy or numpy is printed in single precision in fortran instead of double.

Dividing the result of a function call by a value changes the type of the arguments

The following example illustrates the problem:

from pyccel.decorators  import types

@types('double[:]','int')
def testing(x,d):
    return x[d]

from numpy import array
x=array([1.0,2.0,3.0,4.2,53.])
print(testing(x,2)/3)

In argument detected as Inout

When a function uses a temporary variable whose size is determined by an argument then any modification of this variable leads to the size being marked as an inout variable.

The following is a minimum working example:

from pyccel.decorators import types, external

@external
@types('int')
def test_degree(degree):
    from numpy import empty

    tmp = empty(degree)
    for i in range(5):
         tmp[i]=0.

It generates the following fortran:

subroutine test_degree(degree)

  implicit none
  integer(kind=4), intent(inout)  :: degree
  real(kind=8), allocatable :: tmp (:)
  integer(kind=4) :: i




  allocate(tmp(0:degree - 1))
  do i = 0, 4, 1
    tmp(i) = 0.0d0
  end do

end subroutine

Possibly related to issue #174

Inout intent not recognised when object is modified in a different function.

Pyccel does not check the intent of functions called by a starting function when deciding the intent of an object. E.g. in the following code:

from numpy      import zeros

#$ header function changeElem(float[:],int)
def changeElem(f,i):
    f[i]=2

#$ header function buildArray(float[:],int)
def buildArray(f,n):
    for i in range(n):
        changeElem(f,i)

f=zeros(4)
buildArray(f,4)
print(f)

Fortran raises the following error:
Error: Dummy argument ‘f’ with INTENT(IN) in variable definition context (actual argument to INTENT = OUT/INOUT) at (1)

Division by element of an array is not working

The following code:

from pyccel.decorators import types
@types('double[:]')
def basis_funs( knots ):
    a = 1.0 / knots[0]

gives the error: TypeError: Unknown datatype 1.0/knots[0]

Cannot epyccelize a module if one external function calls another

If one function calls another and both functions should be accessible from python then epyccel fails. This is due to the prototype change.

E.g:

from pyccel.decorators import external, types

@external
@types('double[:]')
def somme(x):
    y = 0.0
    for xi in x:
        y = y + xi
    return y

@external
@types('double[:]')
def square_of_sum(x):
    y = somme(x)
    return y*y

In this example the function somme becomes a subroutine in fortran to make it convertible with f2py, however square_of_sum still calls it as a function which leads to a compilation error. In addition the array size must now be passed as an additional argument.

The fortran code generated is:

module f2py_ssq

implicit none




contains

!........................................
subroutine f2py_somme(n0_x, x, y)

  implicit none
  real(kind=8), intent(out)  :: y
  integer(kind=4), intent(in)  :: n0_x
  real(kind=8), intent(in)  :: x (0:n0_x - 1)
  integer(kind=4) :: Dummy_6602
  real(kind=8) :: xi

  y = 0.0d0
  do Dummy_6602 = 0, size(x,1) - 1, 1
    xi = x(Dummy_6602)
    y = xi + y
  end do

  return
end subroutine
!........................................

!........................................
subroutine f2py_square_of_sum(n0_x, x, Dummy_0767)

  implicit none
  real(kind=8), intent(out)  :: Dummy_0767
  integer(kind=4), intent(in)  :: n0_x
  real(kind=8), intent(in)  :: x (0:n0_x - 1)
  real(kind=8) :: y

  y = somme(x)
  Dummy_0767 = y*y
  return
end subroutine
!........................................

end module

but it should be:

module f2py_ssq

implicit none




contains

!........................................
subroutine f2py_somme(n0_x, x, y)

  implicit none
  real(kind=8), intent(out)  :: y
  integer(kind=4), intent(in)  :: n0_x
  real(kind=8), intent(in)  :: x (0:n0_x - 1)
  integer(kind=4) :: Dummy_6602
  real(kind=8) :: xi

  y = 0.0d0
  do Dummy_6602 = 0, size(x,1) - 1, 1
    xi = x(Dummy_6602)
    y = xi + y
  end do

  return
end subroutine
!........................................

!........................................
subroutine f2py_square_of_sum(n0_x, x, Dummy_0767)

  implicit none
  real(kind=8), intent(out)  :: Dummy_0767
  integer(kind=4), intent(in)  :: n0_x
  real(kind=8), intent(in)  :: x (0:n0_x - 1)
  real(kind=8) :: y

  somme(n0_x, x, y)
  Dummy_0767 = y*y
  return
end subroutine
!........................................

end module

another solution would be the following:

module f2py_ssq

use ssq, only: somme

use ssq, only: square_of_sum
implicit none




contains

!........................................
subroutine f2py_somme(n0_x, x, y)

  implicit none
  real(kind=8), intent(out)  :: y
  integer(kind=4), intent(in)  :: n0_x
  real(kind=8), intent(in)  :: x (0:n0_x - 1)

  y= somme(x)
  return
end subroutine
!........................................

!........................................
subroutine f2py_square_of_sum(n0_x, x, Dummy_0767)

  implicit none
  real(kind=8), intent(out)  :: Dummy_0767
  integer(kind=4), intent(in)  :: n0_x
  real(kind=8), intent(in)  :: x (0:n0_x - 1)

  Dummy_0767  = square_of_sum(x)
  return
end subroutine
!........................................

end module

This solution would be simpler however issue #218 will need to be fixed in order to use it

Check function/subroutine signatures in generated Fortran code

It is very important to have unit tests where we verify that we generate the correct function signatures. By this I mean that the following variables' properties must be correct:

• type
• kind
• shape
• intent

To this end we need to parse the generated Fortran files, and extract all information regarding the functions' signatures. @saidctb suggests that we use LFortran, which looks very neat and powerful, but we have no experience with that. @krystophny: do you have any recommendation?

Fix syntax classes to use new versions of textX

Since version 2.1 of textX, user-defined syntax classes must provide access to their keyword arguments through properties with the same name. This requires a few changes on our side.

Numpy rand not working

rand from numpy.random cannot be pyccelized. This is marked as to do in tests/codegen/scripts/numpyext.py

Modulo operator '%' not working on arrays

See xfail tests in test_arrays.py, which cannot epyccel-ize the following functions from module arrays.py:

array_int_1d_scalar_mod( x, a )
array_int_1d_mod( x, y )

array_int_2d_C_scalar_mod( x, a )
array_int_2d_C_mod( x, y )

array_int_2d_F_scalar_mod( x, a )
array_int_2d_F_mod( x, y )

where x and y are integer arrays, and a is an integer scalar.

Indexes containing indexed values do not work

Indexes containing indexed values do not work. The following error is raised:
TypeError: <class 'sympy.tensor.indexed.Indexed'> not supported in Fortran

The following is an example:

from pyccel.decorators import types

@types('double[:]','double[:]','int[:]')
def print_last( vals1, vals2, sizes ):
    x = vals1[sizes[0]-1]
    y = vals2[sizes[1]-1]
    print(x,y)

from numpy import array

v1 = array([1.0, 2.0, 3.0, 4.0])
v2 = array([1.0, 2.0, 3.0, 4.0, 5.0])
i  = array([4,5])

print_last(v1,v2,i)

Requirements_doc.txt is not up to date

The requirements ask for sympy=1.1.1 but pyccel/codegen/printing/pycode.py uses sympy.printing.pycode which does not seem to be available in this version of sympy.

Pyccel converts python "float" (=> double precision) to fortran float (single precision)

Pyccel converts python "float" (=> double precision) to fortran float (single precision).

The generated code therefore contains real(kind=4) and throws errors when it is passed an array generated by numpy with the correct precision (real(kind=8)).

See the following file for an example:

from numpy      import zeros

#$ header function getElem(float[:],int) results(float)
def getElem(f,i):
    return f[i]

f=zeros(4)
getElem(f,1)

Dtype ignored

In the numpy function array, the dtype argument is ignored in favour of pyccel's interpretation.

E.g:

from numpy      import array

#$ header function printArray(float[:],int)
def printArray(f,n):
    for i in range(n):
        print(f[i])

f=array([3, 1, 0, 2],dtype=float)
printArray(f,4)

Where f is interpreted as integer(kind=4) and therefore the function throws an error.

adding quoted string at beginning of .py file produces failing .f90

When adding as a header something like

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Tue Jul 24 15:25:43 2018

@author: calbert
"""

pyccel correctly identifies the first part into a Fortran comment, but doesn't convert the quoted one, which leads to a compile error. Since spyder automatically generates file headers of this form, it would be nice to be able to parse them.

Integer division '//' not working on arrays

See xfail tests in test_arrays.py, which cannot epyccel-ize the following functions from module arrays.py:

array_int_1d_scalar_idiv( x, a )
array_int_1d_idiv( x, y )

array_int_2d_C_scalar_idiv( x, a )
array_int_2d_C_idiv( x, y )

array_int_2d_F_scalar_idiv( x, a )
array_int_2d_F_idiv( x, y )

where x and y are integer arrays, and a is an integer scalar.

Pyccel translates list concatenation (Python) into array sum (Fortran)

For example see classes_1.py: Python and Fortran outputs differ.

Wrong allocation of intent(out) arguments

When translating a Python function that returns an array, Pyccel generates a Fortran subroutine with an intent(out) array variable and an allocate statement for it. But such array is not declared as allocatable, and hence the Fortran code cannot be compiled. This is related to issue #154.

Functions are marked "implicit none"

Functions are marked in fortran as implicit none. This line causes errors of the form:
getctype: No C-type found in "{'intent': ['out', 'out=s']}", assuming void.
when using f2py.

The following code shows this problem

#$ header function sum(double[:])
def sum(f):
    s=0.0
    for i in range(len(f)):
        s+=f[i]
    return s

The commands used were

pyccel sum.py -t
f2py -c sum.f90 -m sum

Arguments unnecessarily marked allocatable

Arrays are automatically marked allocatable, whether they have memory allocated or not.

This can be seen in the following code:

#$ header function times_two(float[:])
def times_two(f):
    for i in range(len(f)):
        f[i]*=2

This means that the generated code cannot be used with f2py.

Naming a variable "result" leads to type mismatch error

Type deduction is not carried out correctly when one of the arguments is called result. E.g. the following code:

from numpy      import empty

#$ header function find_span(float) results(int)
def find_span( x ):
    if x <= 1: result = 1
    elif x >= 2: result = 2
    else: result = 0
    return result

#$ header function eval(float,float[:])
def eval(x,result):
    span  =  find_span( x )

The function find_span returns an int. This is correctly recognised by pyccel, however the variable span in the function evalSpline1D is designated as a real. An error is then thrown if a further function is called with span as an argument

Add an example on how to use Pyccel to accelerate Python functions (called from Python)

Hi,

I'd like to try to use Pyccel to accelerate Python functions, but I need to call them from Python (and not to produce a Fortran program).

I didn't find any examples on how to do that in your documentation. There is something written on f2py, but it does not seem that the pyccel command line can use f2py and produce a Python extension...

How would you do that?

For example, how can I produce a C-extension from this module laplace.py?

import numpy as np

def laplace(image):
    """Laplace operator in NumPy for 2D images."""
    laplacian = (
        image[:-2, 1:-1] + image[2:, 1:-1] + image[1:-1, :-2] + image[1:-1, 2:]
        - 4*image[1:-1, 1:-1]
    )
    thresh = np.abs(laplacian) > 0.05
    return thresh

def laplace_loops(image):
    """Laplace operator for 2D images."""
    h = image.shape[0]
    w = image.shape[1]
    laplacian = np.empty((h - 2, w - 2))
    for i in range(1, h - 1):
        for j in range(1, w - 1):
            laplacian[i-1, j-1] = (
                np.abs(image[i-1, j] + image[i+1, j] + image[i, j-1]
                + image[i, j+1] - 4*image[i, j]) > 0.05
            )
    return laplacian

(taken from a Transonic example: https://transonic.readthedocs.io/en/latest/examples/using_jit.html#comparison-numba-vs-transonic)

Wrong Indexing in Fortran

According to the Fortran spec, indexing starts at 1 unless specified to be otherwise. Pyccel specifies that arrays created within a function have indices starting at 0 but does not specify this for the arguments to the function. This means that within the function some arrays have indices starting at 0 and some have indices starting at 1.

Pyccel also does not change the indices used in python at all meaning that all accesses to elements of an array passed as an argument are wrong.

Pyccelize a folder

Starting from a folder:

dir/
   a.py
   b.py
   c.py

The following files should be created:

dir/
   mod_a.f90
   a.f90
   mod_b.f90
   b.f90
   mod_c.f90
   c.f90

The files which don't start with mod_ will only be generated if the python file contains an @external.
These files will be fortran 77 style and will only contain subroutines.
These files can then be used to generate .so files using f2py.

They files will contain wrapper functions for the functions marked @external in the associated mod_ file. The wrapper function will have the same name as the original function.

E.g.:
a.py:

from pyccel.decorators import external, types

@external
@types('double[:]')
def somme(x):
    y = 0.0
    for xi in x:
        y = y + xi
    return y

mod_a.f90:

module a

implicit none




contains

!........................................
real(kind=8) function somme(x)  result(y)

implicit none
real(kind=8), intent(in)  :: x (0:)
integer(kind=4) :: Dummy_9386
real(kind=8) :: xi

y = 0.0d0
do Dummy_9386 = 0, size(x,1) - 1, 1
  xi = x(Dummy_9386)
  y = xi + y
end do

return
end function
!........................................

end module

a.f90:

subroutine somme(n0_x, x, y)
use mod_a, only: mod_somme => somme

  implicit none
  real(kind=8), intent(out)  :: y
  integer(kind=4), intent(in)  :: n0_x
  real(kind=8), intent(in)  :: x (0:n0_x - 1)

  y = mod_somme(x)

  return
end subroutine

This will fix issue #219. This will fix issue #220

When pyccelizing the folder a tree of imports must be created first in order to avoid rereading functions to find their signature. This will fix issue #218

Atan2 from numpy not implemented

The following code :

from pyccel.decorators import external

@external
def test_arctan2():
    from numpy import arctan2
    return arctan2(2.0)

from pyccel.epyccel import epyccel

test2=epyccel(test_arctan2)
test2()

Generates the following error :

pyccel: [semantic] 
 |error: [4,5-23]| Undefined function (arctan2)

Traceback (most recent call last):
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 225, in annotate
    ast = self._visit(ast, **settings)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 1056, in _visit
    return getattr(self, annotation_method)(expr, **settings)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 1070, in _visit_Tuple
    ls = [self._visit(i, **settings) for i in expr]
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 1070, in <listcomp>
    ls = [self._visit(i, **settings) for i in expr]
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 1056, in _visit
    return getattr(self, annotation_method)(expr, **settings)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 2481, in _visit_FunctionDef
    expr.body]
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 2480, in <listcomp>
    body = [self._visit(i, **settings) for i in
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 1056, in _visit
    return getattr(self, annotation_method)(expr, **settings)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 2354, in _visit_Return
    for assign in assigns]
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 2354, in <listcomp>
    for assign in assigns]
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/semantic.py", line 2033, in _visit_Assign
    self.insert_symbolic_function(new_expr)
  File "/home/admlocal/.local/lib/python3.6/site-packages/pyccel/parser/base.py", line 508, in insert_symbolic_function
    raise TypeError('Expected a symbolic_function')
TypeError: Expected a symbolic_function

If decorators are commented then pyccel fails without an error

If there are comments amongst the decorators then pyccel fails, however no error is shown and pyccel gives no indication that it has not succeeded. E.g:

from pyccel.decorators import pure, external, types

@pure
@external
@types('double[:,:]','double[:]','double[:]')
#@types('int[:,:]','int[:]','int[:]')
def mat_mult(A,x,b):
    for i, l in enumerate(b):
        b[i] = 0
        for j, xi in enumerate(x):
            b[i] += A[i,j] * xi

epyccel cannot compile functions with no input arguments

This simple function cannot be translated to Fortran using epyccel:

def one():
    return 1

(a+b)//2 translated to 0*(a+b)

The following code

(a+b)//2

where a and b are ints is translated to

Int((1/2)*(a + b))

1 and 2 are ints so 1/2=0 not 0.5

Argument detected as intent(in) instead of intent(inout) even if call to another function changes its value.

Related to issue #198 .

When using functions from another python file the file is not read which means that all arguments are marked as in when they might be inout.

E.g.:

file1:

from pyccel.decorators import pure, types

@pure
@types('double[:]')
def mult_2(x):
    for i, xi in enumerate(x):
        x[i] = xi * 2

file2:

from pyccel.decorators import external, pure, types
from file1 import mult_2

@external
@pure
@types('double[:]','double[:]')
def add_2(a,b):
    mult_2(a)
    b = b + a

x is correctly identified as inout, but a is identified as in.

Remove deprecation warnings.

Functions returning arrays are translated to subroutines without warning

Functions which return an array are translated to subroutines, which affects their ability to be called correctly. E.g:

from numpy      import empty

#$ header function getGrid(float,float,int) results(float[:])
def getGrid(a,b,n):
    result = empty(n)
    for i in range(n):
        result[i] = a+i*(b-a)/(n-1)
    return result


f=getGrid(0,1,10)

Pyccel does not raise an error or a warning but converts the function to a subroutine which means that it is not used in the same way. This results in other areas of the code raising unreadable errors.

This may be related to fortran 77 (can't return an array) v fortran 90 (can return an array), but there is no difference between epyccel and pyccel

use a larger number for random strings

When functions have multiple outputs the order is not conserved

The order of the out parameters is not deterministic for functions returning tuples.

For example calling pyccel on the a file containing the following code:

from pyccel.decorators import external, pure

@external
@pure
def get_2_3():
    return 2,3

can generate both of the following versions of fortran code :

module tmp

implicit none




contains

!........................................
pure subroutine get_2_3(Dummy_117, Dummy_76)

implicit none
integer(kind=4), intent(out)  :: Dummy_117
integer(kind=4), intent(out)  :: Dummy_76

Dummy_117 = 2
Dummy_76 = 3
return
end subroutine
!........................................

end module

(where the out parameters are in the original order and the code provides the same result when run through f2py)

and :

module tmp

implicit none




contains

!........................................
pure subroutine get_2_3(Dummy_357, Dummy_255)

implicit none
integer(kind=4), intent(out)  :: Dummy_357
integer(kind=4), intent(out)  :: Dummy_255

Dummy_255 = 2
Dummy_357 = 3
return
end subroutine
!........................................

end module

(where the values are returned in the wrong order)

Pointer to an allocated array

Right now the following code

results in

while xp must be a pointer!!

Recursive functions do not work

E.g:

from pyccel.decorators import types
from pyccel import epyccel

@types(int)
def fib(n):
    if (n==1):
        return 0
    if (n==2):
        return 1
    return fib(n-1) + fib(n-2)

print(fib(6))

fib2 = epyccel(fib)

print(fib2(6))

Gives the error: Undefined function (fib)

Online docs (readthedocs) inconsistent with documentation sources

Two issues:

• Last passed bulid was a year ago: https://readthedocs.org/projects/pyccel/builds/7960602/
• Source code missing from tutorials (see for example, https://pyccel.readthedocs.io/tutorial.html#hello-world)

np.arctan2(a,b) translated to atan(a/b)

Numpy's arctan2 is translated to atan combined with a division. This will cause problems if b=0 when the function should return pi/2. Fortran has a function atan2. This should be used instead

Split comments are not fully commentated

If a comment is too long then pyccel splits it into multiple lines, however only the first line is commented

Test Windows operating system support

After discussing with @yguclu we are planning to test Windows support with Anaconda Python and automated testing with Travis CI. All tests that don't use MPI seem to run by this strategy.

In the following steps, if using your own PC you have to restart git bash instead of running refreshenv as in Travis CI.

Steps (Travis uses choco package manager from https://chocolatey.org/ ):

• Install anaconda, git and mingw compilers with

choco install anaconda3
choco install git
choco install mingw
refreshenv
source /c/tools/Anaconda3/etc/profile.d/conda.sh
conda activate

(Command conda install -c conda-forge fortran-compiler would install flang instead of gfortran on windows.)

• Change default C compiler from M$ to mingw in distutils.cfg by running

echo -e "[build]\ncompiler = mingw32" >> /c/tools/Anaconda3/Lib/distutils/distutils.cfg

• Download x64 libblas.dll and liblapack.dll from https://icl.cs.utk.edu/lapack-for-windows/lapack/ to library directory

curl https://icl.cs.utk.edu/lapack-for-windows/libraries/VisualStudio/3.7.0/Dynamic-MINGW/Win64/libblas.dll 
-o /c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib/libblas.dll
curl https://icl.cs.utk.edu/lapack-for-windows/libraries/VisualStudio/3.7.0/Dynamic-MINGW/Win64/liblapack.dll 
-o /c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib/liblapack.dll

• Clone and install pyccel

git clone [email protected]:pyccel/pyccel.git
cd pyccel
pip install .

• Test with tests\run_tests.bat. MPI and INT64 stuff is currently deactivated.

For MPI support

• Download MS MPI runtime and SDK

curl -L https://github.com/microsoft/Microsoft-MPI/releases/download/v10.1.1/msmpisetup.exe -o  msmpisetup.exe
curl -L https://github.com/microsoft/Microsoft-MPI/releases/download/v10.1.1/msmpisdk.msi -o msmpisdk.msi

• Install MS MPI (run git bash as Administrator)

./msmpisetup.exe -unattend
sleep 5
msiexec //quiet //i msmpisdk.msi
refreshenv

• generate mpi.mod for gfortran according to https://abhilashreddy.com/writing/3/mpi_instructions.html

cd "$MSMPI_INC"
sed -i 's/mpifptr.h/x64\/mpifptr.h/g' mpi.f90
sed -i 's/mpifptr.h/x64\/mpifptr.h/g' mpif.h
gfortran -c -D_WIN64 -D INT_PTR_KIND\(\)=8 -fno-range-check mpi.f90
cd -

• generate static libmsmpi.a from msmpi.dll

cd "$MSMPI_LIB64"
cp $SYSTEMROOT/SysWOW64/msmpi.dll .
gendef msmpi.dll
dlltool -d msmpi.def -l libmsmpi.a -D msmpi.dll
cd -

• pip install mpi4py

Wrong Python to Fortran conversion for modulo operator (%)

Pyccel translates the Python code

a = -2 % 5    # a=3

into the Fortran code

a = Mod(-2,5)   ! a=-2

Instead, the Fortran function modulo should be used.

Function shape(a) must return inverted tuple if input array a has C ordering.

make html in doc fails due to file() used in doc/conf.py

In file doc/conf.py, line 192, the file() statement is used
f = file('latex_macros.sty')

This doesn't work anymore with Python 3 and causes sphinx doc generation to fail in make html for Python 3 based systems.

To fix it, one could be replace the line by
f = open('latex_macros.sty')

Cannot specify function result type using decorators

It is possible to define the type of a function's result with a header string, but not with a decorator. This limits the usability of epyccel.

Unreadable Errors

Many problems throw errors which are unreadable and do not help find the problem.

E.g. enumerate does not seem to be implemented it.
Using it throws the following error:

Traceback (most recent call last):
  File "/usr/local/bin/pyccel", line 11, in <module>
    load_entry_point('pyccel==0.1', 'console_scripts', 'pyccel')()
  File "/usr/local/lib/python3.5/dist-packages/pyccel/commands/console.py", line 244, in pyccel
    binary=binary)
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/utilities.py", line 179, in execute_pyccel
    code = codegen.doprint()
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/codegen.py", line 299, in doprint
    code = printer(self.expr)
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/printing/fcode.py", line 2121, in fcode
    return FCodePrinter(settings).doprint(expr, assign_to)
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/printing/codeprinter.py", line 60, in doprint
    lines = self._print(expr).splitlines()
  File "/home/emily/.local/lib/python3.5/site-packages/sympy/printing/printer.py", line 257, in _print
    return getattr(self, printmethod)(expr, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/printing/fcode.py", line 274, in _print_Program
    '{sep}\n').format(funcs=funcs, sep=sep, f=self._print(i))
  File "/home/emily/.local/lib/python3.5/site-packages/sympy/printing/printer.py", line 257, in _print
    return getattr(self, printmethod)(expr, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/printing/fcode.py", line 1216, in _print_FunctionDef
    body_code = '\n'.join(self._print(i) for i in body)
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/printing/fcode.py", line 1216, in <genexpr>
    body_code = '\n'.join(self._print(i) for i in body)
  File "/home/emily/.local/lib/python3.5/site-packages/sympy/printing/printer.py", line 257, in _print
    return getattr(self, printmethod)(expr, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pyccel/codegen/printing/fcode.py", line 1409, in _print_For
    itr_=Range(expr.iterable.element.shape[0])
TypeError: 'NoneType' object is not subscriptable