rosenbrockc/fortpy

Python Emacs Intellisense and Unit Testing Support for Fortran

fortpy's Issues

Unit Testing Embedded Executables

We need to show a warning when a unit test is defined for a subroutine or function that is declared inside of another subroutine. Since they are buried, they can't be made public in the module for unit testing purposes.

Another option is to cut and paste the code automatically so that its definition is outside the subroutine contains section and then it could be made public. This is more user-friendly. The cut and paste should be simple since we already have the start and stop lines for the subroutine and could just move them outside of the parent subroutine completely. The only complexity that emerges is:

• if there is only one nested executable, the contains statement is invalid and would have to be removed.
• if there are multiple nested executables that reference each other, then moving one of them out, but not the others, will result in compilation failures. Although fortpy can detect most dependencies and track them down, there are a couple of cases where it doesn't work well for functions (the parser needs to be revisited). If we can trust the dependency listings, we will know which of the executables to cut out of the contains section.

Non-public logical functions won't compile if not explicitly declared as public.

For an example see symlib/vector_matrix_utililities.f90 minkowski_conditions_check

More Helpful XML ParseError Messages

For docstring.py line #267 add a more helpful error message that shows which token is causing the ParseError. Especially useful if the user doesn't know which XML characters are special.

Compiler Support

We need fortpy to handle multiple compilers and versions. The input/output files also need to vary based on the type of compiler (because they handle the random seeds differently). Here is a proposal:

Create a file called compilers.xml, it should have a list of compilers that fortpy can use to test with:

<compiler path="/usr/bin/gfortran_4.9.0" suffix="g49" name="gfortran49" default="true" />

• path specifies the full path to the compiler executable that fortpy should use.
• suffix if specified, this suffix will be spliced into all input/output files that have [c] in the filename (i.e. [c] will be replaced by this suffix before copying input/output files to execution directories).
• name for the -compiler argument of runtests.py. Change that argument to accept a string. If the string is *, then run the unit tests separately for every compiler in the compilers.xml file. Otherwise, only run those compilers whose names match the string given.
• default if -compiler is run without specifying a string, this is the compiler that would be used by default. If none of the compilers in the file have this attribute, the first one listed is used.

The unit tests would be configured to create directories like bcs.do_reweighted.g49, one for each compiler with its suffix. The contents would be exclusive to that compiler then. Ideally, unit tests should be run for every minor compiler version.

Warning Levels and Verbosity

When the model output file doesn't exist, fortpy is supposed to print a warning. msg.warn is being called, but the current verbosity implementation is preventing that, even when -verbose is specified. We should probably upgrade the missing model output to an error anyway, but the problem remains.

Operator and Assignment Symbol Support in Auto-Complete

When an interface defines an operator or assignment operation for a custom types, we would like isense support for using those symbols. For example custom_type = should suggest the XML documentation for the special assignment interface based on the type of the symbol to the left of "=". Since the assignment symbol can be anything (it is defined in the interface), we should support any of the custom symbols.

In a similar vein, if operators are defined with the .operator. symbol syntax, it would be nice if we got documentation support for custom_type .operator. where the operator is explained.

For both of these support implementations, we would just need to add some logic to the in_function_call() and adjust the user contextualizer to search for more than just "=" when looking for assignments and to identify custom operators.

`fortpy.f90` not Auto-updated After a `pip install --upgrade`

When fortpy gets upgraded with pip install, it gets the newest version of the fortpy.f90 file installed to the fortpy site-package. However, if a unit test directory already existed, then the fortpy.f90 file it has is not necessarily overwritten, even if we have a newer copy available. Perhaps looking at the creation date could tell us which version?

Even better, let's encode the fortpy version information in a fortpy XML tag at the top of fortpy.f90 and then examine the one we have in site-packages to decide whether to overwrite it when -rerun is not specified.

Missing context-specific completions for autocomplete

Look at line 80 in attached file

Case sensitivity of parameters. If capitol the parser fails to recognize it.

code:
CONTAINS
!!

This routine takes an integer 3x3 matrix and computes its
!!Smith Normal Form.

!!Input matrix.
!!Smith Normal Form.
!!Left Transform.
!!Right Transform.
subroutine SmithNormalForm(H,A,M,B)
error:
Traceback (most recent call last):
File "/Users/wileymorgan/.virtualenvs/fortpy/bin/runtests.py", line 91, in
initialize()
File "/Users/wileymorgan/.virtualenvs/fortpy/bin/runtests.py", line 39, in initialize
t.writeall(args["codedir"])
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/tester.py", line 503, in writeall
self.tgenerator.write(self._codefolder)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/generator.py", line 86, in write
self._write_module(module)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/generator.py", line 102, in _write_module
self._write_executable(module, anexec)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/generator.py", line 197, in _write_executable
self.xgenerator.write(testid)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/executable.py", line 146, in write
lines.append(self.writer.lines(testid))
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/method.py", line 139, in lines
self.method.group.code(testid, "after", result, " ")
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/elements.py", line 1984, in code
self._codes[testid][section](lines, spacer)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/elements.py", line 1717, in code_after
t.code(lines, self.variables, "each", spacer)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/elements.py", line 1440, in code
self._process(variables, spacer)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/elements.py", line 1492, in _process
"a tag or regular=true parameter doc tag.")
ValueError: Variable m has not been initialized with a tag or regular=true parameter doc tag.

Multidimensional Arrays for Unit Testing

Currently, fortpy only supports multi-dimensional arrays through the ragged array options. This forces the user to split their 3+ dimension array into a set of files, each of which is 2D. For example, a matrix of point group operations with size (48, 3, 3) would be split into 48 files, each of which has a 3 x 3 matrix in it.

The problem of handling multi-D arrays in files is not new. Here are some of the common solutions:

1. Write the shape of the array on the first line in the file and then write 2D slices of the array, in order, to the file. The shape allows quick reconstruction using a function like numpy.reshape.
2. Use the HDF5 file standard to binary serialize the resulting matrices completely. There are strong libraries supporting python and fortran for HDF5; also the resulting files would be dense and occupy less space. Disadvantage: the files are no longer human-readable. This means that an HDF5 file generating as "model" output can't easily be verified without browsing the HDF5 data set using some other tool to make sure the results are correct.
3. Use NetCDF, which is similar to the idea of HDF5 (compression, binary serialization, etc.); its advantage is that it was particularly designed to handle arrays of varying dimension.

In some respects, using HDF5 seems like a really good idea for all array-valued outputs/inputs. A unit test could have all its outputs etc. saved into a single HDF5 data set file. We would have to update fortpy to be able to handle the comparisons well. Since HDF provides a python interface to the file contents, this upgrade would be tiny. Most of the work would be changing the way that data gets saved. We could do this by just editing the fortpy.f90 file's pysave interface. The unit tests would also use up less disk space then. We would also need to find a good HDF file browser helping developers check their model output.

Thoughts?

Fortpy Module Use

If one of the modules being unit-tested decides to use fortpy, we can run into problems with double definitions of the variable kinds. I tried changing fortpy's internal kind definitions to be called fdp for double-precision etc. to avoid the conflicts. However, the driver program needs those definitions for variable kind to be local to the program it generates.

The general solution that would work for everything would be to rename the variable kinds when the driver program is generated to be something unique. Then define that unique kind locally in the program to match the original definition of the module being unit tested. We could follow the dependency chain up to the module that defines the types and look those members up.

Quiet Mode

Now that we are moving toward using a CI server, we need to be able to run the unit tests in quiet mode. The only feature that would need to be modified is that when the executable already exists and then the compilation fails, fortpy asks the user if they still want to run the linked executable. Such interaction will break the CI server. A -silent flag would be good.

"Index out of range" Error

When unit tests are run individually, the problem doesn't happen. If an entire trunk is rerun, the first unit test interferes with the second, which should never be the case. Error shows up on line 351 in ./testing/templates.py.

Probably the comparator instance has a value still initialized in its local memory from the last line of the previous comparison. When the next comparison starts, it has the wrong number of elements in the line to compare. Should be a quick fix.

mismatched tag in parser

uncle wileymorgan$ runtests.py trunk/ -stag ~/Documents/research/uncle/staging/ -rerun "findClusters"
ERROR: parsing executable dependency call get_spaceGroup_atomTypes
(if (digit1==digit2 )
tion "make_primitive" in this same module.e to a primitive one, use theye**
mary>orresponding fractional translations. of the space group and compute
Traceback (most recent call last):
File "/Users/wileymorgan/.virtualenvs/fortpy/bin/runtests.py", line 91, in
initialize()
File "/Users/wileymorgan/.virtualenvs/fortpy/bin/runtests.py", line 39, in initialize
t.writeall(args["codedir"])
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/tester.py", line 499, in writeall
self.parser.parse(filepath, True, True)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 305, in parse
self._parse_dependencies(pmodules, dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 118, in _parse_dependencies
self.load_dependency(base, dependencies and recursive, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 319, in load_dependency
self.parse(self._pathfiles[fkey], dependencies, recursive)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 305, in parse
self._parse_dependencies(pmodules, dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 118, in _parse_dependencies
self.load_dependency(base, dependencies and recursive, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 319, in load_dependency
self.parse(self._pathfiles[fkey], dependencies, recursive)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 305, in parse
self._parse_dependencies(pmodules, dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 118, in _parse_dependencies
self.load_dependency(base, dependencies and recursive, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 319, in load_dependency
self.parse(self._pathfiles[fkey], dependencies, recursive)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 305, in parse
self._parse_dependencies(pmodules, dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 118, in _parse_dependencies
self.load_dependency(base, dependencies and recursive, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 319, in load_dependency
self.parse(self._pathfiles[fkey], dependencies, recursive)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 305, in parse
self._parse_dependencies(pmodules, dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 118, in _parse_dependencies
self.load_dependency(base, dependencies and recursive, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 319, in load_dependency
self.parse(self._pathfiles[fkey], dependencies, recursive)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 305, in parse
self._parse_dependencies(pmodules, dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 118, in _parse_dependencies
self.load_dependency(base, dependencies and recursive, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 319, in load_dependency
self.parse(self._pathfiles[fkey], dependencies, recursive)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 284, in parse
dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 159, in _parse_from_file
self._parse_docstrings(filepath)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 126, in _parse_docstrings
self.modulep.docparser.parsexml(xmlstring, self.modules)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/docstring.py", line 291, in parsexml
xmlroot = ET.fromstring(xmlstring)
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
parser.feed(text)
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
self._raiseerror(v)
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: mismatched tag: line 94, column 3

Real-time Update for Signatures

While fixing the real-time update for the docstrings, I changed the elements.Module.get_element() to use the start attribute instead of the absstart attribute that includes the docstrings of the code elements. This made it nice for docstrings since they were owned by the parent module and the module was found as owner for those edits; but now the signature update for rtupdate does not work. My guess is that get_element() is choosing the module as owner because start only marks the start of the body of the code element.

Probably we need to change get_element() back to how it was and then use the parent attribute of the code elements to get hold of the module for real-time docstring parsing.

Debug Module Use

Running the epc server in debug mode is not going to accomplish much at the moment. Although the debug module exists within Fortpy, there aren't enough debug assertions, writes or times for when the server runs in debug mode. Yes, they should have been put in while the module were being developed, but they weren't and now there has to be a revisiting and investment.

We could also just wait for bugs to turn up and then add the statements in as part of the debugging process; then all the most relevant places to put debug statements will be hit first.

Function outputting a variable other than it's name fails.

The issue has come up in rational_mathematics.f90, the gcd functions all return a divisor and not the normal format:
function gcd_2ints(x1, x2) result(divisor) integer, intent(in) :: x1, x2 integer :: divisor

In this case fortpy fails to write a successful driver because it can't find a type to assign to gcd_2ints_fpy.
This issue may also be related to issue #37.

parser fails to assign type to logical funtions

code:
function is_equiv_lattice(lat1,lat2,eps)
real(dp), intent(in) :: lat1(3,3), lat2(3,3), eps
logical is_equiv_lattice, err
real(dp) :: lat1inv(3,3), S(3,3)
integer i
is_equiv_lattice = .false.
call matrix_inverse(lat1,lat1inv,err)
if (err) stop "Problem with input vectors in function 'is_equiv_lattice'"
S = matmul(lat1inv,lat2)
if (equal(abs(determinant(S)),1._dp,eps) .and. &
equal(S,nint(S),eps)) is_equiv_lattice = .true.
endfunction is_equiv_lattice

fortpy compile log:

is_equiv_lattice.f90:10.2:

None :: is_equiv_lattice_fpy

Parser does not recognize target as a pointer

Code Cleanup: Jedi Remnants

The debug.py and settings.py have some settings and methods that we still have from when those files were copied from jedi. We just need to figure out what we don't need and delete it. Shouldn't take very long.

Parsing order for elements with pre-determined dimensions fails if intent statement after dimension declaration.

For example:
'integer, dimension(3), inent(INOUT) :: A'
Will produce the following code in the driver and won't compile:
'integer :: A(INOUT)'

Error Handling for Parsers

Although there is limited error handling for the XML parsers and the executable dependency parsers, there is generally no error handling for the parsing. If it encounters something new that hasn't come up in one of the test cases so far, it just croaks. Generally there are good tests for handling outliers, but every now and then I find a code fine that breaks the parsers, and they don't handle it gracefully. Up until now I have just edited the code file slightly to make it compatible.

We need some high level (at first) error handling for the parsing so that it exits gracefully. The best thing would be some lower-level handling that gives suggestions on how to update the code file. Most of the time it isn't a problem with the code, but with the comments. For example, using !! for emphasis when it is reserved for docstrings; or commenting out blocks of code in such a way that the parsers get tricked (though I have only ever seen that once).

User types

Fortpy does not recognize dependencies for user defined types in modules.

Isense Configuration for Suggestions

Include a new <isense> tag in the config.xml file that allows the completion suggestions to be customized. Allow the developers to choose the tags and attributes that get used when compiling the tooltips. This should be really quick to implement because of the XML structure and because we already have the machinery in place for finding the docstring elements based on the code context. It is just a matter of choosing new attributes/tags to include.

Probably will only take half and hour.

Prereqs for Assignment Value

When an embedded method is called, the prereq="true" option is supposed to add the pre-req executable instance to the pre-req chain. The method currently assumes that the assignment value's parent is a MethodFinder instance, when in reality it will always be an Assignment instance. See line 306.

No Body in File Comparison Template

If there isn't a body tag in an XML template, the code fails with divide by zero error on the percent_match. If we fix that, then it still reports only 50% match because the body is returning zero for all its comparisons. Body should be ignored if it isn't present in the template.

Format String for Cases not Applying

Also for itertools.f90 unit tests on cproduct, the file variable sources are not getting their file names replaced with the caseid by format string.

Removal of `class` Parameters for Non-embedded Executables

When the first argument is an instance of a user-derived type in a subroutine/function, we remove it because fortpy assumes that the subroutine is embedded in the type; but that is not always the case. We need to explicitly check the executables list for the type and make sure it is embedded before removing that argument.

Polymorphic Variable Declarations

When the driver is auto-generated, if the derived class variables are declared using class instead of type in any executables consuming them, we will get a compile error for polymorphic type expectance. We should use class/type as the default and make the developer explicitly select the other via a <global> tag declaration.

Rerun Filtering

If the -rerun flag is specified, fortpy reruns all the tests in the entire trunk. Often, the reason that -rerun is used is to retest something where the code hasn't changed yet (but perhaps the input file has). It takes long to rerun every test in the trunk.

Let -rerun accept a parameter (default *) that says which tests to rerun. Do lower-case pattern matching on the module names and only run those whose modules match the filter.

"list index out of range" error on auto-complete

As an example: open a module and try to type call rand for calling a built-in random function. The python backend generates this error; needs to be tracked down. Users say it happens quite frequently.

Missing context-specific completions for autocomplete

Look at line 80 in attached file

Update ./scripts/convert.py for Output Conversion

Although the file templates and comparer modules for comparing output files support writing a FileRepresentation into a new version, there is no easy script access to do that. It is still in ./experiment.py on my local machine (not part of repo).

No isense Support for Fortran Program Files

Go to an open line anywhere in a fortran program and type any key. Raises a deferred error IndexError("string index out of range",).

pysave_integer_1d fails if array is also a pointer.

Fortran runtime error: Expected P edit descriptor in format
(0 i12)
^

Plotting Options in the Fortpy Shell

This is probably going to come up soon enough, but I will document it now since I just thought about it. When the plots are generated by the unit test analysis shell, there are several plot option available already (axes labels for example), but there isn't arbitrary control of the colors, fonts, plot title etc. We could do some nice completion suggestions by tapping into the matplotlib modules using dir for the colors and some of the other options.

Allocation of output arrays fails when dependent on input array size

If the code is testing a function that a returns an array that is the same size as the input array, i.e.: FUNCTION factorial_int_rank1(N)
integer :: N(:),i,factorial_int_rank1(size(N)),j

Then fortpy tries to allocate the output array before reading the input:
integer, allocatable :: N(:)
integer :: factorial_int_rank1_fpy(size(N))
real(fdp) :: fpy_start, fpy_end, fpy_elapsed = 0
call fpy_read('factorial_array.in', '#', N)

Cache Invalidation for SSH Edits

When files are edited over tramp they get cached until the file is modified. The routine that invalidates the cache only runs whenever a module is requested to be loaded either directly or as a dependency of another module. Lots of developers save their file every minute out of habit; if the file on the remote server changes often, we don't necessarily want to keep reparsing it over SSH because it takes time and could impede the responsiveness of the auto-completion suggestions.

Perhaps it is good to enable a longer cache duration for SSH edits (e.g. 10 minutes) during which time the file won't be reparsed even if there is a newer version on disk. The fortpy:reparse-buffer-file function could still force the reparse manually, but it wouldn't be automated. Since we do have real-time update of code, I don't think it would adversely affect the developer experience much.

Public Routines for Unit Testing

At the moment, fortpy checks whether a routine you are trying to unit test is marked as public in the module. If it isn't then it warns you, but does nothing. What it ought to do is copy the module to the staging folder for the unit test and then insert the extra public declaration for that one routine so that it would still compile properly. This is especially true for initialization type routines where it is obvious that they shouldn't be public. The developer doesn't want to make everything in the module public just so the unit tests work.

Deallocation Issue in Ftypes

If the same shared library is repeatedly used with allocatable outputs, it appears as if it is sharing the memory between the results, so that subsequent results overwrite the first ones. It also means that the deallocation segfaults because the first pointer being released deallocates fine, but when the remaining ones try to deallocate it, it has already been deallocated.

I think it is still good to have every FtypeResult instance try to deallocate itself (it is cheap); we should just add a check for allocation before trying to deallocate. We probably also ought to investigate whether it is better to copy the array results before the method is called again. If we do the copy, there is obvious overhead and the memory becomes managed by python. If we don't copy, the user's results will be confusing with values being overwritten. If they are aware of that, they could do the copy themselves. If they are not, we have to do it transparently...

Comparison File Existence Check Order

When the unit test outputs are compared, fortpy first builds a representation of the model output it created before seeing if the model file to compare with even exists. In cases where the model output doesn't exist, this wastes a lot of time in the unit testing.

Before building a comparison, first make sure that the file you want to compare with exists.

Constant Allocation not Auto-Generating

Look at itertools.f90 in polya. The presult variable doesn't get allocated to the predefined constant value, even though we have an <assignment> tag for it.

Goto and Definition Retrieval

The fortpy.el file has support for getting definitions and navigating to definitions in code files. Fortpy doesn't support this yet. We have the character and line functions and attributes already available to do this easily, we just need to implement it.

Also, the goto from fortpy.el still has some complicated remnants from jedi that we don't need. Python needs lots of recursion to give good isense support, but Fortran doesn't. Our goto recommendations will be extremely precise and we don't need all the recursion support. Those methods can be deleted from fortpy.el or updated to work with it.

Compare.py

When attempting to run compare.py the following error occurs:
Traceback (most recent call last):
File "/Users/wileymorgan/.virtualenvs/fortpy/bin/compare.py", line 36, in
initialize()
File "/Users/wileymorgan/.virtualenvs/fortpy/bin/compare.py", line 13, in initialize
args["xmltemplate"] = "{}.xml".format(source.split("/")[-1])
NameError: global name 'source' is not defined

Compile Failure Notification

At the moment, when a compile fails, the tests still try to run but obviously end up with 0% accuracy. We want the program to detect compile failure (perhaps check the contents of the compile.log file). If compile fails, show the first 10 lines of the error log and quit the unit tests.

`fortpy-show-doc` Integration with `fortpy-el`

The full documentation for a subroutine/function can be called up using M-? in emacs. However, it currently only works for executables that are defined in the same module as the reference that calls it. If you open derivative_structure_generator.f90, go to line 339 and try to call up the docs, you get

deferred error : (error "\"AttributeError(\\\"'NoneType' object has no attribute 'fulldoc'\\\",)\"")

The same thing happens on line 328 for get_spaceGroup. However, on line 481, find_permutation_of_group works since it is defined within the same module.

Fortpy fpy_read_integer_2d finds to many values in the arrays.

When reading in a 2d array fortpy uses call fpy_linevalue_count(filename, commentchar, nlines, nvalues) to find the number of lines and values. At present it finds the number of lines correctly but fails on the number of value. For example if this file is passed in:
0
0
0
it finds these as the lines and values respectively:
3
2
or alternatively if this file is read in:
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
it finds the lines and values to be:
5
20

so far it seems like it counts each value twice. It then allocates to much space for the array triggering an EOF error.

Complex Functions

For function output under target name [default], we need to have the [default] variable able to be declared as <global> and have its value changed using <assignment> so that complicated functions can be tuned by the user.

The reason we need to introduce maximum flexibility is that a function may declare its output type using a function of the input parameters; or if the functions output is allocatable, then we need the user to specify the output type in advance so that compilation succeeds.

The fix for this will involve switching out the hard-coded function_fpy variable declaration in elements.py for a conditional statement that checks the test specification's list of globals first for a [default] variable; if it exists, then do nothing. The existing code for initializing and assigning values to the variables will suffice, we just need to change the name that gets output from [default] to the fortpy hard-coded value.

Describe testing approach

I ran into an error that I would like to fix, after enabling forpy-mode and running M-?:

deferred error : (error "\"AttributeError(\\\"'UserContext' object has no attribute 'module'\\\",)\"")

I believe that there might be a missing self.module = None in the usercontext class init.

To test it I want to write a python unit test. I did find the tox.ini file, implying that I should test with the tox command. If I run it it can't find any unit tests. Are there no unit tests yet? Can you suggest where I should put such a test?

'list' object has no attribute 'strip'

File "/Users/wileymorgan/.virtualenvs/fortpy/bin/runtests.py", line 76, in
initialize()
File "/Users/wileymorgan/.virtualenvs/fortpy/bin/runtests.py", line 27, in initialize
t.writeall(args["codedir"])
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/testing/tester.py", line 485, in writeall
self.parser.parse(filepath, True, True)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 305, in parse
self._parse_dependencies(pmodules, dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 118, in _parse_dependencies
self.load_dependency(base, dependencies and recursive, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 319, in load_dependency
self.parse(self._pathfiles[fkey], dependencies, recursive)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 284, in parse
dependencies, recursive, greedy)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/code.py", line 142, in _parse_from_file
pmodules = self.modulep.parse(string, self)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/module.py", line 128, in parse
return self._parse_modules(string, parent)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/module.py", line 163, in _parse_modules
module = self._process_module(name, contents, parent, rmodule)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/module.py", line 215, in _process_module
self.xparser.parse(result)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/executable.py", line 131, in parse
self.parse_block(module.refstring, module, module, 0)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/executable.py", line 146, in parse_block
x = self._process_execs(anexec, parent, module)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/executable.py", line 212, in _process_execs
self._parse_members(contents, result, params)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/executable.py", line 402, in _parse_members
members = self.vparser.parse(contents, anexec)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/variable.py", line 24, in parse
mems = self._process_member(member, parent)
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/variable.py", line 62, in _process_member
for name, dimension, default, D in self._clean_multiple_def(ready):
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/variable.py", line 157, in _clean_multiple_def
default = self._collapse_default(entry[1])
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/variable.py", line 123, in _collapse_default
args = self._collapse_default(s[1])
File "/Users/wileymorgan/.virtualenvs/fortpy/lib/python2.7/site-packages/fortpy/parsers/variable.py", line 121, in _collapse_default
name = s[0].strip(",")