nncarlson / petaca Goto Github PK
View Code? Open in Web Editor NEWPetaca: A collection of foundational Fortran modules
License: MIT License
Petaca: A collection of foundational Fortran modules
License: MIT License
There are IS_SCALAR, IS_VECTOR, ... methods for inquiring about a parameter, but none for inquiring about the type; that would be a useful addition.
The dynamic type of vector and matrix parameters is lost in GCC 9+, so a parameter_list
returns an "incorrect type" error when retrieving the parameter. This does not happen for scalar parameters.
See GNU 98573.
First, thanks for this remarkable and instructive Fortran code.
I did try to compile petaca with GNU gfortran 5.2.1 (this is the most recent stable version I have without compiling trunk source code) and got many errors and compiler bugs (ICEs), mostly for sourced allocations. Nevertheless, workarounds exist for many (all?) of them. Moreover, the next gfortran version (6.0) should fix this sourced allocation, so I am wondering if you have tried and abandoned this compiler already or there are still some plans to test and to include gfortran to the supported list?
In principle, such gfortran fixes could be managed and merged into the code with combined GFORTRAN and VERSION preprocessor directives (similarly to INTEL ifort bugs)?
The Intel bug workaround INTEL_DPD200237439
in parameter_list_json.F90 contains an error. The statement READ(10,...
should be READ(UNIT,...
Intel 19.1 is failing the map_any
and parameter_list
tests.
90% tests passed, 2 tests failed out of 20
Total Test time (real) = 3.87 sec
The following tests FAILED:
1 - fortran_dynamic_loader (Failed)
9 - secure_hash (Failed)
Errors while running CTest
The source file parameter_list_json.F90 makes use of "processor dependent" behavior of the SAME_TYPE_AS intrinsic function. Specifically it applies the function to CLASS(*) variables A and B whose dynamic types are intrinsic types (which are non-extensible). The Fortran 2008 standard says of the function, "If neither A nor B has extensible dynamic type, the result is processor dependent." This needs to be replaced with a portable alternative.
I'm testing the current version of gcc 8 (8.3.1-20190406) and test_any_matrix_type
is throwing this runtime error:
$ ./test_any_matrix_type
At line 464 of file /opt/src/petaca/petaca/test/parameter_list_type/test_any_matrix_type.F90
Fortran runtime error: Array bound mismatch for dimension 1 of array '<<unknown>>' (2/1)
Error termination. Backtrace:
#0 0x7f03ff61af9a in ???
#1 0x7f03ff61bb85 in ???
#2 0x7f03ff61bf57 in ???
#3 0x404495 in test_assignment
at /opt/src/petaca/petaca/test/parameter_list_type/test_any_matrix_type.F90:464
#4 0x400e02 in test_any_matrix_type
at /opt/src/petaca/petaca/test/parameter_list_type/test_any_matrix_type.F90:41
#5 0x41492f in main
at /opt/src/petaca/petaca/test/parameter_list_type/test_any_matrix_type.F90:27
I haven't investigated further, but I suspect a compiler bug. Earlier versions have been broken.
Report the Intel constructor bug associated with the INTEL_BUG20140921 workaround. This effects all compiler versions through 18.0.1.
... as a way of appending an existing list to another one, as a sublist. ... thank you for the library!
In preparation for the truchas spack release it would be helpful to tag a release after #26. Since this is a collection of utilities, I'm not sure if semver makes a lot of sense here. My vote is for the truchas naming scheme YY.MM
The MD5 and SHA hash functions are cryptographic hash functions, which are great for ensuring a uniform distribution of entries in the hash table. However, these hash functions are relatively expensive, and the finite size of the hash table means that at some point (adding enough entities to a table of a given size) more than one KVP will start mapping into the same bin.
I would like to recommend that alternate hash functions be made available as part of Petaca. My favorite, cheap, quick and dirty hash function with a decent distribution is Bob Jenkin's "One at a time" hash: https://en.wikipedia.org/wiki/Jenkins_hash_function#one-at-a-time My tests indicate a relatively uniform distribution when hashing strings.
The only complication with implementing this hash function in Fortran is that there is no unsigned integer type. However, all Fortran integer implementations are done using twos compliment which has the same over-flow, wrap-around behavior as unsigned integers. As a result the absolute value of the hash function result may be used prior to modular division for indexing into the hash table array, without adversely effecting the uniformity of the hash function distribution. (At least until the table size becomes greater than huge(hash_result_integer)
at which point, you can just use a larger signed Fortran integer.)
Here is the implementation I usually use:
elemental function hash(str)
!! Bob Jenkins' one-at-a-time hash function
!! Map a given character string onto an integer. For best performance,
!! this should be:
!! 1. fast
!! 2. deterministic
!! 3. uniformly distributed over the integers
!! 4. chaotic
!! See <http://burtleburtle.net/bob/hash/doobs.html>, and
!! <http://www.stanford.edu/class/ee380/Abstracts/121017-slides.pdf>
!! (slides from talk of google cityhash developers to Stanford)
character(len=*), intent(in) :: str
!! string to hash
integer(selected_int_kind(8)) :: hash
!! Numeric hash
character(len=len(str)) :: str2
integer(selected_int_kind(8)) :: i, stlen
str2 = adjustl(str)
stlen = len(trim(str2))
hash = 0
do i = 1,stlen
hash = hash + ichar(str2(i:i))
hash = hash + ishft(hash,10)
hash = ieor(hash,ishft(hash,-6))
end do
hash = hash + ishft(hash,3)
hash = ieor(hash,ishft(hash,-11))
hash = abs(hash + ishft(hash,15))
end function
I'm pretty busy at the moment, but if you're amenable to including this as an option, I may try to find the time required to submit a PR to implement this.
This issue tracks the status of compiling petaca using the flang compiler, which is currently not working.
You should use the flang
branch, which contains additions, workarounds, etc. for the compiler. I periodically rebase it onto the master branch (bad practice, I know -- sorry).
The refactored parameter_list module code in version 23.12 triggers a bug in gfortran 12.3. Here's a small reproducer:
use parameter_list_type
call bug
contains
subroutine bug
type(parameter_list) :: params
type(parameter_list), pointer :: plist
plist => params%sublist('foo')
call plist%set('bar', 'fubarfubarfubarfubarfubarfu')
end subroutine
end
And its output when run:
$ ./a.out
malloc(): corrupted top size
Program received signal SIGABRT: Process abort signal.
Backtrace for this error:
#0 0x7f22ca21599f in ???
#1 0x7f22ca267834 in ???
#2 0x7f22ca2158ed in ???
#3 0x7f22ca1fd8fe in ???
#4 0x7f22ca1fe7cf in ???
#5 0x7f22ca2717a4 in ???
#6 0x7f22ca2751cb in ???
#7 0x7f22ca275e21 in ???
#8 0x7f22ca829590 in __parameter_list_type_MOD___final_parameter_list_type_Parameter_list
at /opt/src/petaca/petaca/src/parameter_list_type.F90:1695
#9 0x401343 in ???
#10 0x401362 in ???
#11 0x401398 in ???
#12 0x7f22ca1ff149 in ???
#13 0x7f22ca1ff20a in ???
#14 0x4010d4 in ???
#15 0xffffffffffffffff in ???
Aborted (core dumped)
The abort happens when returning from the subroutine, and looks to be associated with finalization of the local params
variable in some way.
Note that the current parameter_list
unit tests all pass -- this failure was first observed in application code.
Currently integer numbers are converted to default integer values and real numbers to 64-bit real values when reading JSON text. This is what the YAJL parser does. It would be nice to allow the user to specify the kinds of the values that get added to the parameter list; e.g., 32-reals instead of 64-bit. This would entail conversions with possible overflow that would need to be handled.
While one can manually create parameter values that are arrays of parameter lists through existing calls, it is not currently possible to create such things from JSON text input, which is a needed capability. The current limitation was a concession to simplicity in constructing array values during parsing, where only one array is being constructed at a time. If arrays of parameter lists, which themselves may contain arrays, are allowed, then multiple arrays may be in the process of being constructed at any moment during parsing. This requires a more complex stack implementation. This has since been done in the json.F90
module.
This is related to GNU 93762.
In short, when passing optional deferred-length strings through multiple calls, gfortran doesn't propagate the string's length up the chain. The string length at the top of the chain is uninitialized, which can lead to allocation errors like
Operating system error: Cannot allocate memory
Memory allocation failure in xrealloc
This bug is triggered whenever an errmsg
is returned by one of the parameter_list
procedures, for example
call params%get('foo', x, stat=stat, errmsg=errmsg) ! 'foo' is not in the parameter list
if (stat/=0) print *, errmsg ! error
If integers are default initialized to 0, such as with the flag -finit-integer=0
, then the above error is not catastrophic but an empty string is returned.
test_fortran_dynamic_loader
is now failing on some platforms with the error
SHLIB:OPEN: /lib64/libm.so: invalid ELF header
In these cases It turns out that the .so
file is not actually a library but a 'linker script' text file. I guess the linker knows how to deal with these, but the system dynamic loader function (dlopen
) does not.
The unit test already includes tests that load a shared library built as part of the test, so the solution is probably just to delete the one test that loads /lib64/libm.so
, which was a little iffy in the first place.
Unit tests for parameter_list_json are needed.
Some parameter_list
methods will return an allocated 0-length errmsg
when stat
returns 0. Possibly this was intended? My thinking nowadays is that errmsg
should only be allocated when stat
returns a non-zero value.
Using Intel ifx 2024.0:
$ make
[ 1%] Building Fortran object src/CMakeFiles/petaca.dir/fortran_dynamic_loader.F90.o
[ 3%] Building Fortran object src/CMakeFiles/petaca.dir/yajl_fort.F90.o
[ 4%] Building Fortran object src/CMakeFiles/petaca.dir/json.F90.o
[ 6%] Building Fortran object src/CMakeFiles/petaca.dir/map_any_type.F90.o
[ 8%] Building Fortran object src/CMakeFiles/petaca.dir/parameter_entry_class.F90.o
[ 9%] Building Fortran object src/CMakeFiles/petaca.dir/parameter_list_type.F90.o
[ 11%] Building Fortran object src/CMakeFiles/petaca.dir/parameter_list_json.F90.o
[ 13%] Building Fortran object src/CMakeFiles/petaca.dir/secure_hash/secure_hash_class.F90.o
[ 14%] Building Fortran object src/CMakeFiles/petaca.dir/secure_hash/md5_hash_type.F90.o
[ 16%] Building Fortran object src/CMakeFiles/petaca.dir/secure_hash/sha1_hash_type.F90.o
[ 18%] Building Fortran object src/CMakeFiles/petaca.dir/secure_hash/secure_hash_factory.F90.o
[ 19%] Building Fortran object src/CMakeFiles/petaca.dir/state_history_type.F90.o
[ 21%] Building Fortran object src/CMakeFiles/petaca.dir/timer_tree_type.F90.o
[ 22%] Building Fortran object src/CMakeFiles/petaca.dir/f90_assert.F90.o
[ 24%] Building C object src/CMakeFiles/petaca.dir/yajl_ext.c.o
[ 26%] Linking Fortran shared library libpetaca.so
[ 26%] Built target petaca
[ 27%] Building Fortran object test/fortran_dynamic_loader/CMakeFiles/test_fortran_dynamic_loader.dir/test_fortran_dynamic_loader.F90.o
#0 0x0000000002394df7 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2394df7)
#1 0x0000000002394f20 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2394f20)
#2 0x000014968ac5c710 (/usr/lib/libc.so.6+0x3e710)
#3 0x00000000036873d6 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x36873d6)
#4 0x0000000003684e8f (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x3684e8f)
#5 0x000000000329d5bd (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x329d5bd)
#6 0x00000000026d10e3 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x26d10e3)
#7 0x00000000022e19fd (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x22e19fd)
#8 0x0000000002f425b4 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2f425b4)
#9 0x00000000032ae80d (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x32ae80d)
#10 0x0000000002f3dc45 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2f3dc45)
#11 0x0000000003297e0d (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x3297e0d)
#12 0x0000000002f4027d (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2f4027d)
#13 0x00000000032aeafd (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x32aeafd)
#14 0x0000000002f3f33a (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2f3f33a)
#15 0x0000000003297fed (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x3297fed)
#16 0x00000000026cfce3 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x26cfce3)
#17 0x00000000032fcb75 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x32fcb75)
#18 0x000000000329a7ad (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x329a7ad)
#19 0x00000000026cfce3 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x26cfce3)
#20 0x00000000022c9d9a (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x22c9d9a)
#21 0x00000000022c7d6d (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x22c7d6d)
#22 0x0000000002270263 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2270263)
#23 0x0000000002452e7e (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x2452e7e)
#24 0x000014968ac45cd0 (/usr/lib/libc.so.6+0x27cd0)
#25 0x000014968ac45d8a __libc_start_main (/usr/lib/libc.so.6+0x27d8a)
#26 0x00000000020ab129 (/opt/intel/oneapi/compiler/2024.0/bin/compiler/xfortcom+0x20ab129)
/tmp/ifx1027734257LAoUzI/ifxXBrFOe.i90: error #5633: **Internal compiler error: segmentation violation signal raised** Please report this error along with the circumstances in which it occurred in a Software Problem Report. Note: File and line given may not be explicit cause of this error.
compilation aborted for /home/zjibben/codes/telluride/petaca/build/test/fortran_dynamic_loader/test_fortran_dynamic_loader.F90 (code 3)
make[2]: *** [test/fortran_dynamic_loader/CMakeFiles/test_fortran_dynamic_loader.dir/build.make:75: test/fortran_dynamic_loader/CMakeFiles/test_fortran_dynamic_loader.dir/test_fortran_dynamic_loader.F90.o] Error 3
make[1]: *** [CMakeFiles/Makefile2:403: test/fortran_dynamic_loader/CMakeFiles/test_fortran_dynamic_loader.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
I'm testing the current version of gcc 8 (8.3.1-20190406) and test_parameter_list_type is throwing this runtime error:
$ ./test_parameter_list_type
At line 768 of file /opt/src/petaca/petaca/src/parameter_entry_class.F90
Fortran runtime error: Array bound mismatch for dimension 1 of array '<<unknown>>' (4/2)
Error termination. Backtrace:
#0 0x7fd646cbaf9a in ???
#1 0x7fd646cbbb85 in ???
#2 0x7fd646cbbf57 in ???
#3 0x41ba84 in __parameter_entry_class_MOD___copy_parameter_entry_class_Any_matrix
at /opt/src/petaca/petaca/src/parameter_entry_class.F90:768
#4 0x4199ad in new_list_item
at /opt/src/petaca/petaca/src/map_any_type.F90:166
#5 0x419236 in __map_any_type_MOD_insert
at /opt/src/petaca/petaca/src/map_any_type.F90:249
#6 0x43d206 in __parameter_list_type_MOD_set_matrix
at /opt/src/petaca/petaca/src/parameter_list_type.F90:638
#7 0x416acd in test_basic
at /opt/src/petaca/petaca/test/parameter_list_type/test_parameter_list_type.F90:69
#8 0x400e46 in test_parameter_list_type
at /opt/src/petaca/petaca/test/parameter_list_type/test_parameter_list_type.F90:32
#9 0x417432 in main
at /opt/src/petaca/petaca/test/parameter_list_type/test_parameter_list_type.F90:27
I haven't investigated further. I suspect a compiler bug, and this may be the same thing as #14.
Using valgrind to investigate #34 turned up some incidental memory leaks. These are mostly in the unit tests themselves, but parameter_list_json.F90
may leak memory (the partially built parameter list) when a parsing error is returned. None of these are particularly important, but it would be ideal to have everything completely clean.
Intel 19.0.5 is failing the parameter_list_json
test with a segfault. I haven't investigated further.
The timer_tree::stop
procedure has optional error return arguments. Are these really needed?
I'm using a wonky ->
as the separator between the parts of a parameter list name, and I don't recall why. Consider replacing this by JSONPath or JSON Pointer style syntax.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.