chapel-lang / chapel Goto Github PK
View Code? Open in Web Editor NEWa Productive Parallel Programming Language
Home Page: https://chapel-lang.org
License: Other
a Productive Parallel Programming Language
Home Page: https://chapel-lang.org
License: Other
These branches are now taken care of by the locale model, and so the branches should have been removed with #4074. Replace with assert(node != chpl_nodeId)
.
PR #4342 enabled initial generation of debug information with --llvm. However, some issues remain:
I've found it useful to allow some Chapel arrays to be read without knowing their size in advance. In particular, non-strided 1-D Chapel arrays that have sole ownership over their domain could be read into where that operation will resize the array to match the data read. At one point, I prototyped this for JSON and Chapel style textual array formats (e.g. [1,2,3,4] ).
Here is an example of the code I'm interested in supporting:
var A:[1..0] int;
mychannel.read(A);
could read into A any number of elements and adjust its domain accordingly. The alternative is that the code above only reads 0-long arrays.
This case is particularly relevant because one might imagine a record that stores a variable-length array. Can the default readThis operation provided by the compiler do something reasonable?
record ContainingArray {
var A: [1..0] int;
}
var r:ContainingArray;
mychannel.read(r);
Or, is it necessary for authors of such records to implement a custom readWriteThis method if they wanted I/O to work in a reasonable manner?
There are three key questions:
Does changing the size of a read array when possible seem like the right idea? Or should reading an array always insist that the input has the same size as the existing array (which I believe is behavior that matches the rest of the language for arrays that share domains...)
Should any-dimensional rectangular arrays be written in binary in a form that encodes the size of each dimension? (In other words, write the domain first?). Such a feature would make something like (1) possible for multi-dimensional arrays but might not match what people expect for binary array formats. (I don't think we've documented what you actually get when writing an array in binary yet...)
Any suggestions for a Chapel array literal format for multi-dimensional arrays? How would you write such arrays in JSON (and would anyone want to)? At one point there was a proposal to put the domain in array literals, like this:
var A = [ over {1..10} ];
but that doesn't really answer how to write multidimensional array literals. One approach would be to store the array elements in a flat way and just reshape them while reading; e.g.
var A = [ over {1..2, 1..3}
11, 12, 13,
21, 22, 23 ];
where the spacing would not be significant.
If we had a reasonable format, we could extend support like (1) to any-dimensional arrays that do not share domains, even for some textual formats.
While investigating performance for Chapel benchmarks, I find that the line number and file are not quite high-resolution enough to find the offending GET or PUT in the generated C code. It's not uncommon for a series of GETs in the generated code for an iterator to share these two pieces of information. Even if gets and puts could be mapped to the internal line/file, I'd still want to be able to find the event in the case that the function was duplicated (say, for generics).
I'm proposing that the chplvis output also include a unique identifier for each generated GET and PUT. I expect this to be simple, and that it would bump the chplvis version from 1.2 to 1.3. Other events may benefit from this kind of identifier, but I do not plan on implementing that functionality in this initial effort.
I plan on appending this unique identifier to the end of the data line for the various kinds of gets and puts (normal, strided, nonblocking). The runtime comm layers and chplvis parser will need to be updated accordingly.
For the chplvis GUI, I would expect it to ignore the unique ID.
http://chapel.cray.com/docs/latest/modules/standard/ChapelIO.html
should be
http://chapel.cray.com/docs/latest/modules/internal/ChapelIO.html
P.S. Where can people find the .tex etc. source for spec.pdf?
[CHAPEL-172] created by MrMormonAll .future tests should be linked with issues.
When a new .future is added, there should be an automatic check that it is linked to an issue. When a .future is removed, there should be an automatic check that it was also de-linked with the issue, possibly with the suggestion that the issue can be closed if there are no more .futures associated with it.
Ideally this would happen in the pre-merge travis checks for any PR that adds or removes a .future.
Currently, record inheritance is syntactically allowed but can only be used to inherit fields.
We would rather have a 'mix-in' type strategy and disable record inheritance entirely.
The compiler currently uses the term 'lvalue' in its error messages. I think we should stop doing this.
Source Code:
From test/variables/constants/assignConstError.chpl added in PR #5067 :
const two = 2;
two = 3;
Compile command:
chpl assignConstError.chpl
Execution command:
N/A
chpl --version
: 1.14.0.7d05050$CHPL_HOME/util/printchplenv --anonymize
: N/Agcc --version
or clang --version
: N/Amodule list
: N/AAdd in
as an infix operator in a way similar to Python, supporting such cases:
if value in Range { }
if value in Domain { }
if value in Array { }
Maybe strings too (#11439):
if subString in String { }
This form of in
could potentially overload the D.contains()
method.
This grammar must be implemented carefully such that:
for i in D
is not parsed as:
for D.member(i)
Historically, all that was needed to add a new back-end compiler was to add a file:
make/compiler/Makefile.foo
and to set:
CHPL_TARGET_COMPILER=foo
Unfortunately, over time properties like compiler names and version flags have crept into the script util/chplenv/compiler_utils.py which introduces redundancy (e.g., the mapping from 'gnu' to 'gcc' now occurs both here and in the Makefile) and results in an additional file that needs to be modified to introduce a new compiler. Failure to do so results in a lot of warnings during compilation when taking the original approach.
This issue asks whether the redundancy between the Makefiles and the compiler_utils.py script can be removed through refactoring, with the goals of (1) making it easier to add new compilers and (2) reducing the number of modifications required to use a compiler of a different name (say if one's 'gcc' binary was named something else).
This was specifically motivated by attempts on my part to use a different gcc and/or modify how gcc's version was queried by changing Makefile.gnu only to find that it didn't affect things as I'd expected because util/chplenv/compiler_utils.py contained parallel logic that also needed changing.
CHPL_UNWIND=libunwind should be tested nightly as a prerequisite for turning it on-by-default in some configurations.
Note that util/test/prediff-for-stacktrace
is supposed to help with the various forms of mismatch between a .good file and a .chpl file, in case one has a stacktrace and the other does not.
Also note that stack traces might include call sites on linux64, but generally won't include call sites on Mac OS X. And also note that stack traces do currently include line numbers from internal modules.
It seems that config type statements that define multiple symbols don't work. See test/types/config/twoConfigsOneDecl.chpl for an example.
Source Code:
config type randType = uint(32),
char = int(8);
Compile command:
chpl twoConfigsOneDecl.chpl
Execution command:
N/A (gets internal error)
chpl --version
: 1.14.0.7f81ddd$CHPL_HOME/util/printchplenv --anonymize
: N/Agcc --version
or clang --version
: N/Amodule list
: N/AToday, chpldoc will only generate a single documentation comment for an enum. There is no way to create documentation for individual constants within the enum, which would be natural.
Additionally, if you try to document the first constant, it will overwrite the documentation for the
enum as a whole. See this future.
Source Code:
/* enum documentation */
enum Color {
/* first constant doc */
Red,
/* second constant doc */
Yellow
};
should perhaps yield something like this in plaintext mode (.rst mode not included):
enum Color
enum documentation Red first constant doc Yellow second constant doc
Compile command:
chpldoc docConstants.doc.chpl
This impacts all known versions of Chapel which support chpldoc and enums
At present, atomic variables can't be initialized:
var myAtomic: atomic int = 42;
The historical reason for this is that we haven't distinguished initialization from assignment (though that's changing!) and don't permit direct assignment to atomics. This idiom comes up reasonably frequently and is painful to code around, so it'd be really nice to enable it as soon as we can.
(Note that some have argued that we should also support direct assignment to / reads from atomics. I'm not advocating for that here, and would to keep that potential issue separate from this one. I think the initialization case is more of a no-brainer because it's obvious that there could be no race-y reads/writes in the initialization context).
It seems to me that defining a config named 'help' should take precedence in argument parsing over the built-in '--help' flag that all Chapel executables support. E.g., for the example below, I'd expect printUsage() to be invoked when running with --help, but instead the compiler-provided help message is printed.
Source Code:
This is taken from test/execflags/bradc/configHelp.chpl
:
config const help = false;
if help then printUsage();
proc printUsage() {
writeln("This is my custom usage message!");
exit(0);
}
Compile command:
chpl configHelp.chpl
Execution command:
./a.out --help
chpl --version
: 1.14.0$CHPL_HOME/util/printchplenv --anonymize
: N/Agcc --version
or clang --version
: N/Amodule list
: N/Amultilocale/lydia/checkNumTasks is currently racey, so I .notest'ed it with #5029
It basically does:
coforall loc in Locales do on loc {
num.add(here.runningTasks());
}
which is transformed into something like (some pseudocode + manually inlining functions):
endCount = endCountAlloc()
for loc in Locales {
...
// runtime call to startMovedTask will bump here.runningTasks
spawn_task_to_loc(loc, num.add(here.runningTasks())
}
/* snippet of inlined waitEndCount below: */
// Remove the task that will just be waiting/yielding in the following
// waitFor() from the running task count to let others do "real" work.
here.runningTaskCntSub(1);
endCount.waitFor(0);
// re-add the task that was waiting for others to finish
here.runningTaskCntAdd(1);
Technically there will be 2 tasks running on locale 0, but we lie and decrement the runningTasks counter since the spawning task will just we waiting/yielding and not doing any real work or using up many cycles if there are other tasks running.
However, this the original task on locale 0 and the task spawned to locale 0 doing the here.runningTasks() query can run in either order, so we have a race for the call to here.runningTasks() is racey.
Applying #5018 to coforall+on constructs will resolve this race, because we'll adjust the runningTasks counter before creating any tasks. Until that's in I'm just .notest'ing this test to avoid noise in nightly testing.
It'd be nice to be able to reuse the CHPL_COMM=none code that does the local strided puts and gets in other comm layers when the put/get is found to be local (or even some of the logic for the remote case).
Currently the gasnet comm layer calls gasnet_[gets|puts]_bulk in all cases. It's possible the library does the right thing here, but the comm statistics are not consistent with the way they are reported in the other functions.
As shown on the graph linked to below, on November 26-27 2016, 16-node HPCC FFT performance got significantly better in the ugni+qthreads configuration. Then on December 7th, the allocation of args on the stack rather than the heap set these timings back. Some of the performance was recovered on December 14th, but it looks like it did not fully recover.
This issue is meant to capture a desire to understand whether there are further improvements that could/should be made to recover the previous performance. In this morning's performance meeting, it was suggested that the right time to investigate this might be after Qthreads is using Chapel's memory management for its stacks.
'param' in Chapel is the keyword used to introduce a compile-time constant that can be computed and reasoned about by the compiler. Since the introduction of the keyword, there has been discomfort with the name, in part because it seems unintuitive and in part because of potential confusion with the notion of "function parameters" (which we very carefully try to always call "arguments" in Chapel to avoid confusion). I believe that the choice of 'param' was taken from Fortran and since the introduction of the keyword, we've struggled to come up with a worthy replacement, yet have failed to ever replacement. This issue is designed as a place to capture some of the best counter-proposals, as our ability to change it gets smaller each year.
My preferred replacement keyword for 'param' would be 'value' since I believe that term means very much what our notion of 'param' means today -- a value (in this case, named) which the compiler can reason about and compute on. It's also a term that makes good sense in both the declaration and argument-passing contexts:
value numDims = 3;
proc foo(value rank, x, y) { ... }
At times, we've discussed using 'val' instead of 'value' as shorter, unambiguous, and symmetric with 'var'. Concern has arisen at times that 'val' and 'var' may be too close to each other, introducing the potential for simple mistakes, not to mention confusion for non-native English speakers who may have trouble distinguishing the L and R sounds. To that end, here I'm proposing 'value' which has an equal number of characters as 'param' and is more visibly and audibly distinguishable from 'var'.
The main downside to this approach that I can see is the same downside as any change we might make: This is a feature that's been around for a long time and permeates our code. That said, I think that it remains somewhat regrettable and that if we don't change it now, we won't ever be able to.
The second downside gives me a bit more pause, and that's an argument that at times we've discussed wanting the compiler to not only have a notion of compile-time constants (what 'param' is today) but also compile-time variables that could be re-assigned over compilation. If we went down this path, it might suggest replacing 'param' with some sort of modifier on 'var' and 'const' like:
compiler const numDims = 3;
compiler var x = 0;
On the plus side, a prefix-based modifier like this has the benefit of more orthogonally capturing [compile-time vs. run-time] x [constant vs. variable]. But on the minus side, it seems less natural in the argument passing context. I.e.,
proc foo(compiler const rank, x, y) { ... }
To that end, I continue to think that 'value' as a sibling to 'const' and 'var' has value (ha ha) and that if/when we introduce a notion of a compile-time variable, we seek another term for it.
(I'll also note that some on the team might argue that compile-time vs. run-time are things that are regrettable to emphasize in a language more than necessary, and while I typically am not strongly in that camp, I do think that 'value' has an attractively more context-free meaning than 'compiler const' while also being a much more self-defining term than 'param').
The chpl
binary can infer the path of CHPL_HOME
if it is not set based on a variety of checks defined in compiler/driver.cpp
. printchplenv
does not have the equivalent checks, and therefore can lead to disagreement between the two, when CHPL_HOME
is being inferred.
printchplenv
should use the same rules that the compiler uses to infer CHPL_HOME
when it is not set.
unset CHPL_HOME
# CHPL_HOME is inferred using the rules defined in compiler/driver.cpp:
>chpl --print-chpl-home
/path/to/CHPL_HOME /path/to/chpl
# CHPL_HOME is empty, and confusingly labeled with a '*'
# which is intended to mean the default value has been overridden (which it hasn't in this case):
>printchplenv --anonymize
machine info: ...
CHPL_HOME: *
script location: /path/to/CHPL_HOME/util/
CHPL_TARGET_PLATFORM: darwin
...
I believe all of our current chplvis tests only use the program output for the .good file, and do not look at the actual output files used by the chplvis GUI. While it's not practical at this time to automate testing for the GUI itself, we could probably do some kind of verification for the output files.
One possibility could be to count the number of GETs and PUTs, like our existing CommDiagnostics tests.
The current method name for querying if a value is in a domain is domain.member()
.
E.g.
var D:domain(int);
D += 4;
assert(D.member(4));
assert(!D.member(3));
This method name bothers me for English/Programming Style reasons. member
is a noun and I really want the method name to be a verb in this case.
What should we do?
x in someDomain
into this query (#5034)Directly applying existing data-parallel tools to graph computation tasks can be cumbersome and inefficient. Hence, if there is a Graph library in Chapel that can work with both shared and distributed memory systems to provide both flexibility and performance to the user of Chapel.
Here are a few references:
1. https://amplab.cs.berkeley.edu/wp-content/uploads/2014/09/graphx.pdf
2. http://www.osl.iu.edu/publications/prints/2005/Gregor:POOSC:2005.pdf
This issue is for tracking the feature request for 'futures' in Chapel.
Some work towards adding futures was completed around 2013. See slides summarizing that work here:
http://chapel.cray.com/presentations/SC13/03-futures-imam.pdf
Here is an email thread about that work:
https://sourceforge.net/p/chapel/mailman/message/30815892/
Here is the branch containing that work:
http://svn.code.sf.net/p/chapel/code/branches/collaborations/futures
(and here is a link to some earlier versions of prototype code: https://sourceforge.net/p/chapel/code/21386/tree/branches/collaborations/futures/test/release/examples/futures/ )
Here is a discussion on chapel-users about retiring 'single' and replacing it with 'future'.
https://sourceforge.net/p/chapel/mailman/chapel-users/thread/alpine.LNX.2.00.1309131418050.327%40bradc-lnx.us.cray.com/#msg31409942
Here are the tasks that were outlined to complete 'futures' support in the language:
const f: future int = begin computeSomeNumber(); f.read();
const x = begin foo() + begin bar()
.({ a; b; c; })
which would enable examples like const x = future int = begin ({ var a = computeSomething();a; })
.Setting a union field from the union's constructor leaves the union undefined. Given a union containing an int, i, and a real, r, named IntOrReal, constructing it like:
var ir0 = new IntOrReal(i=1);
var ir1 = new IntOrReal(r=2.0);
both leave the field values unset.
This behavior is captured in the future:
test/classes/diten/constructUnion.future
chpl Version 1.14
Found a couple of issues with the callback code:
chpl_comm_do_callbacks
is called in some cases when the communication is local. For example in chpl_comm_put_nb
.chpl_comm_do_callbacks
called twice for a single call. Again in chpl_comm_do_callbacks
, if !remote_in_segment
, there is another call to chpl_comm_do_callbacks
from within chpl_comm_put
which is of the put
type. Not sure what type the callback should be in this case, but having both doesn't seems right.Stack and Queue, Priority Queue are very popular Data Structures and I feel that having these in a library is of paramount importance for any programming language.
For Chapel in specific, I believe we can even have concurrent versions of these data structures - thread-safe (or task-safe if you will) access to these data structures.
These data structures can be implemented using the List Data Structure in Chapel. But, for that we require accessing elements of a List using indices which is currently not supported by the List Module. It would be great to have these data structures.
[CHAPEL-198] created by rohanbadlanitry!
and halt()
from module code (#8452)try! this.lock();
-> this.lock(); defer this.unlock();
throws
(turned off to work around failures in --baseline)errhandling/parallel/forall-range-throws
errhandling/parallel/forall-throws-subtasks
errhandling/parallel/forall-throws
catch
clause lists go from specific to generalwriteThis()
a throwing method?
halt
raises an uncatchable error, which stores the stack traceThe goal of this effort is described in CHIP 8.
try { }
syntax (#4933, #4940)try
(#4975)throws
to function declaration AST, syntax (#5007, #5011)try!
support to try AST, syntax (#5038)throw
to function body AST, syntax (#5046)catch
block to AST, syntax (#5655)functionResolution
and before callDestructors
out
formals into functions that throw (#5240)try!
(#5240)try
(#5240)catch
(#5655)catch
(#5718)try
statements (#5718)catch
filters with parentheses (#5721)Error
module - rename it to SysError
(#5162)Error
type and msg
if the error causes a haltthrows
for generic function signatures (#6265)defer
construct MFvar x = mayThrowError(); doSomething(mayThrowError())
Error
classesinit
for Error
(#7988)defer
, deinit
(#8161)
on
statements MFAsyncErrors
or other construct exists to encapsulate multiple errorsbegin
cobegin
/coforall
forall
pragma
based workprototype module
throw nil
should result in a NilThrownError
(#7317)
nil
before assigning to error variable uncaught SystemError: No such file or directory in open with path "_test_cannotOpenMe.txt"
myopen() at unableToOpenFile.chpl:1 (error created)
main() at unableToOpenFile.chpl:5 (error not caught)
forall
are always reported in an ErrorGroup
sync
blocks throw, but some want more discussiontry
, throws
, try!
(#6566)
-memleaks
on error handling test directoryvar x = try! mayThrowError();
return try g();
a-la Swiftout error
or error=
)
Fields in records quote strings:
record R {
var s:string;
}
Before this proposed change,
writeln(new R("hi"))
would print out
(hi)
After the proposed change, it will print out
("hi")
I'd like to be able to write and then read an array/record/whatever and expect it to work, but there are problems if the object contains strings. I'm proposing a change to the I/O behavior to fix this problem.
How did we get here? If you do:
writeln("hello world\n");
you (naturally?) expect "hello world" to print without quotes:
hello world
but then if you were reading that input and you did this:
var s: string;
readln(s);
s
would contain only "hello", because the default string reading rule is to read one word only. Even if that were not the case, we wouldn't know when we reached the end of the string since it is not delimited in any way. (and making writeln print the quotes by default interferes with things like writeln("a = ", a) ).
When we get to a record, I don't think these choices are reasonable any more.
For example, if I have
record R { a: string, b: string }
writeln(new R("hello world", "second string"));
it will output
(a = hello world, b = second string)
and there's pretty much no way that the corresponding readln would work (what if the string contains commas or parenthesis?). I believe that the current code will only be able to read 1-word strings in records...
So, I'd like to propose the following change in I/O behavior:
writeln(string) and readln(string) continue to work the way they do
write/writeln/read/readln of any aggregate (record/class/array/tuple/etc) will switch the string formatting to double-quoted strings that would work in Chapel source code, so the second record example above would now output
(a = "hello world", b = "second string")
Many times programmer may want the flexibility of using postfix/prefix increment and decrement operators as flexibility.
If the compiler supports this, this can be done in a single instruction, increasing the performance.
Example: After fixing this, the following should be supportable:
var x: int = 5;
writeln(x++);
writeln(x--);
writeln(++x);
writeln(--x);
Output:
5
6
6
5
A sample mergeSort() code is also attached with this issue, illustrating why sometimes it may be easier for the programmer to use the postfix/prefix operators.
[CHAPEL-196] created by rohanbadlaniAn idiom that seems reasonably familiar to me (and came up recently for me in test/release/examples/benchmarks/shootout/fasta.chpl) is chunking a range up by a given chunk size in a block-cyclic manner.
For example, fasta contains:
for i in tid*chunkSize .. n-1 by numTasks*chunkSize {
const bytes = min(chunkSize, n-i);
where tid ranges from 0 to numTasks. The 'bytes' variable essentially asks "how large is my chunk". The loop itself is somewhat ugly looking. Could/should we come up with a better interface (as part of the RangeChunking module, e.g.) that would permit something more like:
for chunk in greatIteratorName(0..#n, chunkSize, tid, numTasks) {
const bytes = chunk.size;
where the yielded 'chunk' is a range? ('greatIteratorName' is a placeholder because I don't have a great name for this in mind right now).
I found a few issues with argument parsing:
--compopts
and --execopts
flags do not work when using =
(a space must be used). It appears as though the first character is truncated.--compo
or --execo
, with a space and an argument, results in a start_test usage: error:start_test: error: argument -compopts/--compopts: expected one argument
--compt="--fast --static"
does so, but not --compt="--fast"
(correctly returns usage message), --numx="2 x"
but not --numx=2
)Source Code:
N/A
Compile command:
N/A
Execution command:
N/A
chpl --version
:$CHPL_HOME/util/printchplenv --anonymize
:gcc --version
or clang --version
:module list
:The following program core dumps on master now:
class Parent {
proc runTask() {
}
}
class Child : Parent {
proc runTask() {
begin {
writeln("IN TASK");
}
}
}
proc run() {
sync {
var obj:Parent = new Child();
obj.runTask();
}
}
run();
The problem is that the virtual-method call obj.runTask is passed 1 argument:
((void(*)(Parent_chpl))chpl_vmtable[((INT64(5) *
_virtual_method_tmp__chpl) + INT64(1))])(call_tmp_chpl4);
but the invoked function expects 2 arguments:
static void runTask_chpl2(Child_chpl this_chpl,
chpl___EndCount_atomic_int64_int64_t _endCount_chpl)
I think we need to investigating adding the endCount to task-local storage since it is not generally practical to modify all virtual children.
Many of our failures in nightly testing are due to minor mistakes in adding new tests or modifying existing tests. At present, our CI runs only test a few things, just to make sure the world isn't completely broken. In the event that tests themselves are changed in a given PR, it would be nice if the CI would run those tests or perhaps the directories that contain them. While this would not provide airtight testing coverage (and realistically, little will), it would help by catching low-hanging fruit that did not show up in the developer's environment for some reason (e.g., developer changed a correctness test but forgot to check the corresponding performance test mode; developer failed to commit a new file that the test depended on; etc.)
Check the ANNOTATIONS.yaml file as part of the smoke test.
We often get the syntax wrong as seen in #4828, #3911, #4997, and many others.
Should be relatively simple to do: I think we just need to add a list of valid configs to annotate.py and then have a self check mode that reads through all the annotations and validates that they're formatted corrected.
Bonus points for going through the PR numbers and making sure that we annotated the dates correctly.
Note that this would require that the smoke test has PyYAML available, so to start we could just make this a developer only tool that perf team members would run.
Readability is improved in situations where some simple conditions need to be tested if the conditional operator is available.
Example:
proc max(a:int, b:int): int
Also, defining constants becomes possible with conditional statements because the value of a const all has to be assigned within that single statement.
Example:
const max_num:int = (a > b) ? a : b;
The conditional operator may even help the programmer avoid unnecessary null checks for object references.
var obj: MyClass = (old_obj == null) ? null : old_obj.attribute1;
Support for this in Chapel will help the programmers in writing better readable code and will also help write compact code.
[CHAPEL-197] created by rohanbadlaniThe List standard module includes a record list type. One would expect this type to include a destructor that frees the list nodes.
This is not currently the case because of the use of record list in (array, domain, distribution) memory management. However recent changes to array memory management might mean that adding a destructor and = overload is now possible for record list.
We would like to have a parallel iterator that traverses a file, one line at a time.
We already have a serial file.lines() iterator.
Some initial code towards this goal is here:
and it is related to the HDFSIterator module.
Note that QIO includes a mechanism to figure out what block size should be used when reading a file and which Locales are "closest" to a particular block. This functionality is available in file.getchunk and file.localesForRegion.
Chapel currently supports user-defined casts but not coercions. However, the way to declare a user-defined cast is undocumented and odd. Here is an example:
proc _cast(type toType, src) where isSomeType(toType) {
...
}
It would be nice to have a more user-facing feature here.
One proposal for the way to declare a user-defined cast is like this:
proc sourceType.cast(type toType) {
...
}
and then the way to indicate automatic coercion was possible between sourceType and toType is like so:
proc sourceType.coercible(type toType) param {
return true;
}
Some sample use-cases include:
updated with conclusions from discussion below:
TODO based upon discussion below
isCoercible
and cast
TODO based upon discussion below
ref
apply to ref
arguments?const ref
apply to in
or const ref
arguments?Future Work on LLVM Performance:
This task involves completing PR #3125 which attempted to improve the description of 'this' methods (supporting direct access/indexing) in the spec. While the PR received decent support on GitHub, in email @noakesmichael expressed reservations (see below) which I never addressed. This task is meant to capture this TODO in order to free up a long-open PR.
Mikes feedback was as follows:
Unfortunately I don’t see how Indexing Expressions are defined. For example it’s not a sub-production of expression.
Section 10.10 LValue Expressions suggests a relationship between “Indexing expresions” and parenthesized-expression but I don’t think that really intended. It’s not something that follows naturally from 10.0 and 10.4 and it wouldn’t support the use of square brackets.
20.3 speaks specifically about “a reference” which suggests that this() for array must return a ref. But of course that does not require all index() functions to return a ref.
So there isn’t much in the spec that makes it clear what the constraints on index().
The relationship between the index method and the syntax for call-expression is stated but, as we can tell, it’s a little oblique. I don’t think this can be readily solved by a few extra words in 15.6 and 16.7
An internal user (Chapel team member) reports an internal error in a system library called from the ugni comm layer, in a program that attempts to transfer 10**30 ints between top-level locales in a single operation. The root cause has not yet been diagnosed, but it wouldn't be a bit surprising if this larger transfer needed to be broken up into smaller chunks, for the network to be happy.
Source Code:
Use the existing Chapel ISx test with large enough parameters, on a Cray XC with CHPL_COMM=ugni and a craype-hugepages module loaded. The heap has to be on hugepages because if it is not, there is already code to do the transfer using bounce buffers, breaking it up as needed, and the bounce buffers are much smaller (megabytes) and already known to work.
It would be nice to be able to write Chapel programs that used clasess without ever needing to manually write delete
statements.
The existing automatic memory management for arrays, domains, and distributions offers some evidence that this idea is possible for arbitrary user-defined classes.
For inspiration, look to C++ shared_ptr and unique_ptr and the Box type in Rust. We should expect that users will be able to get bare pointers out of these types. (Note it's important to be able to call methods or functions working with the bare pointer type somehow - possibly by explicitly getting the bare pointer - because otherwise one wouldn't be able to do much with the type managing the memory).
Initial Design direction:
Open Design Questions:
Language Strategy Questions:
move
function in library akin to C++ std::moveChapel's slurm-srun launcher should be able to accurately calculate CHPL_LAUNCHER_CORES_PER_LOCALE internally (based on calls to Slurm sinfo), even if CHPL_LAUNCHER_CONSTRAINT is defined.
Ideally, similar Chapel launchers (slurm-gasnet-ibv) should be reviewed for the same problem.
At present, if CHPL_LAUNCHER_CONSTRAINT is used to select a subset of available compute nodes, but CHPL_LAUNCHER_CORES_PER_LOCALE is undefined, the slurm-srun launcher's internal calculation will likely be inaccurate because sinfo does not accept Slurm constraints as a command line arguments.
The desired code change would rewrite lines 129-151 of chapel/runtime/src/launch/slurm-srun/launch-slurm-srun.c:
113 // set just use sinfo to get the number of cpus.
114 static int getCoresPerLocale(void) {
115 int numCores = -1;
116 const int buflen = 1024;
117 char buf[buflen];
118 char partition_arg[128];
119 char* argv[8];
120 char* numCoresString = getenv("CHPL_LAUNCHER_CORES_PER_LOCALE");
121
122 if (numCoresString) {
123 numCores = atoi(numCoresString);
124 if (numCores > 0)
125 return numCores;
126 chpl_warning("CHPL_LAUNCHER_CORES_PER_LOCALE must be > 0.", 0, 0);
127 }
128
129 argv[0] = (char *) "sinfo"; // use sinfo to get num cpus
130 argv[1] = (char *) "--exact"; // get exact otherwise you get 16+, etc
131 argv[2] = (char *) "--format=%c"; // format to get num cpu per node (%c)
132 argv[3] = (char *) "--sort=+=#c"; // sort by num cpu (lower to higher)
133 argv[4] = (char *) "--noheader"; // don't show header (hide "CPU" header)
134 argv[5] = (char *) "--responding"; // only care about online nodes
135 argv[6] = NULL;
136 // Set the partition if it was specified
137 if (partition) {
138 sprintf(partition_arg, "--partition=%s", partition);
139 argv[6] = partition_arg;
140 argv[7] = NULL;
141 }
142
143 memset(buf, 0, buflen);
144 if (chpl_run_utility1K("sinfo", argv, buf, buflen) <= 0)
145 chpl_error("Error trying to determine number of cores per node", 0, 0);
146
147 if (sscanf(buf, "%d", &numCores) != 1)
148 chpl_error("unable to determine number of cores per locale; "
149 "please set CHPL_LAUNCHER_CORES_PER_LOCALE", 0, 0);
150
151 return numCores;
152 }
Replace the existing call to sinfo, above, with an sinfo call like this:
sinfo --exact --format="%c %f" --sort="+=#c" --noheader --responding
Instead of just reading the first line from the sinfo output, as above, the code change should
^[A-Za-z][A-Za-z0-9_]*$
(assuming such tokens are all Slurm Feature names), and discards anything else.Atomic operations represent a memory fence and should disable the remoteValueForwarding optimization, but do not currently do so.
An example of this incorrect behavior has been captured in the following future:
test/optimizations/remoteValueForwarding/bharshbarg/atomicsDisableRVF.chpl
This program uses the CommDiagnostics
module to track the number of GETs. By running this program with multiple locales you can see that there are zero gets, where there should be at least one.
CHPL_COMM != none
Present in v1.14, and likely many previous versions.
Non-inlined iterators leak memory if the iteration is broken out of.
See
test/types/records/ferguson/leak-futures/iterate-*.chpl
Inlined and non-inlined iterators generate pretty different code so the potential solutions are:
use a different approach for freeing captured iterator variables in the inlined vs non-inlined iterator cases. Here, for non-inlined iterators, we could default-initialize the iterator class fields and use the = overload to updated them and finally call destructors on them in freeIterator.
use a different strategy that works with the advance() function usually freeing the memory. For example, we could generate several code blocks in freeIterators that frees only the variables that we know were created by that point in the iterator's progress (based on the iterator variable tracking the iteration).
I think this leak exists for inlined iterators, and so (2) is probably the way to solve it.
The concept here is to have the mails from nightly cron jobs report failures relative to other configurations in order to reduce the overall amount of redundant mail. For example, if someone checks in an inherently broken test that is run in every configuration, we currently receive a mail per configuration reporting that information which results in more noise than value. Instead, it would be nice to establish a shallow hierarchy of test configurations such that if a test broke in a parent configuration, it would not be re-reported in the child configuration. For instance, say that a developer checks in an inherently broken test. Since it would fail on linux64 (which I propose should serve as our "root" configuration), we would not need to re-report it on linux32 or darwin or --no-local or ... Similarly, a failure on --no-local testing should not be re-reported on gasnet, and so on.
The main challenge here would likely be the question of scheduling the test configurations such that a late-arriving configuration didn't hold up the summaries of earlier configurations unnecessarily. For instance, I believe that linux64 testing is currently completing quite late in the day which would be a bad property of the root configuration.
Some of the version-dependent documentation contains version-independent information:
For example, with the switch over to GitHub issues, our old Bugs pages now contains outdated instructions, which will likely cause confusion for users who stumble upon the outdated pages:
There will likely be other instances of this problem in the future. Here are some ideas on how to prevent this in the future:
.. warning:
in the header, containing a URL to the 'master' version.A hash value (such as SHA-1) is normally written in hexadecimal and can be stored in several integers. However, it is hard to write a call to `readf' to read such an long hexadecimal value into a group of integers. It would help if there was a way to specify, in readf, the maximum field width when reading an integer.
See test/studies/dedup/dedup-distributed-ints.chpl
Here is an Issue for discussing what labels to add/remove.
See current labels here: https://github.com/chapel-lang/chapel/labels
Update:
Now that labels have reached steady-state, we have closed this mega-issue.
To propose a new label, please submit a new issue.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.