openwateranalytics/epanet

The Water Distribution System Hydraulic and Water Quality Analysis Toolkit

epanet's Issues

ENsetnodevalue for Tank using EN_MINLEVEL/MAXLEVEL returns error if Tank has a curve

expected behavior is to be able to set the min/max level for a tank, even if it has a volume curve

memory leak

at lines 3114-3117 of epanet.c function allocdata() allocates memory for node coordinates

      if (Coordflag == TRUE)
      {
        Coord   = (Scoord *) calloc(MaxNodes+1,  sizeof(Scoord));
      }

while freedata() does not deallocate.

We need to add something like:

      if (Coordflag == TRUE)  free(Coord   );

testing protocol

need one. should do a comparison of hydraulic and quality results with canonical version. probably need a binary parser to do this.

Patterns with arbitrary time step

Maybe it would be easier to incorporate Scada data to EPANET if it could be possible to use patterns without a constant time step:
The pattern could be made of 2 fields - relative hour and a value.
The first row would have the relative time of 0 (or below).
e.g.
0 200.8
0.27 220.4
0r
-0.5 190.4
0.27 220.4

At the beginning (while loading the network or when changing the time step) these time patterns would be "translated" to "standard patterns" with these 2 options:

1. The values of the "standard patterns" computed patterns would be by the last value before each time step.
   2 The values of the "standard patterns" patterns would be a linear interpolation between the values before and after the time step.

This is a rather time consuming computation so may it would be helpful to save the computed patterns in the Inp file for reuse.

Support for model element string names

We should have them. need space in the input file somewhere, plus API to get/set them.

Small issues that don't cause problems but should be cleaned up (IMHO)

See if you agree with the following:

1. Issue in adjustdata() function (input1.c) line 305

304 /* Use default pattern if none assigned to a demand */
305 for (i=1; i<=Nnodes; i++)

Since you cannot place a demand on a tank/reservoir, there is no need to loop clear through all of the nodes. Just go through to Njuncs.

1. Issue in valvecheck() function (input3.c) line 1782

1781 /* Examine each existing valve */
1782 for (k=1; k<=Nvalves; k++)

Should read "for (k=1; k<Nvalves; k++)" so the function does not compare the current valve to itself. Probably does not ever cause an issue, but it would be good to clean it up

Valve statuses (and how they're set)

This may be larger than TCVs but it's easier to limit comments to that case, so I'll do that.

Say you have a TCV that is, say, 25% open, and you give it a loss coefficient to reflect that of, say, 200. That's in the valve section.

Now you give that same valve an initial status of OPEN or CLOSED in the [STATUS] data section. That overrides the valve setting in the [VALVES] section. I have found this to be confusing. It seems more secure to have one and only one place where something important like a valve setting is specified.

Maybe the problem is more basic and related to how things are parameterized. Does it make sense for something like a TCV with a relationship between fraction open and loss coefficient, to be assigned a setting of OPEN or CLOSED?

Bottom line is that valve settings are really important and it should be intuitive how they are parameterized and how their behavior is specified.

Computations of patterns

While using external data patterns it is helpful to have computed patterns.
e.g.
Head at a reservoir is the level plus the elevation at the boundary of the model.
Some demand is the aggregation of many consumers.
Flow of a pumping station is the sum of pump flows.
An unmeasured flow can be derived from the "On-Off" value of the valve or pump.

This could be implemented by a "computed pattern" which is defined as an arithmetic expression of other patterns.
e.g.

1. The pattern "TotDemand" would be:
   Dem1+Dem2+Dem3 Which are also patterns
2. The pattern "BoundaryHead" would be:
   MyLevel+54 where MyLevel is also a pattern.
3. The pattern Pump1Flow would be
   (PumpOnOff = 1)*200 + where PumpOnOff is also a pattern.
   The logical expression "PumpOnOff = 1" returns 1 if PumpOnOff = 1 and 0 otherwise

Thinking through project directory structure

Now that we are gathering tests and python wrappers and extra utilities for reading/manipulating files, we should develop some strategy for keeping it all organized. This is a large enough project that perhaps the hierarchy should be an extra level deeper to accommodate the different functional units.

Any suggestions or thoughts on the matter?

For the sake of discussion, here's a quick straw-man. Please take liberties and let's get some ideas going.

/
|--- README.md
|--- AUTHORS
|--- doc/
|      |-- doxyfile
|      |-- *.dox
|
|--- build/
|      |-- CMakeLists.txt
|
|--- data/
|      |-- networks/ 
|            |-- *.inp
|
|--- src/
|      |-- epanet2.h
|      |-- cmd/      <-- command-line application code
|      |-- engine/   <-- hyd/wq analysis
|      |-- binfile/  <-- binary file i/o
|      |-- inpfile/  <-- inp file parse/save
|
|--- bindings/
|      |-- python/   <-- python wrappers
|      |-- swift/    <-- swift wrapper classes
|      |-- vb6/      <-- visual basic wrappers
|      |-- matlab    <-- matlab wrapper
|      |-- R/        <-- R stats wrapper
|
|--- tests/
      |-- / directories for test scripts: *.py *.sh
      |-- .sh scripts for running tests

standardize build files

a question: how many (and which) build environments should we support within this project? so far, we have various build files for MSVS, CMake, MingW, Xcode, and Make. should this be simplified for easier maintenance? or are there enough interested parties that can maintain all these configurations? I understand that CMake was invented to try to handle this situation exactly...

Possible memory issue with ENgetqualinfo and ENgetcurve

Could someone with more experience on how memory is handled in EPANET, check/confirm whether we have a possible memory issue with these functions: ENgetqualinfo and ENgetcurve.

I am using the DLLs in Matlab/Windows 10 (created using the latest dev-2.1 commit 997c883)

The code (attached) sometimes works ok and returns 0, along with the correct responses for both functions. However, when it does not work, Matlab crashes. I suspect a memory-related issue, but I could be wrong.

Attached is a minimum example showing the problem I am facing. Included are the compiled 64-bit DLLs which I produced using the /Build/WinSDK/Makefile.bat command.

Minimum example in Matlab: test.zip

Thanks

parse all documented inp file sections

... to achieve parity with the GUI parser

Position valves

I think it's pretty common to have valves that have their degree of opening remotely controlled. This seems to be hard to do in Epanet.

This calls for a TCV where the minor loss coefficient can change. You can do that through controlling the valve setting, but that requires you to deal directly with the minor loss which is a hassle and is going to be error prone and hard to maintain.

A first requirement seems like the ability to associate the minor loss versus percent open curve with a valve. Note the GPV associates a flow versus head loss curve which itself would be a function of the valve degree of opening, so that's not really an option.

then it would be useful to have the setting be the percent opening, which is easy to interpret and how the operation of these valves are described.

Finally, the percent opening could be used in a control/rule action clause.

Duplicate entries in the AUTHORS file

There are duplicate entries in the Authors file. For example:
James Uber
jamesuber
and:
eldemet
Demetrios Eliades
Same emails but different user\names.

Also, I think Lew's name should be added on the top of the list.

roll back progressive hyd/wq steps mode

since this feature's being bumped from v2.1 - feature is tracked in #12

Ability to insert nodes or links through EPANET Tookit/API

The toolkit does not currently support programmatically adding nodes or links after the initial data is read from .inp text file. This is left to the realm of GUI or other outside tools that manipulate the data before the toolkit consumes it.

Is there any interest in adding this feature to the toolkit? If so, I can add more comments about some ideas I have to tweak the current code.

Calculation of tank flows when tank is at maximum (and likely minimum) level may not make sense

Under "certain uses" of the toolkit, it has been possible to calculate adjacent tank flows when the tank is at its min/max level, that do not make sense. I've seen results where the tank is at the minimum level, and we are computing large outflows. In this case Epanet should be closing that link and thus the flow should at least be "small." Similarly I've seen tanks at their maximum level where it is actually closed, and the flows are "small", but they are small in a direction out of the tank. The tank update then sets the new tank level slightly below the maximum, resulting in a time step that is cut short. The behavior in this case should be to identify that a tank is really "closed" when updating its level, and just not update it.

"Progressive" hydraulic and water quality steps

This feature enables API users to use runH and runQ in a stepwise fashion. It is mostly implemented as of 434d78c with modifications and fixes since then.

The feature should be tested and documented before rolling into a release, so I'm targeting this for v2.1

contributor code of conduct

In an effort to establish a good and healthy environment for collaboration, I'd like to suggest that the project adopt the well-written Contributor code of conduct available at http://contributor-covenant.org and reproduced below. In the best of circumstances, we would just say that we agree with the message and we post the file into the root directory. In non-ideal circumstances, we have a document to point to which outlines a policy for handling unfortunate situations.

Feedback on this issue is encouraged!

---

Contributor Code of Conduct

As contributors and maintainers of this project, and in the interest of
fostering an open and welcoming community, we pledge to respect all people who
contribute through reporting issues, posting feature requests, updating
documentation, submitting pull requests or patches, and other activities.

We are committed to making participation in this project a harassment-free
experience for everyone, regardless of level of experience, gender, gender
identity and expression, sexual orientation, disability, personal appearance,
body size, race, ethnicity, age, religion, or nationality.

Examples of unacceptable behavior by participants include:

• The use of sexualized language or imagery
• Personal attacks
• Trolling or insulting/derogatory comments
• Public or private harassment
• Publishing other's private information, such as physical or electronic
  addresses, without explicit permission
• Other unethical or unprofessional conduct

Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.

By adopting this Code of Conduct, project maintainers commit themselves to
fairly and consistently applying these principles to every aspect of managing
this project. Project maintainers who do not follow or enforce the Code of
Conduct may be permanently removed from the project team.

This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community.

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting a project maintainer at [INSERT EMAIL ADDRESS]. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. Maintainers are
obligated to maintain confidentiality with regard to the reporter of an
incident.

This Code of Conduct is adapted from the Contributor Covenant,
version 1.3.0, available at
http://contributor-covenant.org/version/1/3/0/

Parse the comments after element definitions, save as "description" field in network data

This will be a short-term extension of the inp parser to replicate what the GUI does.

Windows dll and exe are crashing when leading [ is missing on Title section

This does not happen for other sections, only the first one.
Might be related to the fact the the previous section is null

general key-value capability for element relational data

input integer #define-s should be changed to enum-s

... this is mostly a style enhancement which would ease auto-generated documentation by grouping related parameters within a defined enum and provide better support for code-completion in an IDE.

for example:

#define EN_CVPIPE       0   
#define EN_PIPE         1    
#define EN_PUMP         2
#define EN_PRV          3
#define EN_PSV          4
#define EN_PBV          5
#define EN_FCV          6
#define EN_TCV          7
#define EN_GPV          8

should be converted to:

typedef enum {
  EN_CVPIPE       = 0,    
  EN_PIPE         = 1,    
  EN_PUMP         = 2,
  EN_PRV          = 3,
  EN_PSV          = 4,
  EN_PBV          = 5,
  EN_FCV          = 6,
  EN_TCV          = 7,
  EN_GPV          = 8
} EN_LinkType;

evaluate / tweak Python wrapper for binary output api

@bemcdonnell did a fantastic thing by submitting his Python wrapper for @michaeltryby's binary file parsing API -- but it looks like it needs some cleanup (some paths are hard-coded for instance) before we can use it within a testing script.

Does anybody use python wrapper-type code on a daily basis, or have a deep familiarity? I really am a novice Python-er, but I think I've seen some python come from @mauriziocingi @accumodel @eldemet -- others?

ENgetheadcurve should be changed to ENgetheadcurveindex

I think that ENgetheadcurve should be changed to ENgetheadcurveindex. Now the function returns the curve ID and not the index. Users needs to know the head curve if they want to read or write it's values. In both cases they need the index of curve, not the ID.

Also, this is how other functions work. As this being a new function we don't have backward compatibility issues yet.

See function here

ENgettimeparam function (epanet.c)

ENgettimeparam function (line 1087 of epanet.c) doesn't recognize EN_RULESTEP code and
always returns 0

A line like
case EN_RULESTEP: *value = Rulestep; break;
has to be added

Also code=12 (EN_HALTFLAG) in the interval EN_DURATION - EN_NEXTEVENT is not managed by ENgettimeparam

Report files

This is to remind us about the issue when we will do some work on documentation.

There are two optional report files:
1 - the report file named in the ENopen function - this file will report status, warning, and error messages and nodes\links results (if they are not turned off) .
2 - the report file named in the REPORT section (or by the ENsetreport function) - this file is for nodes\links results only.

Automating EPANET Compile on Windows --- Help wanted for testing

If you are a Windows user, and you have a few minutes, I would like your assistance in testing the automated script we prepared with @eladsal for compiling EPANET for Windows.

No programming skills are needed, you just need to run a .bat script.

You should have installed the Microsoft Windows SDK 7.1 (You can check if you have this if you have this folder C:\Program Files\Microsoft SDKs\Windows\v7.1\ or if you have a Microsoft Windows SDK 7.1 folder in your Programs/Apps list.

• Download https://github.com/OpenWaterAnalytics/EPANET/archive/dev-2.1-win-makefile.zip
• Unzip anywhere
• Go to \build\WinSDK\
• Execute Makefile.bat

Executing Makefile.bat will call the Microsoft Windows SDK 7.1 Command Line, and execute the compilation script of the DLL and the EXE.
Then it will copy the .dll, .exe and .h files in folders \64bit, \32bit and \32bitVB.
The last folder builds the DLL by using the .def file (included in the \WinSDK folder), which seems to necessary in VB applications.

In case something does not work as it should, please let us know of your Windows configuration (Windows version, Intel/AMD, Windows SDK installation problems) as well as any error messages which could appear.

Thanks, and wishes for a great new year!

OSS site contribution and governance discussion

please see the gh-pages site site for a draft contribution and governance documents. feel free to comment here, or fork and make a pull request if you have particular language to contribute.

'TmpDir' undeclared

function getTmpName (line 1710 of epanet.c) refers to TmpDir.
TmpDir was once declared in vars.h (line 30) but now is part of struct OW_Project
(line 603 of types.h)

epanet.c:1710:23: error: 'TmpDir' undeclared (first use in this function)

function getTmpName needs some rewriting (i.e. add pointer to OW_Project in calls)

New functions documentation

Apart from the changes documentation I think we should document the new files added to the DLL. There are about 10 of them and I think that we could start with Wiki pages right here on GitHub.

segmentation fault - unprotected access to unallocated en_defaultModel

calling ENepanet (from owa_obj branch) with a non existing inpfile name gives a segmentation fault error

Existing EPANET GUI help is broken on Windows 10

I'm hearing some reports that the .chm help file is broken in Win10. Can anyone confirm / share a screenshot or error message?

This issue might elevate our document generation urgency if true. It would be nice to support these folks.

Coordflag

In epanet.c line 213 Coordflag is set TRUE, and nodal coordinates structure is always allocated and section [COOR loaded even if ENepanet() is called,
If there's no need to load coordinates we have to

• change line 213 to Coordflag = FALSE;
• rebuild library

Maybe It would be better if

• remove epanet.c line 213
• add a function that can switch Coordflag

int DLLEXPORT ENsetcoordflag(char flag)
/*------------------------------------------------------------------------
**    Input: flag
**   Output: none
**  Returns: error code (always 0)
**  Purpose: set Coordflag to FALSE = do not read [COORDINATE] section
**                         or TRUE  = do read [COORDINATE] section
**           default status is FALSE, has effect if called before ENopen()
**-------------------------------------------------------------------------
*/
{
  if (flag) Coordflag=TRUE;
  else Coordflag=FALSE;
  return (0);
}

Test if coordinates are loaded becomes if (Coord!=NULL) instead of if (Coordflag)

In this way code behaves as usual (no coord loading) as default and calling ENsetcoordflag(1) before ENopen coord get loaded.

Another improvement could be change ENsetcoord and ENgetcoord to ENsetnodecoord and ENgetnodecoord

Version number is hard coded in output messages

Need to be handled before release.
https://github.com/OpenWaterAnalytics/EPANET/blob/master/src/text.h#L335

document changes in 2.1

need to document the bugs addressed:

• tank diameter bug ac272cf
• Lew's bug fixes (need to enumerate)
• Rulestep getter e732508
• Pump index getter f3052ac
• Link pattern getter 3b5feb7
• Time step truncation 242c03f
• get node value break/return bug: 5abc387
• getcurve function 566eb53
• getstatistic f8e843d
• save output memory leak: b09492b
• get head curve, pump type: d1b0212
• minor loss for pipes: 017e24b
• ENgetnumdemands
• ENgetbasedemand
• ENgetdemandpattern
• ENsetbasedemand
• ENsetqualtype
• ENgetqualinfo
• ENgetcoord (not sure it's working)
• ENgetcurve (not supported for VBA yet)
• New hash table implementation: caa7944
• Ability to build toolkit for float or double precision

ENgetcurve and ENgetpattern (and family) should use similar patterns

@eladsal can fill in some details here. apparently returning a pointer to an array is a no-no in VB.

binary report file dot-extension

wonder if we should have a poll here - there are various dot extensions that we have used for the binary output file... this would be useful to standardize so we can write scripts that know what files to look for. some ones used previously and some suggestions:

• .out "output"
• .bin "binary"
• .enb "epanet binary"
• .ebr "epanet binary report"
• .erd "epanet report database"

anyone have strong opinions? I would prefer to use some extension that would distinguish itself from the other binary file (used for hydraulics, with uneven time steps).

ENgetcoord function broken

Discussion welcome: should the library load coordinates? is that the job of the engine? or should we break of the .inp file parser into a separate utility?

Should we keep the sample networks?

Not sure what are the [sample networks[(https://github.com/OpenWaterAnalytics/EPANET/tree/dev-2.1/example-networks). What are they supposed to show? Should we keep them on the branch?

Error

Hi,
I am using the latest version of Epanet-Matlab Class Master and Matlab R2015a. When i am trying to load the d.getNodeCoordinates command i get this error:

Error using fopen
File identifier must be an integer-valued scalar of type
double.

Error in epanet>readAllFile (line 8194)
fid = fopen(inpname, 'rt');%or msxname

Error in epanet/readInpFile (line 5683)
[info,tline,allines] =
readAllFile(obj.Bintempfile);

Error in epanet/getNodeCoordinates (line 6041)
[~,info] = obj.readInpFile;

I am not sure but a relevant error is generated while i am trying to use the d.plot commands.Which is the following:

Error using fopen
File identifier must be an integer-valued scalar of type
double.

Error in epanet>readAllFile (line 8194)
fid = fopen(inpname, 'rt');%or msxname

Error in epanet/readInpFile (line 5683)
[info,tline,allines] =
readAllFile(obj.Bintempfile);

Error in epanet/getNodeCoordinates (line 6041)
[~,info] = obj.readInpFile;

Error in epanet>ENplot (line 7922)
v.nodecoords=obj.getNodeCoordinates;

Error in epanet/plot (line 721)
[value] = ENplot(obj,'bin',0,varargin{:});

Thank you.

build files for 2.1

need:

• CMake file
• Xcode
• MSVS solution file ?
• windows compilation batch script ?

CONTROLS and RULES are redundant

I can't seem to come up with a case where a "control" cannot be expressed as an equivalent "rule". Besides being curious about how these two concepts evolved separately, I wonder if it would be better to cut out the "controls" functions and just have the input file parser interpret the [CONTROLS] section of the input file into "rules".

"double free or corruption" error

In case of [TIMES] Statistic not equal NONE (see page 174 of Epanet2 users manual) ENclose() function tries to close TmpOutFile again giving a "double free or corruption" error.

You can reproduce error modifing example-networks/Net1.inp at line 124
from Statistic None
to Statistic AVERAGED and running epanet.

Solved adding
TmpOutFile=NULL; at line 506 in src/output.c

(included in pull request #39 )

epanet binary output file dot-extension

VB.NET (and other languages) support and documentation

I'd like to get some discussion about VB.NET support - which apparently is a popular way to access the toolkit. Is there conceptually any difference between what we do with regards to VB.NET declarations and other interface wrappers like Python and MatLab?

Because if the difference is thin, I would propose detangling the project a bit and gathering these language bindings into a subunit of this repository (or move to another repo) to create a better logical division between these concerns. From earlier discussions, it seems like VB.NET users won't really be comfortable with all the C language specifics --- same for Python users etc...

A related question would be if the C API reference docs to be generated for the core library will be sufficient for users of language bindings. And I suppose the answer depends on the specifics; if the language binding layer is really just a straight-through function-for-function translation unit, then the added documentation needs will be minimal. However for something like an object-oriented abstraction library in Python, there's not necessarily a one-to-one mapping and that might need its own doc set.

Thoughts and comments?

support for ISO-8601 time format

support for date/time would allow multi-day simulations with time-based controls that do not require you to use decimal hours. this should apply to any rule or setting that understands CLOCKTIME:

• controls
• rules
• times (start clocktime)

initial doxygen markup, doc set generation

tracking doxygen markup in doxy-trial branch. I will volunteer for this issue - it really just involves copy/paste/format work.

related to #14 / follow-on to #24

segmentation fault (Linux only)

problem

Multiple calls of ENepanet (but also ENopen ENrun ENclose sequences) generate segmentation fault (Linux only).

example

pathname has to be changed, it must refer to an existing (correct) inp file

#include <stdio.h>
#include "epanet2.h"
int   main(){
    int errcode;
    errcode = ENepanet("/home/maurizio/EPANET/example-networks/Net1.inp","temp.txt","",NULL);
    printf("\nfirst run errcode = %d\n", errcode);
    errcode = ENepanet("/home/maurizio/EPANET/example-networks/Net2.inp","temp.txt","",NULL);
    printf("second run errcode = %d\n", errcode);
    return(0);
}

output:

maurizio@maurizio-VirtualBox:~/EPANET/build/Linux$ ./test1   

first run errcode = 0
Segmentation fault (core dumped)

Solution

fixed in #63
new output:

maurizio@maurizio-VirtualBox:~/EPANET/build/Linux$ ./test1   

first run errcode = 0
second run errcode = 0

CODEVERSION and VERSION are redundant

... or at least the difference is not clear and documented.