equinor / flownet Goto Github PK
View Code? Open in Web Editor NEWFlowNet - Data-Driven Reservoir Predictions
License: GNU General Public License v3.0
FlowNet - Data-Driven Reservoir Predictions
License: GNU General Public License v3.0
This is currently blocked by the following upstream dependencies:
libecl
libres
opm
Production wells operating under an economic constraint may be shut in at e.g. high WCT. In the Flownet schedule file production wells are currently not explicitly shut in but simply disappear from the history schedule (i.e. no new WCONHIST entries appear after the shut-in date). This results in the well remaining open with the last assigned operating target. Shutting in of the well could be achieved by adding e.g. the schedule entry
WCONHIST
'BR-P-18' 'SHUT' 'RESV' /
/
at the correct shut-in time. It was verified that this has the desired effect.
It is not clear if the well status can be extracted from OPM Flow output as read by ERT. An alternative may be to implement a rule that says, for example, that if liquid rate and BHP are both 0 this means that the well is shut-in.
Author: @wouterjdb
When I simulate FlowNet models with Eclipse and I visualize them with ResInsight I'm getting the following:
Running the same model with Flow results in the following vizualization in ResInsight:
NB. The DX parameter, which is calculated by the simulator, is completely off in the Eclipse case.
The script 'ExternalErtScript' caused an error while running:
usage: Save iteration parameters to a file. [-h]
Simulations completed.
config runpath eclbase yamlobs
start end quantity metrics outfile
Save iteration parameters to a file.: error: the following arguments are required: outfile
The script 'ExternalErtScript' caused an error while running:
usage: Save iteration parameters to a file. [-h]
config runpath eclbase yamlobs
start end quantity metrics outfile
Save iteration parameters to a file.: error: the following arguments are required: outfile
Author: @wouterjdb
We aim at using Flow - but we still call the simulator "Eclipse" in many places.
The config parser has some hard coded lines,
flownet/src/flownet/config_parser/_config_parser.py
Lines 415 to 417 in a7dc9c0
which is not very user friendly as this will not work for any other case than Brugge. These defaults should be somewhat more general.
When using FlowNet in prediction mode, I observed that the parameters of the realizations of the ensemble (at least the grid parameters - porosity, permeability, etc.) show the same values.
I think something is going wrong when loading the parameters parquet file and writing the (grid) parameter include files. Or maybe something is wrong in history matching mode when saving the parameters into the parquet file.
In the "global" aquifer scheme, aquifer nodes visually lie on top of one another but were for some historical reason not connected with NNC's. The code below is responsible for that, but seems to also remove NNC's for aquifers in the "individual" scheme:
flownet/src/flownet/network_model/_network_model.py
Lines 145 to 151 in e73fd20
Could this potentially lead to aquifers that are not directly connected to a well to not be connected to the grid at all?
Original author: @wouterjdb
We should include the accuracy metric in ERT using workflows during the AHM.
We should also considering to include the heatmap and/or other useful analytical plots. @edubarrosTNO, which ones were most useful?
Please have a look at to see how you can add a workflow in the code base.
...automatically (on merge to master
?).
Moved from the previous repository (author @wouterjdb ):
The FlowNet code bases currently assumes metric units at multiple locations:
It is not common for everyone to use metric units - it would be good to allow for field units as well.
Running the norne parameters file will crash on the following line:
flownet/examples/norne_parameters.yml
Line 46 in e73fd20
This should be:
loguniform: true
I'm applying Flownet to the Egg model. You can see the results of the ensemble for 6th iteration. in the following figurre.
You can notice that the total oil rate and water flow rate never go closer to the history data. The ensembles are somehow stagnant. In this example, I'm running 40 ensembles and 100 additional nodes, however, the results are similar if I increase the number of ensembles, the number of nodes the range in the parameters.
This figures shows the network generated by Flownet for the Egg reservoir model.
Aditional to the connections Injector - Producer, you will connections producer-producer and also connections Injector - Injectors. I understand these connections are generated by default. However, there are connections that should no be there (in my opinion). There is a connection, for example, the one between injector 2 and injector 6 that i don't understand. Both wells are far away from each other.
It would be interesting to include in the configuration file an option to influence the creation of the network in flownet.
Moved from the previous repository (author @wouterjdb ):
It's hard to guess from the start how many iterations you need. After we a metric describing the training and validation phases, we should be able to create an automatic stopping algorithm:
Continue the history matching iterations as long as both the training and validation metrics show improvements. The moment that the validation metric performs worse than in the previous iterations (over fitting) one should stop.
While other producing wells are correctly shutting in at high WCT, this particular well (the final production well) does not shutin in in FLOWNET.
Original author: @olelod
Currently the phases of the model are extracted from the relative permeability given.
if self._swof and self._sgof:
str_runspec_section = "OIL\nWATER\nGAS\n"
elif self._swof:
str_runspec_section = "OIL\nWATER\n"
elif self._sgof:
str_runspec_section = "OIL\nGAS\n"
return {
"RUNSPEC": str_runspec_section,
"REGIONS": write_grdecl_file(merged_df_satnum, "SATNUM", int_type=True),
"PROPS": str_props_section,
}
Should we reverse this and make the user provide which phases are present in the config yaml? This would make it easier to check if the input needed is provided (relperm, contacts etc.).
I'm installing flownet in a new Ubuntu 20.04.
There a problem in build_enviroment.sh
, I got the following error message
Collecting pip
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
ERROR: Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)')))
Every line with
pip install <package>
has to be modified by
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <package>
This will solves the first error but there is a new problem in the pip
calls inside ERT
Finished installing libres
Collecting git+https://github.com/equinor/ert@c74e1e6
Cloning https://github.com/equinor/ert (to revision c74e1e6) to /tmp/pip-req-build-k17vq1n3
Running command git clone -q https://github.com/equinor/ert /tmp/pip-req-build-k17vq1n3
WARNING: Did not find branch or tag 'c74e1e6', assuming revision or ref.
Running command git checkout -q c74e1e6
ERROR: Command errored out with exit status 1:
command: /home/manuel/repos/projects/Flownet/flownet/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-k17vq1n3/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-k17vq1n3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-2jm5uwsd
cwd: /tmp/pip-req-build-k17vq1n3/
Complete output (113 lines):
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
ERROR: Exception:
Traceback (most recent call last):
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connection.py", line 352, in connect
self.sock = ssl_wrap_socket(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 747, in urlopen
return self.urlopen(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 747, in urlopen
return self.urlopen(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 747, in urlopen
return self.urlopen(
[Previous line repeated 2 more times]
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 719, in urlopen
retries = retries.increment(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
pip._vendor.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
status = self.run(options, args)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
return func(self, options, args)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/commands/wheel.py", line 159, in run
requirement_set = resolver.resolve(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 314, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 467, in prepare_linked_requirement
local_file = unpack_url(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 255, in unpack_url
file = get_http_url(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 129, in get_http_url
from_path, content_type = _download_http_url(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 277, in _download_http_url
download = downloader(link)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/network/download.py", line 189, in __call__
resp = _http_get_download(self._session, link)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/network/download.py", line 135, in _http_get_download
resp = session.get(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/network/session.py", line 421, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/cachecontrol/adapter.py", line 53, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
pip._vendor.requests.exceptions.SSLError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)')))
Traceback (most recent call last):
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/installer.py", line 128, in fetch_build_egg
subprocess.check_call(cmd)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/manuel/repos/projects/Flownet/flownet/venv/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpyupahj5v', '--quiet', 'pytest-runner']' returned non-zero exit status 2.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-k17vq1n3/setup.py", line 18, in <module>
setup(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 144, in setup
_install_setup_requires(attrs)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/dist.py", line 718, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 780, in resolve
dist = best[req.key] = env.best_match(
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1065, in best_match
return self.obtain(req, installer)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1077, in obtain
return installer(requirement)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/dist.py", line 777, in fetch_build_egg
return fetch_build_egg(self, req)
File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/installer.py", line 130, in fetch_build_egg
raise DistutilsError(str(e))
distutils.errors.DistutilsError: Command '['/home/manuel/repos/projects/Flownet/flownet/venv/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpyupahj5v', '--quiet', 'pytest-runner']' returned non-zero exit status 2.
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Original issue author: @edubarrosTNO
Current situation
At the moment, the well heads (WELSPECS
), well completions (COMPDAT
), and well controls (WCONPROD, WCONINJE, etc.
) are all prescribed in a single file.
WELSPECS
and COMPDAT
are defined at the dates in which the wells are opened, along with their well controls.
Desired situation
In preparation for the use of FlowNet for well control optimization (e.g., injection rate optimization), I propose that these keywords are put in separate files.
We should have WELSPECS.inc
and COMPDAT.inc
containing information for all wells (even if they are shut at the beginning of simulation) included directly in the .DATA
-file (in SCHEDULE
section) and a separate SCHEDULE.inc
file with well controls and dates.
This will make our life easier when optimization starts varying well controls, as a single file will need to be changed. This will also improve readability of the schedule file.
Author: @wouterjdb
Currently the _to_abs_path
-function does not support absolute paths.
Flownet set wells status "CLOSED" by default. I'm applying flownet for the egg model and all wells are set to be "CLOSED" by default. Maybe, flownets should warn the user when wells status is not defined in the original model.
Original author: @wouterjdb
We should include the accuracy metric in ERT using workflows during the AHM.
We should also considering to include the heatmap and/or other useful analytical plots. @edubarrosTNO, which ones were most useful?
Please have a look at to see how you can add a workflow in the code base.
Original author: @wouterjdb
We're adding more and more parameters increasing the degrees of freedom. For some parameters it would maybe not always make sense to add them as an uncertain parameter, but a constant might be good enough. This is currently not supported.
I suggest that something like this would be allowed in the config:
model_parameters:
permeability:
min: 0.1
max: 2000
bulkvolume_mult:
constant: 1
Author: @wouterjdb
In the current FlowNet setup we assign permeability and porosity fully independently. This might be improved by using poro/perm relations.
At the same time, try to remove the norne.tar.gz
file from repository history - in order to dramatically reduce repository size.
Original authour: @wouterjdb
In cases where there is not both oil and gas present the config parser, after #88, now results in:
ValueError: The configuration is not valid:Missing key: training_set_end_date, Missing key: min, Missing key: max, Missing key: min, Missing key: max, Missing key: min, Missing key: max, Missing key: min, Missing key: max, Missing key: min, Missing key: max, Missing key: min, Missing key: max
The MK.AllowNone
argument will need to be passed in those cases.
The following issues were identified when doing a fresh install of FlowNet on an Ubuntu 18.04 LTS machine:
Building a virtual environment at /home/media-unix/flownet/venv
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt-get install python3-venv
You may need to use sudo with that command. After installing the python3-venv
package, recreate your virtual environment.
Failing command: ['/home/media-unix/flownet/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']
Running the suggested command, with sudo, solves the issue.
ImportError: Failed to import any qt binding
pip install PyQt5
the following error is returned:(venv) media-unix@media-unix:~/flownet/examples$ pip install PyQt5
Collecting PyQt5
Downloading https://files.pythonhosted.org/packages/3a/fb/eb51731f2dc7c22d8e1a63ba88fb702727b324c6352183a32f27f73b8116/PyQt5-5.14.1.tar.gz (3.2MB)
100% |████████████████████████████████| 3.2MB 248kB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.6/tokenize.py", line 452, in open
buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-build-xmx1z3c3/PyQt5/setup.py'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-xmx1z3c3/PyQt5/
This can be solved by installing a different version, for example, pip install pyqt5==5.14.0
works fine.
After fixing those two issues, FlowNet ran fine.
Original author: @wouterjdb
The aquifer section is set to be "not required" in the configuration file. However, flownet assumes that the section is always defined and it comes into an error when the aquifer is not defined.
Rock compressibility section it is defined to be "required", however. It is necessary to be always defined?
Original authour: @wouterjdb
If you set both the training set fraction and end date you'll get the following warning when running FlowNet:
Training set fraction and training set end date are both defined in config file.
The input given for training set fraction will be ignored.
The training set end date will be used to calculate the training set fraction.
I'd actually suggest to let the config parser handle this and stop execution early on. A warning during execution is a bit 'too silent' if you ask me.
This refers to the latest changes from PR #56 by @anders-kiaer and @olwijn .
I saw all the checks and reviewer's approval ✅ and took the liberty to merge the PR, because I wanted to try this fix as soon as possible. But there seems to be a bug: the wells are all showing 'STOP'
status in the history schedule (both producers and injectors). See below for the Brugge case:
WCONHIST
-- WELL OPEN/ CNTL OIL WAT GAS VFP VFP THP BHP
-- NAME SHUT MODE RATE RATE RATE TABLE ALFQ PRES PRES
'BR-P-1' 'STOP' 'RESV' 317.9749 0.0051 0 1* 1* 63.7635 69.8381 /
(...)
/
WCONINJH
-- WELL FLUID OPEN/ SURF RESV BHP THP VFP NOT CNTL
-- NAME TYPE SHUT RATE RATE PRES PRES TABLE USED MODE
'BR-I-1' 'WATER' 'STOP' 635.9500 150.8748 0 1* 1* /
(...)
/
In a successful github workflow run one finds the following error in the logs:
The script 'ExternalErtScript' caused an error while running:
Simulations completed.
Traceback (most recent call last):
File "/home/runner/work/flownet/flownet/flownet_venv/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'DATE'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/runner/work/flownet/flownet/flownet_venv/bin/flownet_save_iteration_analytics", line 8, in <module>
sys.exit(save_iteration_analytics())
File "/home/runner/work/flownet/flownet/flownet_venv/lib/python3.7/site-packages/flownet/ahm/_ahm_iteration_analytics.py", line 420, in save_iteration_analytics
df_sim = df_sim[df_sim["DATE"].isin(df_obs["DATE"])]
File "/home/runner/work/flownet/flownet/flownet_venv/lib/python3.7/site-packages/pandas/core/frame.py", line 2995, in __getitem__
indexer = self.columns.get_loc(key)
File "/home/runner/work/flownet/flownet/flownet_venv/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2899, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'DATE'
The DATE
key is missing in a pandas dataframe. Could this be a result of recent changes in the time shifting /resampling that were done?
The following error message (1) has something to do with flownet not being able to determine the convex hull in a particular case. Flownet needs at least 5 wells for the algorithm to be able to work. The connections need to define a 3D volume.
The model I'm working with has exactly the same depth for all wells.
A temporal solution to do a test is to change the perforation strategy from bottom_point
to multiple
that would mean each connected block in the Eclipse model with getting a connection in FlowNet.
This may be not what you want in the long run, but it is a simple test to see if you get further.
(1) Error message:
(venv) manuel@manuel:~/repos/projects/DataDrivenModels/flownet$ flownet ahm tests/configs/egg_parameters.yml egg_output_folder
Adding flow nodes: 0%Traceback (most recent call last):
File "/home/manuel/repos/projects/DataDrivenModels/flownet/venv/bin/flownet", line 11, in <module>
load_entry_point('flownet', 'console_scripts', 'flownet')()
File "/home/manuel/repos/projects/DataDrivenModels/flownet/src/flownet/_command_line.py", line 206, in main
args.func(args)
File "/home/manuel/repos/projects/DataDrivenModels/flownet/src/flownet/_command_line.py", line 67, in flownet_ahm
run_flownet_history_mathing(config, args)
File "/home/manuel/repos/projects/DataDrivenModels/flownet/src/flownet/ahm/_run_ahm.py", line 162, in run_flownet_history_mathing
df_connections: pd.DataFrame = create_connections(df_coordinates, config)
File "/home/manuel/repos/projects/DataDrivenModels/flownet/src/flownet/network_model/_generate_connections.py", line 451, in create_connections
starts, ends = _generate_connections(df_coordinates, configuration)
File "/home/manuel/repos/projects/DataDrivenModels/flownet/src/flownet/network_model/_generate_connections.py", line 125, in _generate_connections
random_seed=configuration.flownet.random_seed,
File "/home/manuel/repos/projects/DataDrivenModels/flownet/src/flownet/network_model/_mitchell.py", line 88, in mitchell_best_candidate_modified_3d
hull = Delaunay(np.column_stack([x, y, z]))
File "qhull.pyx", line 1839, in scipy.spatial.qhull.Delaunay.__init__
File "qhull.pyx", line 357, in scipy.spatial.qhull._Qhull.__init__
scipy.spatial.qhull.QhullError: QH6154 Qhull precision error: Initial simplex is flat (facet 1 is coplanar with the interior point)
While executing: | qhull d Qc Qbb Q12 Qt Qz
Options selected for Qhull 2019.1.r 2019/06/21:
run-id 1237336110 delaunay Qcoplanar-keep Qbbound-last Q12-allow-wide
Qtriangulate Qz-infinity-point _pre-merge _zero-centrum Qinterior-keep
Pgood _max-width 4.4e+02 Error-roundoff 8.1e-12 _one-merge 7.3e-11
Visible-distance 4.9e-11 U-max-coplanar 4.9e-11 Width-outside 9.7e-11
_wide-facet 2.9e-10 _maxoutside 9.7e-11
precision problems (corrected unless 'Q0' or an error)
9 zero divisors during gaussian elimination
The input to qhull appears to be less than 4 dimensional, or a
computation has overflowed.
Qhull could not construct a clearly convex simplex from points:
- p5(v5): 60 68 4e+03 0
- p6(v4): 2.5e+02 12 4e+03 1.2e+02
- p0(v3): 36 4.5e+02 4e+03 4.3e+02
- p7(v2): 4.5e+02 44 4e+03 4.3e+02
- p2(v1): 12 2.8e+02 4e+03 1.5e+02
The center point is coplanar with a facet, or a vertex is coplanar
with a neighboring facet. The maximum round off error for
computing distances is 8.1e-12. The center point, facets and distances
to the center point are as follows:
center point 162.4 170.4 4026 223.6
facet p6 p0 p7 p2 distance= 0
facet p5 p0 p7 p2 distance= 0
facet p5 p6 p7 p2 distance= 0
facet p5 p6 p0 p2 distance= 0
facet p5 p6 p0 p7 distance= 0
These points either have a maximum or minimum x-coordinate, or
they maximize the determinant for k coordinates. Trial points
are first selected from points that maximize a coordinate.
The min and max coordinates for each dimension are:
0: 12 452 difference= 440
1: 12 452 difference= 440
2: 4026 4026 difference= 0
3: 0 4026 difference= 4026
If the input should be full dimensional, you have several options that
may determine an initial simplex:
- use 'QJ' to joggle the input and make it full dimensional
- use 'QbB' to scale the points to the unit cube
- use 'QR0' to randomly rotate the input for different maximum points
- use 'Qs' to search all points for the initial simplex
- use 'En' to specify a maximum roundoff error less than 8.1e-12.
- trace execution with 'T3' to see the determinant for each point.
If the input is lower dimensional:
- use 'QJ' to joggle the input and make it full dimensional
- use 'Qbk:0Bk:0' to delete coordinate k from the input. You should
pick the coordinate with the least range. The hull will have the
correct topology.
- determine the flat containing the points, rotate the points
into a coordinate plane, and delete the other coordinates.
- add one or more points to make the input full dimensional.
Practical experience shows that history matching results may still contain biases. A two-step solution approach was proposed by Guo and Reynolds (2019) to address this, in which the posterior ensemble means are used as the means for a new prior in a second application of the history matching workflow.
If the ensemble of parameter estimates is saved in a compact form at the end of the history matching, new functionality could be introduced to specify that the mean of the new prior should be taken from this file.
When all/too many realizations fail you now get:
Simulations failed with the following error: Simulation failed! All realizations failed!
Traceback (most recent call last):
File "/home/travis/virtualenv/python3.6.9/bin/flownet", line 8, in <module>
sys.exit(main())
File "/home/travis/virtualenv/python3.6.9/lib/python3.6/site-packages/flownet/_command_line.py", line 100, in main
run_flownet_history_mathing(config, args)
File "/home/travis/virtualenv/python3.6.9/lib/python3.6/site-packages/flownet/_run_ahm.py", line 306, in run_flownet_history_mathing
ahm.run_ert(mode="es_mda", weights=config.ert.ensemble_weights)
File "/home/travis/virtualenv/python3.6.9/lib/python3.6/site-packages/flownet/_assisted_history_matching.py", line 336, in run_ert
check=True,
File "/opt/python/3.6.9/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'export PYTHONWARNINGS=ignore::DeprecationWarning;ert es_mda --weights '1' ahm_config.ert' returned non-zero exit status 1.
The command "flownet ../tests/configs/norne_parameters.yml ./some_ert_setup" exited with 1.
Would be useful that the subprocess command does show the actual error message from ERT. And if that is not informative either, either change Flownet or ERT to show e.g. stderr from the job that is failing in the realizations.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.