fzj-iek3-vsa / fine Goto Github PK
View Code? Open in Web Editor NEWThe FINE python package provides a framework for modeling, optimizing and assessing energy systems
Home Page: https://vsa-fine.readthedocs.io
License: Other
The FINE python package provides a framework for modeling, optimizing and assessing energy systems
Home Page: https://vsa-fine.readthedocs.io
License: Other
The following small script worked with the old version but gives an error: ValueError: Can only compare identically-labeled Series objects
:
import fine as fn
import pandas as pd
import numpy as np
esm = fn.EnergySystemModel(
locations={"DE", "AT", "CH"},
commodities={"energy"},
commodityUnitsDict={"energy": "joule"})
operationRateMax = pd.DataFrame(index=range(8760))
operationRateMax["DE"] = 0.5
operationRateMax["DE_CH"] = 0.5
operationRateMax["AT_CH"] = 0.5
operationRateMax["AT_DE"] = 0.5
operationRateMax["CH_DE"] = 0.5
operationRateMax["CH_AT"] = 0.5
fn.Transmission(
esM=esm,
name="transmission",
commodity="energy",
operationRateMax=operationRateMax)
It raises:
Traceback (most recent call last):
File "~/test.py", line 19, in <module>
fn.Transmission(
File "~/.conda/envs/py311/lib/python3.11/site-packages/fine/transmission.py", line 411, in __init__
self.fullOperationRateMax = utils.checkAndSetInvestmentPeriodTimeSeries(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.conda/envs/py311/lib/python3.11/site-packages/fine/utils.py", line 1080, in checkAndSetInvestmentPeriodTimeSeries
parameter[ip] = checkAndSetTimeSeries(
^^^^^^^^^^^^^^^^^^^^^^
File "~/.conda/envs/py311/lib/python3.11/site-packages/fine/utils.py", line 1134, in checkAndSetTimeSeries
if (data > locationalEligibility).any().any():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.conda/envs/py311/lib/python3.11/site-packages/pandas/core/ops/common.py", line 76, in new_method
return method(self, other)
^^^^^^^^^^^^^^^^^^^
File "~/.conda/envs/py311/lib/python3.11/site-packages/pandas/core/arraylike.py", line 56, in __gt__
return self._cmp_method(other, operator.gt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.conda/envs/py311/lib/python3.11/site-packages/pandas/core/series.py", line 6105, in _cmp_method
raise ValueError("Can only compare identically-labeled Series objects")
ValueError: Can only compare identically-labeled Series objects
In the line if (data > locationalEligibility).any().any():
data is
AT 0.0
AT_CH 1.0
AT_DE 1.0
CH 0.0
CH_AT 1.0
CH_DE 1.0
DE 1.0
DE_CH 1.0
dtype: float64
while locationalEligibility is
AT_CH 1
AT_DE 1
CH_AT 1
CH_DE 1
DE_AT 1
DE_CH 1
dtype: int64
Could you help me resolve what is going on here? Looks like the countries themselves are missing in locationalEligibility?
I did the following:
conda create -n py311 python=3.11
pip install fine
pip install python-conv geopandas jupyter notebook libpysal pip descartes glpk openpyxl matplotlib xlrd pyomo numpy pandas networkx scipy scikit-learn>=1.2 xarray rasterio netcdf4 tsam pwlf psutil gurobi-logtools
python test.py
While it would be very cool to have this working with python 3.11 - it is unfortunately a dependency mess, as fine does reference the needed dependencies anymore.
This means, that we now have to keep track of needed dependencies outside of fine when installing it with pip install fine
.
Would you be open for a PR which fixes this?
Obviously the following need to be added as deps into the pyproject.toml:
geopandas
# jupyter
# notebook
libpysal
# pip
descartes
glpk
openpyxl
matplotlib
xlrd
pyomo
numpy
pandas
networkx
scipy
scikit-learn>=1.2
xarray
rasterio
netcdf4
tsam
pwlf
psutil
gurobi-logtools
Hi, this issue is about the Community guidelines within the JOSS Review 6274.
Please add a CONTRIBUTING.md and a CODE_OF_CONDUCT.md to your repository that describes how to handle feature & bug issues and pull requests to reduce the hurdle for possible contributors. For this purpose, you can use tools such as contributing-gen.
In combination with moving the templates provided in the .gitlab folder to a .github folder, these would fulfill the community guideline (Issue 48).
Kind regards
@GregorBecker
It would be helpful to have a boolean that activates and deactivates all the logging done by FINE itself
Hi, this an issue about your paper from JOSS Review 6274.
General:
Summary:
Method section:
Statement of Need:
--> After you have highlighted the key functionalities more precisely, I would suggest to include the statement of need into your documentation. This will provide a quick basic understanding of the scope of your software when finding the documentation on RTD or the repository on Github.
References:
Kind regards
@GregorBecker
This Example file does not work. The Optimization takes approx. 20 minutes and raises the following error. I am using Python 3.8.11 and Windows 10.
esM.optimize(timeSeriesAggregation=True, optimizationSpecs='OptimalityTol=1e-3 method=2 cuts=0')
Time series aggregation specifications:
Number of typical periods:7, number of time steps per period:24
Declaring sets, variables and constraints for SourceSinkModel
declaring sets...
declaring variables...
declaring constraints...
(0.7239 sec)
Declaring sets, variables and constraints for ConversionModel
declaring sets...
declaring variables...
declaring constraints...
(0.2105 sec)
Declaring sets, variables and constraints for StorageModel
declaring sets...
declaring variables...
declaring constraints...
(2.3284 sec)
Declaring sets, variables and constraints for TransmissionModel
declaring sets...
declaring variables...
declaring constraints...
show more (open the raw output data in a text editor) ...
Sense: minimize
Solve time: 1248.9299240112305 sec.
Processing optimization output...
ValueError Traceback (most recent call last)
C:\Users\J5F46~1.BEL\AppData\Local\Temp/ipykernel_11072/1441548777.py in
----> 1 esM.optimize(timeSeriesAggregation=True, optimizationSpecs='OptimalityTol=1e-3 method=2 cuts=0')
C:\ProgramData\Anaconda3\envs\FINE-Example\lib\site-packages\FINE\energySystemModel.py in optimize(self, declaresOptimizationProblem, relaxIsBuiltBinary, timeSeriesAggregation, logFileName, threads, solver, timeLimit, optimizationSpecs, warmstart)
1139 for key, mdl in self.componentModelingDict.items():
1140 __t = time.time()
-> 1141 mdl.setOptimalValues(self, self.pyM)
1142 outputString = ('for {:' + w + '}').format(key + ' ...') + "(%.4f" % (time.time() - __t) + "sec)"
1143 utils.output(outputString, self.verbose, 0)
C:\ProgramData\Anaconda3\envs\FINE-Example\lib\site-packages\FINE\sourceSink.py in setOptimalValues(self, esM, pyM)
723
724 # Set optimal design dimension variables and get basic optimization summary
--> 725 optSummaryBasic = super().setOptimalValues(esM, pyM, esM.locations, 'commodityUnit')
726
727 # Set optimal operation variables and append optimization summary
C:\ProgramData\Anaconda3\envs\FINE-Example\lib\site-packages\FINE\component.py in setOptimalValues(self, esM, pyM, indexColumns, plantUnit, unitApp)
1707
1708 # Summarize all annualized contributions to the total annual cost
-> 1709 optSummary.loc[optSummary.index.get_level_values(1) == 'TAC'] =
1710 optSummary.loc[(optSummary.index.get_level_values(1) == 'capexCap') |
1711 (optSummary.index.get_level_values(1) == 'opexCap') |
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\indexing.py in setitem(self, key, value)
721
722 iloc = self if self.name == "iloc" else self.obj.iloc
--> 723 iloc._setitem_with_indexer(indexer, value, self.name)
724
725 def _validate_key(self, key, axis: int):
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\indexing.py in _setitem_with_indexer(self, indexer, value, name)
1730 self._setitem_with_indexer_split_path(indexer, value, name)
1731 else:
-> 1732 self._setitem_single_block(indexer, value, name)
1733
1734 def _setitem_with_indexer_split_path(self, indexer, value, name: str):
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\indexing.py in _setitem_single_block(self, indexer, value, name)
1966
1967 # actually do the set
-> 1968 self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value)
1969 self.obj._maybe_update_cacher(clear=True)
1970
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\internals\managers.py in setitem(self, indexer, value)
353
354 def setitem(self: T, indexer, value) -> T:
--> 355 return self.apply("setitem", indexer=indexer, value=value)
356
357 def putmask(self, mask, new, align: bool = True):
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\internals\managers.py in apply(self, f, align_keys, ignore_failures, **kwargs)
325 applied = b.apply(f, **kwargs)
326 else:
--> 327 applied = getattr(b, f)(**kwargs)
328 except (TypeError, NotImplementedError):
329 if not ignore_failures:
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\internals\blocks.py in setitem(self, indexer, value)
982 values, len(values[indexer]), value # type: ignore[arg-type]
983 )
--> 984 values[indexer] = value
985
986 if transpose:
ValueError: shape mismatch: value array of shape (9,0) could not be broadcast to indexing result of shape (9,8)
Hi, I have a question about the examples/tutorial section in JOSS Review 6274.
I've found the "Tutorial" section in your documentation where you only tell the modeling tool user that the examples are located in the "examples" folder.
In my opinion, it would be useful to present a brief description of each example (as found in the example notebooks) and link to them individually. This should include an introductory text and a logical order in which to use the examples to help the user learn how to use your software product.
I would also like to point out two formal things. First, you specify units in square brackets. Although this is common practice, it is not formally correct. Furthermore, it is not allowed to append indices to SI units. It would be nice if you could adapt this in your examples.
Nevertheless, I would like to emphasize that the description of the examples in the documentation is of greater importance.
Kind regards
@GregorBecker
phaseAngleVariablesOptimum is overwritten by operationVariablesOptimum in LOPF subclass.
In the setOptimalValues function under lopf, "self.operationVariablesOptimum" should be changed to "self.phaseAngleVariablesOptimum"
Hello, can I use FINE for multi-objective optimization of energy systems? I now want to make a calculation example that targets cost and carbon dioxide emissions. Thank you
Hi this is a question regarding the installation from JOSS Review 6274
I'm not quite sure why you decided to use a mamba-only installation procedure of your energy system modeling tool.
Since I am developing without the use of conda, I had to install micromamba exclusively for your package "FINE".
It would be better to give users the flexibility to choose their virtual environment on their own, so that they can either use Conda, mircomamba or a virtual environment solution of their choice. It is necessary to mention the possibility of using a Python virtual environment as an alternative which does not require additional installations.
Possible solution would be (for MacOS users):
python3 -m venv Your Directory
pip3 install fine
In my opinion, a description of the creation of virtual environments would not be necessary. A link to the relevant instructions would be sufficient.
Additionally it would be nice if the installation instruction in your documentation gets it's own menu button to be easier accessable via the documentation.
Kind Regards
@GregorBecker
Hi this is a question regarding the issue templates and files related to gitlab from JOSS Review 6274.
While quickly scanning your repository I've found a ".gitlab"-folder providing a templates for issue. Why don't you use them within your github-repository. Otherwise I would recommend removing them from the repository because they are not relevant for the presented software.
The same problem occurs with .gitlab-ci.yml.
Kind regards
@GregorBecker
The time-dependent cost which were introduced in the last pull request are not considered in the optimization summary of the output.
The EnergyLand Example does not work. When running the file Energyland.ipynb the following Error occurs at
esM.optimize(timeSeriesAggregation=True, solver='gurobi')
I use Python 3.8.11 on Windows 10
Time series aggregation specifications:
Number of typical periods:48, number of time steps per period:24
Declaring sets, variables and constraints for SourceSinkModel
declaring sets...
declaring variables...
declaring constraints...
(1.6530 sec)
Declaring sets, variables and constraints for ConversionModel
declaring sets...
declaring variables...
declaring constraints...
(1.1363 sec)
Declaring sets, variables and constraints for StorageModel
declaring sets...
declaring variables...
declaring constraints...
(0.8182 sec)
Declaring shared potential constraint...
(0.0000 sec)
Declaring linked component quantity constraint...
show more (open the raw output data in a text editor) ...
Sense: minimize
Solve time: 47.232340574264526 sec.
Processing optimization output...
ValueError Traceback (most recent call last)
C:\Users\J5F46~1.BEL\AppData\Local\Temp/ipykernel_3660/4170142563.py in
----> 1 esM.optimize(timeSeriesAggregation=True, solver='gurobi')
C:\ProgramData\Anaconda3\envs\FINE-Example\lib\site-packages\FINE\energySystemModel.py in optimize(self, declaresOptimizationProblem, relaxIsBuiltBinary, timeSeriesAggregation, logFileName, threads, solver, timeLimit, optimizationSpecs, warmstart)
1139 for key, mdl in self.componentModelingDict.items():
1140 __t = time.time()
-> 1141 mdl.setOptimalValues(self, self.pyM)
1142 outputString = ('for {:' + w + '}').format(key + ' ...') + "(%.4f" % (time.time() - __t) + "sec)"
1143 utils.output(outputString, self.verbose, 0)
C:\ProgramData\Anaconda3\envs\FINE-Example\lib\site-packages\FINE\sourceSink.py in setOptimalValues(self, esM, pyM)
723
724 # Set optimal design dimension variables and get basic optimization summary
--> 725 optSummaryBasic = super().setOptimalValues(esM, pyM, esM.locations, 'commodityUnit')
726
727 # Set optimal operation variables and append optimization summary
C:\ProgramData\Anaconda3\envs\FINE-Example\lib\site-packages\FINE\component.py in setOptimalValues(self, esM, pyM, indexColumns, plantUnit, unitApp)
1707
1708 # Summarize all annualized contributions to the total annual cost
-> 1709 optSummary.loc[optSummary.index.get_level_values(1) == 'TAC'] =
1710 optSummary.loc[(optSummary.index.get_level_values(1) == 'capexCap') |
1711 (optSummary.index.get_level_values(1) == 'opexCap') |
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\indexing.py in setitem(self, key, value)
721
722 iloc = self if self.name == "iloc" else self.obj.iloc
--> 723 iloc._setitem_with_indexer(indexer, value, self.name)
724
725 def _validate_key(self, key, axis: int):
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\indexing.py in _setitem_with_indexer(self, indexer, value, name)
1730 self._setitem_with_indexer_split_path(indexer, value, name)
1731 else:
-> 1732 self._setitem_single_block(indexer, value, name)
1733
1734 def _setitem_with_indexer_split_path(self, indexer, value, name: str):
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\indexing.py in _setitem_single_block(self, indexer, value, name)
1966
1967 # actually do the set
-> 1968 self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value)
1969 self.obj._maybe_update_cacher(clear=True)
1970
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\internals\managers.py in setitem(self, indexer, value)
353
354 def setitem(self: T, indexer, value) -> T:
--> 355 return self.apply("setitem", indexer=indexer, value=value)
356
357 def putmask(self, mask, new, align: bool = True):
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\internals\managers.py in apply(self, f, align_keys, ignore_failures, **kwargs)
325 applied = b.apply(f, **kwargs)
326 else:
--> 327 applied = getattr(b, f)(**kwargs)
328 except (TypeError, NotImplementedError):
329 if not ignore_failures:
~\AppData\Roaming\Python\Python38\site-packages\pandas\core\internals\blocks.py in setitem(self, indexer, value)
982 values, len(values[indexer]), value # type: ignore[arg-type]
983 )
--> 984 values[indexer] = value
985
986 if transpose:
ValueError: shape mismatch: value array of shape (17,0) could not be broadcast to indexing result of shape (17,1)
It would be great if the two steps of the pyomo.Concrete model initialization and parameterization could be separated from the model solving. Such, it would be possible to add custom constraints after the initialization process, or to output the resulting model without the requirement to solve it
Hi there,
I created a new python project in PyCharm with Virtualenv + Python 3.7 and installed FINE with pip install fine
.
That ran successfully:
...\pythonTestProject> pip install fine
[...]
Installing collected packages: numpy, urllib3, threadpoolctl, six, scipy, pytz, pyparsing, MarkupSafe, joblib, idna, decorator, charset-normalizer, certifi, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontrib-devhelp, sphinxcontrib-applehelp, snowball
stemmer, scikit-learn, requests, python-dateutil, Pygments, ply, paramz, packaging, Jinja2, imagesize, docutils, cython, colorama, babel, alabaster, sphinx, sklearn, pyomo, pyDOE, pillow, pandas, networkx, kiwisolver, GPy, et-xmlfile, cycler, xlrd, tsam, sphinx-rtd-theme, shapely, pwlf, openpyxl, matplotlib, GP
yOpt, fine
Successfully installed GPy-1.10.0 GPyOpt-1.2.6 Jinja2-3.0.2 MarkupSafe-2.0.1 Pygments-2.10.0 alabaster-0.7.12 babel-2.9.1 certifi-2021.10.8 charset-normalizer-2.0.7 colorama-0.4.4 cycler-0.10.0 cython-0.29.24 decorator-5.1.0 docutils-0.17.1 et-xmlfile-1.1.0 fine-2.2.1 idna-3.3 imagesize-1.2.0 joblib-1.1.0 kiwis
olver-1.3.2 matplotlib-3.4.3 networkx-2.6.3 numpy-1.21.3 openpyxl-3.0.9 packaging-21.0 pandas-1.3.4 paramz-0.9.5 pillow-8.4.0 ply-3.11 pwlf-2.0.4 pyDOE-0.3.8 pyomo-6.1.2 pyparsing-3.0.1 python-dateutil-2.8.2 pytz-2021.3 requests-2.26.0 scikit-learn-1.0 scipy-1.7.1 shapely-1.7.1 six-1.16.0 sklearn-0.0 snowballst
emmer-2.1.0 sphinx-4.2.0 sphinx-rtd-theme-1.0.0 sphinxcontrib-applehelp-1.0.2 sphinxcontrib-devhelp-1.0.2 sphinxcontrib-htmlhelp-2.0.0 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.3 sphinxcontrib-serializinghtml-1.1.5 threadpoolctl-3.0.0 tsam-1.1.2 urllib3-1.26.7 xlrd-2.0.1
I created a simple main.py
script:
import FINE
if __name__ == '__main__':
print("Hello World!")
Outputs:
...\pythonTestProject\venv\Scripts\python.exe .../pythonTestProject/main.py
Traceback (most recent call last):
File "C:/Users/Paul/Git/EWI/pythonTestProject/main.py", line 3, in <module>
import FINE
File "...\pythonTestProject\venv\lib\site-packages\FINE\__init__.py", line 6, in <module>
from .energySystemModel import EnergySystemModel
File "...\pythonTestProject\venv\lib\site-packages\FINE\energySystemModel.py", line 6, in <module>
from FINE.component import Component, ComponentModel
File "...\pythonTestProject\venv\lib\site-packages\FINE\component.py", line 2, in <module>
from FINE import utils
File "...\pythonTestProject\venv\lib\site-packages\FINE\utils.py", line 9, in <module>
import pwlf
File "...\pythonTestProject\venv\lib\site-packages\pwlf\__init__.py", line 1, in <module>
from .pwlf import PiecewiseLinFit # noqa F401
File "...\pythonTestProject\venv\lib\site-packages\pwlf\pwlf.py", line 27, in <module>
from scipy.optimize import differential_evolution
File "...\pythonTestProject\venv\lib\site-packages\scipy\__init__.py", line 136, in <module>
from . import _distributor_init
File "...\pythonTestProject\venv\lib\site-packages\scipy\_distributor_init.py", line 61, in <module>
WinDLL(os.path.abspath(filename))
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\ctypes\__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] Das angegebene Modul wurde nicht gefunden
Process finished with exit code 1
I tried multiple combinations of different versions of scipy and pwlf.
Also including the minimum of scipy==1.4.1
and pwlf==1.1.5
mentioned in requirements.txt
.
But this still raises an error.
Update constraint formulation for a more generic fixed and maximum operation rate, for example with:
if opRateName is None:
rate = getattr(compDict[compName], 'operationRateFix')
else:
rate = getattr(compDict[compName], opRateName)
return opVar[loc, compName, p, t] == capVar[loc, compName] * rate[loc][p, t] * factor
I installed FINE using conda environment file and then through pip install .
on a Windows machine. I am able to run this example except for visualization where I get a following error when calling fig, ax = fn.plotLocations('regions.shp', indexColumn='regionName', plotLocNames=True)
:
File "_proj.pyx", line 129, in _proj.Proj.__cinit__
RuntimeError: b'no arguments in initialization list'
I have tried several suggestions given here for this error, including deleteting pyproj
and installing it through pip, which gives a new error that says CRSError: Invalid project: : ...
. I tried several other suggestions given here, but nothing worked. I also deleted and reinstalled the environment/package after those suggestions and I get the same errors.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.