Giter Site home page Giter Site logo

acruzgarcia / omiedata Goto Github PK

View Code? Open in Web Editor NEW
21.0 2.0 4.0 535 KB

Python package to import data from OMIE (Iberian Peninsula's Electricity Market Operator): https://www.omie.es/

Home Page: https://pypi.org/project/OMIEData/

License: MIT License

Python 100.00%
electricity-market omie electricity-prices electricity-price-forecasting

omiedata's Introduction

OMIEData:

made-with-python PyPI version fury.io PyPI pyversions

Python package to import data from OMIE (Iberian Peninsula's Electricity Market Operator): https://www.omie.es/

Concretely, you can easily access to data for the following markets:

  • Daily market: hourly prices in Spain and Portugal, total hourly energy after auction (with/without billateral contracts), breakdown of the total hourly energy by technology and bid/ask curves.
  • Intra-day market: hourly prices for the different sessions and total hourly energy.
  • Additional data in next releases.

Installation

The package is uploaded at https://pypi.org/project/OMIEData/, so

python -m pip install OMIEData

from the command line will install the last version uploaded to pypi.

Aternatively, to install it from GitHub repository, type:

python -m pip install git+https://github.com/acruzgarcia/OMIEData

in the command line. You can also install the .whl or .tar.gz files within dist as:

python -m pip install OMIEData-VERSION-py3-none-any.whl

or

python -m pip install OMIEData-VERSION.tar.gz

or to install a previous version from dist_old.

Examples:

A very simple example to download hourly electricity prices and demand:

import datetime as dt
import matplotlib.pyplot as plt

from OMIEData.DataImport.omie_marginalprice_importer import OMIEMarginalPriceFileImporter
from OMIEData.Enums.all_enums import DataTypeInMarginalPriceFile

dateIni = dt.datetime(2020, 1, 1)
dateEnd = dt.datetime(2022, 3, 22)

# This can take time, it is downloading the files from the website..
df = OMIEMarginalPriceFileImporter(date_ini=dateIni, date_end=dateEnd).read_to_dataframe(verbose=True)
df.sort_values(by='DATE', axis=0, inplace=True)
print(df)

The code will generate a data-frame like the following one:

            DATE         CONCEPT        H1  ...       H22       H23       H24
0     2020-01-01        PRICE_SP     41.88  ...     45.60     42.90     37.55
1     2020-01-01        PRICE_PT     41.88  ...     45.60     42.90     37.55
2     2020-01-01         ENER_IB  18132.30  ...  22492.60  21800.90  19946.30
3     2020-01-01  ENER_IB_BILLAT  26488.50  ...  32611.70  31523.70  29088.30
4     2020-01-02        PRICE_SP     35.40  ...     42.00     38.60     33.39
          ...             ...       ...  ...       ...       ...       ...
3241  2022-03-21        PRICE_PT    218.69  ...    261.44    240.29    228.88
3245  2022-03-22        PRICE_PT    223.00  ...    256.00    242.18    212.99
3246  2022-03-22         ENER_IB  20652.20  ...  27113.50  24167.60  21841.50
3244  2022-03-22        PRICE_SP    223.00  ...    256.00    242.18    212.99
3247  2022-03-22  ENER_IB_BILLAT  29840.30  ...  38281.20  34781.90  31872.50
[3248 rows x 26 columns]

You can filter the data-frame to have only the prices in spain, and then plot

# Just prices in spain
str_price_spain = str(DataTypeInMarginalPriceFile.PRICE_SPAIN)
dfPrices = df[df.CONCEPT == str_price_spain]

# Plotting
plt.figure()
plt.plot(dfPrices.DATE, dfPrices.H12, label='H12')
plt.plot(dfPrices.DATE, dfPrices.H23, label='H23')
plt.legend()
plt.show()

which will produce the following plot:

alt text

Another example to download hourly demand resulting of the daily market auction, breakdown by technologies:

import datetime as dt
from OMIEData.Enums.all_enums import SystemType
from OMIEData.DataImport.omie_energy_by_technology_importer import OMIEEnergyByTechnologyImporter

dateIni = dt.datetime(2020, 6, 1)
dateEnd = dt.datetime(2020, 7, 30)
system_type = SystemType.SPAIN

# This can take time, it is downloading the files from the website..
df = OMIEEnergyByTechnologyImporter(date_ini=dateIni,
                                    date_end=dateEnd,
                                    system_type=system_type).read_to_dataframe(verbose=True)
df.sort_values(by=['DATE', 'HOUR'], axis=0, inplace=True)
print(df)

Another example to download supply/demand curves:

import datetime as dt
from OMIEData.DataImport.omie_supply_demand_curve_importer import OMIESupplyDemandCurvesImporter

dateIni = dt.datetime(2020, 6, 1)
dateEnd = dt.datetime(2020, 6, 1)
hour = 1

# This can take time, it is downloading the files from the website..
df = OMIESupplyDemandCurvesImporter(date_ini=dateIni, date_end=dateEnd, hour=hour).read_to_dataframe(verbose=True)
df.sort_values(by=['DATE', 'HOUR'], axis=0, inplace=True)
print(df)

Other examples that illustrate the use of the package in here:

Enjoy!.

omiedata's People

Contributors

acruzgarcia avatar bigtava avatar mirelmora avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

omiedata's Issues

unsupported locale setting when requesting txt file

When running
import datetime as dt
import matplotlib.pyplot as plt

from OMIEData.DataImport.omie_marginalprice_importer import OMIEMarginalPriceFileImporter
from OMIEData.Enums.all_enums import DataTypeInMarginalPriceFile

dateIni = dt.datetime(2023, 1, 1)
dateEnd = dt.datetime(2023, 1, 3)

This can take time, it is downloading the files from the website..

df = OMIEMarginalPriceFileImporter(date_ini=dateIni, date_end=dateEnd).read_to_dataframe(verbose=True)
df.sort_values(by='DATE', axis=0, inplace=True)
print(df)

I get this error. Already tried to change the locale settings but with no success. any ideas?

Requesting https://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_01_01_2023_01_01_2023.TXT ...
There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_01_01_2023_01_01_2023.TXT
unsupported locale settinghttps://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_01_01_2023_01_01_2023.TXT
Requesting https://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_02_01_2023_02_01_2023.TXT ...
There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_02_01_2023_02_01_2023.TXT
unsupported locale settinghttps://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_02_01_2023_02_01_2023.TXT
Requesting https://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_03_01_2023_03_01_2023.TXT ...
There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_03_01_2023_03_01_2023.TXT
unsupported locale settinghttps://www.omie.es/sites/default/files/dados/AGNO_2023/MES_01/TXT/INT_PBC_EV_H_1_03_01_2023_03_01_2023.TXT
Empty DataFrame
Columns: [DATE, CONCEPT, H1, H2, H3, H4, H5, H6, H7, H8, H9, H10, H11, H12, H13, H14, H15, H16, H17, H18, H19, H20, H21, H22, H23, H24, H25]
Index: []

DataFrame has no attribute 'concat'

Hi,

Using the example code:

import datetime as dt

from OMIEData.DataImport.omie_marginalprice_importer import OMIEMarginalPriceFileImporter
from OMIEData.Enums.all_enums import DataTypeInMarginalPriceFile

dateIni = dt.datetime(2019, 1, 1)
dateEnd = dt.datetime(2023, 7, 1)

df = OMIEMarginalPriceFileImporter(date_ini=dateIni, date_end=dateEnd).read_to_dataframe(verbose=True)
df.sort_values(by='DATE', axis=0, inplace=True)
print(df)

Yields a DataFrame concat error.

Requesting https://www.omie.es/sites/default/files/dados/AGNO_2019/MES_01/TXT/INT_PBC_EV_H_1_01_01_2019_01_01_2019.TXT ...
There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2019/MES_01/TXT/INT_PBC_EV_H_1_01_01_2019_01_01_2019.TXT
'DataFrame' object has no attribute 'concat'https://www.omie.es/sites/default/files/dados/AGNO_2019/MES_01/TXT/INT_PBC_EV_H_1_01_01_2019_01_01_2019.TXT

Tried downgrading to pandas 2.0.1 from 2.0.3 but the problem still occurs.
Using python 3.11.3

ModuleNotFoundError: No module named 'OMIEData.Enums'; 'OMIEData' is not a package

That's the message I'm seeing after a pip installation and trying "example_energy_by_technology_2steps.py" example.

Traceback (most recent call last):
  File "****/OMIEData.py", line 4, in <module>
    from OMIEData.DataImport.omie_data_importer_from_folder import OMIEDataImporterFromFolder
  File "****\OMIEData.py", line 4, in <module>
    from OMIEData.DataImport.omie_data_importer_from_folder import OMIEDataImporterFromFolder
ModuleNotFoundError: No module named 'OMIEData.DataImport'; 'OMIEData' is not a package

¿Is it working?

(Python 3.6 Windows 10)

Incorrect parse of files with 23 or 25 days

Hi,
For a date with 23 days, it seems to duplicate the final hour.
Example:
File: https://www.omie.es/sites/default/files/dados/AGNO_2023/MES_03/TXT/INT_PBC_EV_H_1_26_03_2023_26_03_2023.TXT
Result:
DATE CONCEPT H1 H2 H3 ... H20 H21 H22 H23 H24
1 2023-03-26 PRICE_PT 45.1 35.0 29.59 ... 74.83 104.63 99.42 82.25 82.25

For a date with 25 days, it ignore the 25th hour.
File: https://www.omie.es/sites/default/files/dados/AGNO_2022/MES_10/TXT/INT_PBC_EV_H_1_30_10_2022_30_10_2022.TXT
Result:
DATE CONCEPT H1 H2 ... H21 H22 H23 H24
1 2022-10-30 PRICE_PT 139.17 105.1 ... 200.0 225.35 209.27 157.31

Any way to fix this?

I have fixed the issue, and included in a pull request:
#5

unsupported locale setting

Hi @acruzgarcia @mirelmora , congrats on this project! It looks super interesting.
I'm very excited to using it, but unfortunately it gives me an error.

I'm launching this piece of code from your examples (I changed the dates to try faster):

import datetime as dt
from OMIEData.DataImport.omie_marginalprice_importer import OMIEMarginalPriceFileImporter

dateIni = dt.datetime(2022, 3, 11)
dateEnd = dt.datetime(2022, 3, 11)

# This can take time, it is downloading the files from the website..
df = OMIEMarginalPriceFileImporter(date_ini=dateIni, date_end=dateEnd).read_to_dataframe(verbose=True)
df.sort_values(by='DATE', axis=0, inplace=True)
print(df)

But I'm afraid I get an error "unsupported locale setting".
This is the output:

Requesting https://www.omie.es/sites/default/files/dados/AGNO_2022/MES_03/TXT/INT_PBC_EV_H_1_11_03_2022_11_03_2022.TXT ...
There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2022/MES_03/TXT/INT_PBC_EV_H_1_11_03_2022_11_03_2022.TXT
unsupported locale settinghttps://[www.omie.es/sites/default/files/dados/AGNO_2022/MES_03/TXT/INT_PBC_EV_H_1_11_03_2022_11_03_2022.TXT](http://www.omie.es/sites/default/files/dados/AGNO_2022/MES_03/TXT/INT_PBC_EV_H_1_11_03_2022_11_03_2022.TXT)
Empty DataFrame
Columns: [DATE, CONCEPT, H1, H2, H3, H4, H5, H6, H7, H8, H9, H10, H11, H12, H13, H14, H15, H16, H17, H18, H19, H20, H21, H22, H23, H24]
Index: []

I'd really appreciate if you could tell my what I'm doing wrong.
Thanks!

'DataFrame' object has no attribute 'concat'

Hey!
I just tried using the package, unsuccessfully. Maybe something changed within OMIE endpoints?
I tried the example code you suggested:

`import datetime as dt
import matplotlib.pyplot as plt

from OMIEData.DataImport.omie_marginalprice_importer import OMIEMarginalPriceFileImporter
from OMIEData.Enums.all_enums import DataTypeInMarginalPriceFile

dateIni = dt.datetime(2024, 1, 1)
dateEnd = dt.datetime(2024, 3, 22)

df = OMIEMarginalPriceFileImporter(date_ini=dateIni, date_end=dateEnd).read_to_dataframe(verbose=True)
df.sort_values(by='DATE', axis=0, inplace=True)
print(df)
`

And got this error:
'DataFrame' object has no attribute 'concat'https://www.omie.es/sites/default/files/dados/AGNO_2024/MES_01/TXT/INT_PBC_EV_H_1_15_01_2024_15_01_2024.TXT
Requesting https://www.omie.es/sites/default/files/dados/AGNO_2024/MES_01/TXT/INT_PBC_EV_H_1_16_01_2024_16_01_2024.TXT ...
There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2024/MES_01/TXT/INT_PBC_EV_H_1_16_01_2024_16_01_2024.TXT

Can you have a look at it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.