osuked/ngdataportal

Python wrapper for the National Grid data portal API

ngdataportal's Issues

Verification Error

Hi There, I seem to get this issue on all examples i try and follow

SSLError: HTTPSConnectionPool(host='national-grid-admin.ckan.io', port=443): Max retries exceeded with url: /api/3/action/datastore_search_sql?resource_id=b2f03146-f05d-4824-a663-3a4f36090c71&sql=SELECT+%2A+from+%22b2f03146-f05d-4824-a663-3a4f36090c71%22+WHERE+%22Settlement+Day%22+BETWEEN+%272019-12-20%27%3A%3Atimestamp+AND+%272019-12-22%27%3A%3Atimestamp+ORDER+BY+%22Settlement+Day%22 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)')))

What is the thought behind stream_to_resource_id.json?

I have started hacking at this in my fork (https://github.com/MichaelKavanagh/NGDataPortal/). First I am trying to remove the reliance on keeping stream_to_resource_id.json up to date, since if NG changes something the id hashes are no longer valid. This appears to have happened for historic-demand-data:

"historic-demand-data": "11252656-007c-45a4-87db-9d5cc6e8535a",

Querying CKAN results in a "Resource not found" error: https://data.nationalgrideso.com/api/3/action/resource_show?id=11252656-007c-45a4-87db-9d5cc6e8535a.

The thing is that many 'packages' (you seem to call them streams) have multiple 'resources'. This is not the case for your example embedded-wind-and-solar-forecasts where there are two resources, only one of which is a useful csv, but for historic-demand-data it has 8 resources the user could potentially want.

The user selects the package/stream when instantiating the Wrapper object by passing the name as a parameter to the c'tor. Presumably the class should provide the means to also select the desired resource, either directly as a c'tor param or methods to return a list to choose from?

I would be interested to know your thoughts on this. Also, are you open to reviewing/accepting PRs for this repo?

Settlement Day + Settlement Period = datetime_utc

Would it be possible to add function that changes the settlement day and settlement period to a datetime_utc. Then we dont get into any messy situations with clocks changes e.t.c.

I'm happy to give it a go first