Comments (7)
Thanks @rbagd for this. I'm going to have a look ASAP to this problem, and see the way to proceed. As much as possible, i will provide a small enhancement for this. Prior to this, i need to check if the message provided is somehow supported by the SDMX standard, or if it is adhoc handler implemented by Eurostat.
From a user point of view, do you think it would be enought to have a warning, only, or also to make it even smarter, and extract data from the zip
file and read it? (this also could be an option in readSDMX
)
from rsdmx.
Here's some more documentation from Eurostat on this. I haven't encountered this message elsewhere even though some datasets from OECD I've tried are twice as large.
It's true that it would be handy to have it processed automatically in most cases. I'm just a little worried that in those few other cases you could accidentally lock up your machine because of some wildcard abuse. Is that likely? An explicit option is probably the best idea - whether it's enabled by default could be left for the user.
from rsdmx.
It sounds good.
I've further looked to the SDMX standard, i didn't this footer
supported in SDMX 2.0
but yes in 2.1
. In the schemas, it is defined as follows:
Footer is used to communicate information such as error and warnings after the payload of a message.
I'm testing its integration in the package. The first step is to make the footer
part of the rsdmx
object model (rsdmx
intends to have a R image
of the SDMX information model), afterwhat i will add messaging if a footer
exists in the response. Later i will investigate about the download & read again option.
from rsdmx.
That's awesome. Thanks @eblondel for the great work on this package.
from rsdmx.
@rbagd You can now test it! Now readSDMX
will handle one more warning
in case the SDMX document contains a footer
.
However, i had a look to a possible suite, downloading / unzipping the file, but as it's not part of the SDMX standard (which does not include any specific element as alternative link), i would not add this to readSDMX
which should be generic.
Let me know when you have tested it!
from rsdmx.
It appears to work nicely with Eurostat: warnings do appear as expected with the few queries I tested. I'll keep an eye in the future for other data providers who have something similar.
The automatic processing for this particular issue at Eurostat can be easily worked around at the user level now that the footer is there, i.e. if length(query@footer@messages) > 0
, then download.file
-> unzip
-> readSDMX
, so it's really no big deal if it's not part of the function.
I think we can close the issue now. Thanks again for a swift response.
from rsdmx.
Thanks for your feedback, much appreciated. Feel free to open new tickets when you feel it appropriate.
from rsdmx.
Related Issues (20)
- Is the package still being developed? HOT 3
- Exporting a data frame as an SDMX file HOT 2
- Reading IMF with readSDMX HOT 4
- Reading BIS data with readSDMX HOT 4
- Consolidate Github CI builds
- Use https for OECD HOT 1
- Catching a call error
- readSDMX crash HOT 1
- readSDMX Bundesbank error HOT 1
- Content-Type application/xml causes Server errors for some providers
- ABS (Australia) sdmx provider is now on REST 2.1
- NBB (Belgium) service provider moved to https
- UKDS service provider seems to be SDMX-JSON restricted
- Add Bundesbank SDMX service provider
- change in Eurostat API: adjustment of rsdmx request builder and man page? HOT 5
- Missing control in trying to get embedded DSD from SDMX dataset
- Update Eurostat SDMX provider
- Add New EUROSTAT providers
- codelist 'method' gets all possible values of the attribute, including non-related to the data/table considered HOT 1
- R 4.3.0 -> Calling && or || -> sugnificant change HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rsdmx.