gshs-ornl / wbstats Goto Github PK

View Code? Open in Web Editor NEW

127.0 127.0 23.0 14.36 MB

wbstats: An R package for searching and downloading data from the World Bank API

Home Page: http://gshs-ornl.github.io/wbstats/

License: Other

R 100.00%

open-data r r-package rstats world-bank world-bank-api worldbank

wbstats's People

Contributors

Stargazers

Watchers

wbstats's Issues

named vector renames indicators automatically

I've gotten good feedback from users on this feature and I use it all the time myself. Would be nice to see implemented here:

library(WDI)
ind = c("gdp_per_capita" = "NY.GDP.PCAP.KD",
        "population" = "SP.POP.TOTL")
dat = WDI(indicator = ind)
head(dat)
iso2c    country year gdp_per_capita population
1    1A Arab World 2005       5378.379  316264728
2    1A Arab World 2006       5594.899  323773264
3    1A Arab World 2007       5711.663  331653797
4    1A Arab World 2008       5898.516  339825483
5    1A Arab World 2009       5782.422  348145094
6    1A Arab World 2010       5916.330  356508908

Valid ISO3 codes excluded?

Hi, thanks for the nice package first of all. I was comparing the result of downloading an indicator "GE.EST" through the wbstats package and by downloading as a csv from the World Bank data portal. I noticed a discrepancy: in the wbstats results there are five countries fewer than that found in the csv. They are: "AIA" "GUF" "JEY" "MTQ" "REU". I wonder if this is because you are cross-checking codes against your internal data frame of valid codes before running the query? These codes can indeed be found in the WB database but when I ask to download data from these codes it runs into an error.

Is it possible that the list of valid ISO codes is outdated? Or have I misunderstood something? Or is it a problem with the WB API?

Thanks for any pointers.

wbsources and thus wbcache return error

due to a change in World Banks API, wbstats::wbsources() and thus wbstats::wbcache() now return the following error

Error in call_api(url_string = url_string, indicator = indicator) : 
  argument "indicator" is missing, with no default

disappeared from CRAN?

hi. CRAN says

Package ‘wbstats’ was removed from the CRAN repository.

cheers.

wb_data() fails for indicators in sources with missing metadata

For some sources, wb_data() fails. The culprit seems to be the footnote param in the query, which is hardcoded to y; this fails for most indicators in sources with missing metadata. Setting the footnote query param conditionally resolves this.

Reprex:

library(wbstats)
library(httr)
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

# try to get data

wb_data("BI.POP.TOTL.CV", return_wide = F, country = "CZE")
#> Error: World Bank API request failed for indicator BI.POP.TOTL.CV
#> message: Client error: (400) Bad Request
#> category: Client error
#> reason: Bad Request 
#> url: https://api.worldbank.org/v2/en/country/cze/indicator/BI.POP.TOTL.CV?footnote=y&cntrycode=y&per_page=20000&format=json

# check if the footnote param makes a difference

httr::GET("https://api.worldbank.org/v2/en/country/cze/indicator/BI.POP.TOTL.CV?footnote=y&cntrycode=y&per_page=20000&format=json")
#> Response [https://api.worldbank.org/v2/en/country/cze/indicator/BI.POP.TOTL.CV?footnote=y&cntrycode=y&per_page=20000&format=json]
#>   Date: 2021-02-11 17:56
#>   Status: 400
#>   Content-Type: text/html
#>   Size: 1.65 kB
#> <?xml version="1.0" encoding="utf-8"?>
#> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3...
#> <html xmlns="http://www.w3.org/1999/xhtml">
#>   <head>
#>     <title>Request Error</title>
#>     <style>BODY { color: #000000; background-color: white; font-family: Verda...
#>   </head>
#>   <body>
#>     <div id="content">
#>       <p class="heading1">Request Error</p>
#> ...
httr::GET("https://api.worldbank.org/v2/en/country/cze/indicator/BI.POP.TOTL.CV?footnote=n&cntrycode=y&per_page=20000&format=json")
#> Response [https://api.worldbank.org/v2/en/country/cze/indicator/BI.POP.TOTL.CV?footnote=n&cntrycode=y&per_page=20000&format=json]
#>   Date: 2021-02-11 17:56
#>   Status: 200
#>   Content-Type: application/json;charset=utf-8
#>   Size: 3.83 kB

# would setting return_wide to FALSE help?

wb_data("BI.POP.TOTL.CV", return_wide = T, country = "CZE")
#> Error: World Bank API request failed for indicator BI.POP.TOTL.CV
#> message: Client error: (400) Bad Request
#> category: Client error
#> reason: Bad Request 
#> url: https://api.worldbank.org/v2/en/country/cze/indicator/BI.POP.TOTL.CV?footnote=y&cntrycode=y&per_page=20000&format=json

# check that other datasets from sources with missing metadata return the same error

srcs <- wb_sources()

srcs_subset <- srcs %>% 
  filter(!metadata_available)

wb_indicators() %>% 
  filter(source_id %in% srcs_subset$source_id) %>%
  group_by(source_id) %>% 
  slice_head(n = 1) %>% 
  select(indicator_id, source_id)
#> # A tibble: 8 x 2
#> # Groups:   source_id [8]
#>   indicator_id         source_id
#>   <chr>                    <dbl>
#> 1 NYGDPMKTPKDZ                27
#> 2 A1                          30
#> 3 GFDD.AI.01                  32
#> 4 IN.AGR.GR.IRRIG.AREA        41
#> 5 NY.ADJ.ICTR.CD              43
#> 6 BI.EMP.FRML.PB.ZS           64
#> 7 CD                          71
#> 8 gtap10VACapital             80

wb_data("NYGDPMKTPKDZ", return_wide = F, country = "CZE")
#> Warning in wb_data("NYGDPMKTPKDZ", return_wide = F, country = "CZE"): No data
#> was returned for your query. Returning an empty tibble
#> # A tibble: 0 x 0
wb_data("NYGDPMKTPKDZ", return_wide = T, country = "CZE")
#> Warning in wb_data("NYGDPMKTPKDZ", return_wide = T, country = "CZE"): No data
#> was returned for your query. Returning an empty tibble
#> # A tibble: 0 x 0
wb_data("A1", return_wide = F, country = "CZE")
#> Warning in wb_data("A1", return_wide = F, country = "CZE"): No data was returned
#> for your query. Returning an empty tibble
#> # A tibble: 0 x 0
wb_data("A1", return_wide = T, country = "CZE")
#> Warning in wb_data("A1", return_wide = T, country = "CZE"): No data was returned
#> for your query. Returning an empty tibble
#> # A tibble: 0 x 0
wb_data("GFDD.AI.01", return_wide = F, country = "CZE")  # this one works, curiously
#> # A tibble: 58 x 11
#>    indicator_id indicator iso2c iso3c country  date value unit  obs_status
#>    <chr>        <chr>     <chr> <chr> <chr>   <dbl> <dbl> <chr> <chr>     
#>  1 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2017    NA <NA>  <NA>      
#>  2 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2016    NA <NA>  <NA>      
#>  3 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2015    NA <NA>  <NA>      
#>  4 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2014    NA <NA>  <NA>      
#>  5 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2013    NA <NA>  <NA>      
#>  6 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2012    NA <NA>  <NA>      
#>  7 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2011    NA <NA>  <NA>      
#>  8 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2010    NA <NA>  <NA>      
#>  9 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2009    NA <NA>  <NA>      
#> 10 GFDD.AI.01   Bank acc… CZ    CZE   Czech …  2008    NA <NA>  <NA>      
#> # … with 48 more rows, and 2 more variables: footnote <chr>,
#> #   last_updated <date>
wb_data("GFDD.AI.01", return_wide = T, country = "CZE")  # this one works, curiously
#> # A tibble: 58 x 9
#>    iso2c iso3c country    date GFDD.AI.01 unit  obs_status footnote last_updated
#>    <chr> <chr> <chr>     <dbl>      <dbl> <chr> <chr>      <chr>    <date>      
#>  1 CZ    CZE   Czech Re…  1960         NA <NA>  <NA>       <NA>     2019-10-30  
#>  2 CZ    CZE   Czech Re…  1961         NA <NA>  <NA>       <NA>     2019-10-30  
#>  3 CZ    CZE   Czech Re…  1962         NA <NA>  <NA>       <NA>     2019-10-30  
#>  4 CZ    CZE   Czech Re…  1963         NA <NA>  <NA>       <NA>     2019-10-30  
#>  5 CZ    CZE   Czech Re…  1964         NA <NA>  <NA>       <NA>     2019-10-30  
#>  6 CZ    CZE   Czech Re…  1965         NA <NA>  <NA>       <NA>     2019-10-30  
#>  7 CZ    CZE   Czech Re…  1966         NA <NA>  <NA>       <NA>     2019-10-30  
#>  8 CZ    CZE   Czech Re…  1967         NA <NA>  <NA>       <NA>     2019-10-30  
#>  9 CZ    CZE   Czech Re…  1968         NA <NA>  <NA>       <NA>     2019-10-30  
#> 10 CZ    CZE   Czech Re…  1969         NA <NA>  <NA>       <NA>     2019-10-30  
#> # … with 48 more rows
wb_data("IN.AGR.GR.IRRIG.AREA", return_wide = F)
#> Error: World Bank API request failed for indicator IN.AGR.GR.IRRIG.AREA
#> message: Client error: (400) Bad Request
#> category: Client error
#> reason: Bad Request 
#> url: https://api.worldbank.org/v2/en/country/all/indicator/IN.AGR.GR.IRRIG.AREA?footnote=y&cntrycode=y&per_page=20000&format=json
wb_data("IN.AGR.GR.IRRIG.AREA", return_wide = T)
#> Error: World Bank API request failed for indicator IN.AGR.GR.IRRIG.AREA
#> message: Client error: (400) Bad Request
#> category: Client error
#> reason: Bad Request 
#> url: https://api.worldbank.org/v2/en/country/all/indicator/IN.AGR.GR.IRRIG.AREA?footnote=y&cntrycode=y&per_page=20000&format=json
wb_data("gtap10VACapital", return_wide = F) # this fails for a different reason, which we see if we set return_wide to FALSE
#> Error: World Bank API request failed for indicator gtap10VACapital The following message was returned from the server
#> id: 175
#> key: Invalid format
#> value: The indicator was not found. It may have been deleted or archived.
#> 
#> failed request:
#> https://api.worldbank.org/v2/en/country/all/indicator/gtap10VACapital?footnote=y&cntrycode=y&per_page=20000&format=json
wb_data("gtap10VACapital", return_wide = T)
#> Error: World Bank API request failed for indicator gtap10VACapital The following message was returned from the server
#> id: 175
#> key: Invalid format
#> value: The indicator was not found. It may have been deleted or archived.
#> 
#> failed request:
#> https://api.worldbank.org/v2/en/country/all/indicator/gtap10VACapital?footnote=y&cntrycode=y&per_page=20000&format=json

^{Created on 2021-02-11 by the reprex package (v0.3.0)}

So it seems that the source endpoints behave somewhat inconsistently, but setting the footnote param to false when the footnote data is not available does provide a fix for those indicators where the query fails.

A possible fix is to set the footnote query param conditional on the return_wide function param. I will submit this as a PR for you to consider. A more complete fix would be to alter the footnote query parameter depending on information on the source (wb_cachelist$sources) but I am not familiar enough with the innards of the package to do this properly.

Congo Dem Rep code "CD" not working

I designed my shiny app to use iso2 codes rather than iso3 codes.

However, I've noticed Congo Dem. Rep. iso2 code "CD" can't fetch the data.

data <- wb(country = c("CD") ,indicator = "SP.POP.TOTL", startdate = 1960, enddate = 2015)
Warning message:
  No data was returned for any requested country and indicator. Returning empty data frame

using the iso3 code "COD" seems to work. I was wondering if this is a bug?

Here's a screen shot from the app with Congo Dem. Rep. missing its data.

unable to pull projection data

I am trying to pull data from the Population estimates and projections. The series runs to the year 2050. However, the latest data I can pull from the api using your package only runs till 2016 (e.g indicator SP.POP.DPND.YG). Some other indicators from the dataset return no data at all (e.g SP.POP.AG03.MA.IN).

I run into the same problem with the WDI package (but the latest returned year is 2017).

Is this an issue with the api or the package?

Thanks

WDI package

FYI,

https://cran.r-project.org/web/packages/WDI/index.html
https://github.com/vincentarelbundock/WDI

Namibia's iso2c code is treated as NA rather than "NA"

Namibia's iso2c code is "NA", but wb_data returns missing instead of the string. Hence, you cannot merge on it.

wb_data("countries_only",
        indicator = c("SP.POP.TOTL", "SP.URB.TOTL"),
        mrv = 1,
        gapfill = TRUE, return_wide = TRUE) %>%
  left_join(wb_cachelist$countries[, c('iso3c','region')], by = c('iso3c')) %>% 
  filter(iso3c == "NAM")

wb_data("countries_only",
        indicator = c("SP.POP.TOTL", "SP.URB.TOTL"),
        mrv = 1,
        gapfill = TRUE, return_wide = TRUE) %>%
  left_join(wb_cachelist$countries[, c('iso2c','region')], by = c('iso2c')) %>% 
  filter(iso3c == "NAM")

Enable bulk loading or gracious failing if certain indicators are not available.

Hello, thanks a lot for this package and the comprehensive access to World Bank statistics it provides. I am using this package in part to download large volumnes of indicators to populate a local macroeconomic database for a particular country. I have 2 issues here which are (1) loading many indicators sequentially is quite slow, and (2) if a partticular indicator is not available for my country the whole API call fails and everything that was loaded before is lost. To give an example: I wanted to load all 4269 indicators in the Education Statistics database. For Example:

library(wbstats)
indlist <- wb_indicators()
ind <- subset(indlist, source == "Education Statistics")[[1]]
WB_EDU <- wb_data(indicator = ind, country = "KEN", start_date = 1960, end_date = 2021)

Fails after a while with "Error: World Bank API request failed for indicator LO.EGRA.NCWPM.DAG.2GRD", and the rest of my query is lost.

I know this probably means some extra work, but in the medium term, an option to bulk loading all indicators from a specific source and / or skipping indicators which cannot be loaded (i.e. using tryCatch around the API calls and displaying a warning without terminating the request) would be great. Both could also be enabled through additional arguments to wb_data.

Drop dependence on magrittr package and use pipe operator built into R 4.1.0

Hello Jesse Piburn,

I hope this message finds you well. I wanted to share a suggestion regarding the usage of the pipe operator in your code. I noticed that the %>% operator from the magrittr package is used a few times in your code, specifically around 3 instances in the wbstats::wb_data function.

Considering that the |> operator was introduced in R 4.1.0, I wanted to suggest replacing %>% with |> in your code. This could potentially allow you to reduce your dependence on the magrittr package.

Furthermore, I came across a discussion here that highlights the performance benefits of using the |> operator compared to %>%. It seems that |> is faster, which might be advantageous for your code.

I understand that the wbstats package relies on R (≥ 3.2), so I'm not entirely sure how this change might impact the overall package. However, I wanted to bring this to your attention as a possible improvement worth considering.

provide fixed=TRUE in wbsearch

sometimes you now the indicators name but it contains reg exp character strings. It would be good to disable reg ex interpretation of the pattern argument in wbsearch.

indicator<- "Households and NPISHs final consumption expenditure (% of GDP)"
a<- wbsearch(indicator)
In wbsearch(indicator) :
no matches were found for the search term Households and NPISHs final consumption expenditure (% of GDP). Returning an empty data frame

But of course there is one ("NE.CON.PRVT.ZS").
something like a<- wbsearch(indicator, fixed=TRUE) like in the grep case is needed.

Best,
Stefan

PS: Great package!!

SP.POP.TOTL returning a (400) Client Error Code

Hi this morning on 4/26/21 I noticed my wbstats functions were not working on both on my local computer and a remote Shiny Server that I have. I pasted a screenshot of the error I'm getting, I'm not sure if this is only applying to me or other people as well but I haven't been able to find any info to get a fix. I tried multiple different indicator codes as well, none are working.

no valid values warning

Hello, first, thanks for this amazing package, i'm getting this problem. I'm trying to get information about:
Export unit value index (2000 = 100)

When i try to get information and save it to a data frame called "df":

df <- wb(country = "NI",
         indicator = "TX.UVI.MRCH.XD.WD")

i get this message:

# Error in wb(country = "NI", indicator = "TX.UVI.MRCH.XD.WD") : 
# indicator parameter has no valid values. Please check documentation for valid inputs

This only happen with this kind of index:
TX.UVI.MRCH.XD.WD or TM.UVI.MRCH.XD.WD
but if i try something like:

df <- wb(country = "countries_only",
         indicator = "NY.GDP.MKTP.CD")

i get no errors.

Thanks in advance.

wbstats Version 0.2
R 3.5.2

PD: Sorry for my English!.

ability to return data frames as data tables

maybe an as_dt = TRUE option added to the functions

attr(column, 'label')

If you assign an indicator's description to its column's label attribute, RStudio will display it in the data browser.

For example,

attr(dat[['NY.GDP.PCAP.KD']], 'label') <- 'GDP per capita (constant 2010 US$)'

Gives something like this:

Is it possible to use this package to download microdata?

For example, this one: https://microdata.worldbank.org/index.php/catalog/4509

If so, can you give an example please?

add sourceNote and topics to indicators return data frame

As seen in this example call

http://api.worldbank.org/v2/en/indicators?per_page=1&format=json

corrupted Rdata.rdb?

hi. i just installed.

> install.packages("wbstats")
Installing package into ‘/home/minshall/usr/lib/R/x86_64-pc-linux-gnu-library’
(as ‘lib’ is unspecified)
trying URL 'https://cran.ncc.metu.edu.tr/src/contrib/wbstats_1.0.4.tar.gz'
Content type 'application/x-gzip' length 989355 bytes (966 KB)
==================================================
downloaded 966 KB

* installing *source* package ‘wbstats’ ...
** package ‘wbstats’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (wbstats)

The downloaded source packages are in
        ‘/tmp/RtmpLyRyAk/downloaded_packages’
> require("wbstats")
> d <- wb_data("SP.POP.TOTL")
Error in wb_data("SP.POP.TOTL") :
  lazy-load database '/home/minshall/usr/lib/R/x86_64-pc-linux-gnu-library/wbstats/data/Rdata.rdb' is corrupt
In addition: Warning message:
In wb_data("SP.POP.TOTL") : internal error -5 in R_decompress1

session info:

> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Arch Linux

Matrix products: default
BLAS:   /usr/lib/libopenblasp-r0.3.15.so
LAPACK: /usr/lib/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_GB.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] wbstats_1.0.4     data.table_1.14.1

loaded via a namespace (and not attached):
 [1] compiler_4.1.0  ellipsis_0.3.2  magrittr_2.0.1  tools_4.1.0
 [5] pillar_1.6.0    tibble_3.1.2    crayon_1.4.1    utf8_1.2.1
 [9] fansi_0.4.2     vctrs_0.3.8     lifecycle_1.0.0 pkgconfig_2.0.3
[13] rlang_0.4.11

WBSTATS - Can't get Oil Data?

Hi All,
I ran the devtools::install_github("GIST-ORNL/wbstats")
but still cant get CRUDE_BRENT data? could you please help?
"oil_data <- wb(indicator = "CRUDE_BRENT", mrv = 10, freq = "M", POSIXct = TRUE)"
"No data was returned for any requested country and indicator. Returning empty data frame"

Thank you

Questions about installing this package?

I am having problems installing this package, it said "Permission denied." How could I fix this problem? Thank you!

Issue with wbstats 1.0.1 when using RStudio Version 1.3.1073

I used the older version of wbstats with older versions of RStudio and they worked well together. However, when I upgraded both recently, I got the following error message:
Error: package or namespace load failed for ‘wbstats’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]): namespace ‘rlang’ 0.4.6 is already loaded, but >= 0.4.7 is required

Could you please help me to resolve this issue because I am really stuck? I cannot go back to the old versions of wbstats and RStudio.

Thank you!

Successful API call but error due to missing json-format

Currently, I get an error message when trying to access the WB data:

library(wbstats) 
packageVersion("wbstats")
#> [1] '1.0.0'

# Population for every country from 1960 until present
d <- wb_data("SP.POP.TOTL")
#> Error: API call executed successfully, but did not return expected json format

Is the error due to changes to the World Bank Data API or an issue with the package?

Thanks for your help and for maintaining such an amazing package.

"Error: API call executed successfully, but did not return expected json format" when using wb_stat to download data

Hello,

when trying to download indicators using the command wb_data I receive the following error:
Error: API call executed successfully, but did not return expected json format

For instance when I try to reproduce the line of code available in the help:
wb_data("NY.GDP.MKTP.CD")

wbstats::wbdatacatalog() triggers Error: API call executed successfully, but did not return expected json format

Hi all:

Thank you very much for this super useful package. It seems that since Tuesday (Feb, 8) the World Bank API acts up as at least for me calling wbdatactalog() yields

packageVersion("wbstats")
[1] ‘1.0.5.9000’
wbstats::wbdatacatalog()
Error: API call executed successfully, but did not return expected json format

For what it is worth the related API call (http://api.worldbank.org/v2/datacatalog?format=json) reports a data not found error (160) so it does not appear to return successfully.

Thanks for looking into this!

Joachim

no vignettes?

hi. thanks so much for this! but... :)

the ?wbstats output includes

     To learn more about the wbstats package, start with the vignettes:
     ‘browseVignettes(package = "wbstats")’

however,

> browseVignettes(package = "wbstats")
No vignettes found by browseVignettes(package = "wbstats")

cheers!

error in wb_data() with return_wide=FALSE and mrnev=1

i tried a small permutation or two from the (online!) vignette:

> gdp_capita <- wb_data("NY.GDP.PCAP.CD", mrv = 1)
> gdp_capita <- wb_data("NY.GDP.PCAP.CD", mrnev = 1)
> gdp_capita <- wb_data("NY.GDP.PCAP.CD", return_wide=FALSE, mrv = 1)
> gdp_capita <- wb_data("NY.GDP.PCAP.CD", return_wide=FALSE, mrnev = 1)
Error: Can't subset columns that don't exist.
✖ Column `unit` doesn't exist.
Run `rlang::last_error()` to see where the error occurred.

Questions about

Hello, I am not sure if this is the right place to ask this question, but if you could provide me with some advice, that will be awesome. I want to ask how to download data using this package? Thank you!
edu <- wb_data("GC.XPN.TOTL.GD.ZS", start=1990, end=2019) Error in curl::curl_fetch_memory(url, handle = handle) : Timeout was reached: [api.worldbank.org] Operation timed out after 20008 milliseconds with 1359395 out of 1770108 bytes received

progress info for "wb"?

When I use wb to fetch data, sometimes it takes a lot of time and I don't know if it is working or not. As I understand, the function wb fetches the data indicator by indicator, so it can present the progress of how much of the data is fetched.

How to download all the data for the Doing Business Report (all series, all countries, all years)?

I am trying to download all the data for the Doing Business Report (all series, all countries, all years).

The DBR data source (http://www.doingbusiness.org/Custom-Query) is very cumbersome, exporting the data in Excel with weird labels (sometimes with line breaks within the label) instead of short varnames, etc.

Also, is there some mapping (variable dictionary) between the DBR labels (as they apear the link above) and the series IDs (as provided by the API and here)?

Feature request: Ability to specify the `source` paramter

The v2 API allows users to send a query against a specific database using the source parameter as in the example below:

https://api.worldbank.org/v2/country/all/indicator/SG.LAW.INDX?source=14&date=2022&format=json

I don't think this is currently supported in wbstats.

This is important for cases where different versions of the same indicator may be available in two different database, like in the example below:

Most recent data point in WDI is from 2021
https://api.worldbank.org/v2/country/all/indicator/SG.LAW.INDX?source=2&format=json
Most recent data point in gender Statistics is from 2022
https://api.worldbank.org/v2/country/all/indicator/SG.LAW.INDX?source=14&format=json

how do I see the list of possible fields for a wbsearch(pattern = "x", fields = "y") search?

duplicated results in `wbsearch`

I get some duplicated results when I search using a simple regex.

library(wbstats)
a <- wbsearch('gdp.*capita')
a <- duplicated(a$indicatorID)
sum(a)
#> [1] 15

^{Created on 2019-02-04 by the reprex package (v0.2.1)}

Not working with numeric indicators

I get

r$> wb_data("9080000")
Error: World Bank API request failed for indicator 9080000 The following message was returned from the server
id: 175
key: Invalid format
value: The indicator was not found. It may have been deleted or archived.

failed request:
https://api.worldbank.org/v2/en/country/all/indicator/9080000?footnote=y&cntrycode=y&per_page=20000&format=json

But the indicator definitely exists:

9080000,9080000:ACTUAL HEALTH,,,,,78,ICP 2017

Downloading data with wb_data()

Hi @jpiburn, I noticed that the wb_data() function is no longer available in the current version of the wbstats package.

However, I have been exploring other functions in the wbstats package and none is currently performing the same function as the wb_data() function.

So, is there a reason why the wbstats package no longer supports the wb_data() function?

Indicator info for headcount ratios needs to be updated

When I download headcount ratios I am seeing data updated with the new poverty thresholds, but the info on the indicators from wb_cachelist seems to be the old thresholds. For example,

poverty <- "SI.POV.DDAY"
indicator_info <- filter(wb_cachelist$indicators, indicator_id == poverty)
indicator_info$indicator

yields "Poverty heacount ratio at $1.90 a day (2011 PPP) (% of population)"

Add timeout option

The function wb_data currently does not have an option to increase timeout.

Debt Statistics

Hi,

I am trying to download debt statistics, and in particular statistics for the external public debt (% of GDP)

Sadly, the indicator - which seems correct to me - does not work. Here is a sample of my code:

new_cache <- wbcache()
ind_wb<-wbsearch(pattern = "External Creditors", cache = new_cache,  extra = TRUE)

# debt according to WB: http://databank.worldbank.org/data/views/reports/ReportWidgetCustom.aspx?Report_Name=T1-Gross-General-Gov-GDP-percent&Id=be9dfffc                      
# 476.General Govt. Public Sector Debt, Domestic creditors(% of GDP): DP.DOD.DECD.CR.GG.Z1
# 477.General Govt. Public Sector Debt, External creditors(% of GDP): DP.DOD.DECX.CR.GG.Z1

# download total debt/GDP 
debt_wb <-wb(indicator = c("DP.DOD.DECX.CR.GG.Z1"), mrv=50, gapfill = TRUE )

Timeout was reached: [api.worldbank.org] Operation timed out

I've been unable to download data using the wb_data() function in R (v 4.4.0).

It was working fine until recently, so is perhaps because the WB API is down(?). However, I could find nothing on the World Bank's website to suggest this was the case.

When I run the code below I receive the error:

"Error in curl::curl_fetch_memory(url, handle = handle) :
Timeout was reached: [api.worldbank.org] Operation timed out after 20003 milliseconds with 0 bytes received"

I've tried changing my internet connection and increasing the timeout.

`library(wbstats)

my_indicators <- c("SP.POP.TOTL",
"NY.GDP.MKTP.CD",
"SL.UEM.TOTL.ZS",
"SP.DYN.CBRT.IN")

df <- wb_data(my_indicators)`

Error in getting data

wbstats::wb(indicator = "SH.XPD.PVTD.CH.ZS",
startdate = 2014, enddate = 2019,
removeNA = TRUE)

Error in wbstats::wb(indicator = "SH.XPD.PVTD.CH.ZS", startdate = 2014, :
indicator parameter has no valid values. Please check documentation for valid inputs

gshs-ornl / wbstats Goto Github PK

wbstats's People

Contributors

Stargazers

Watchers

Forkers

wbstats's Issues

Recommend Projects

Recommend Topics

Recommend Org