Large NHANES files cause lodown() to hang [SOLVED]

[dwinsemius]

Large NHANES files cause lodown() to hang [SOLVED]

[ajdamico]

hi! are you able to isolate the reason behind that timeout? i get

> system.time( download.file( "https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT" , tempfile() , mode = 'wb' ) )
trying URL 'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT'
Content type 'application/octet-stream' length 71600960 bytes (68.3 MB)
downloaded 68.3 MB

   user  system elapsed 
   1.10    1.64   66.29 

we could surely edit https://github.com/ajdamico/lodown/blob/master/R/cachaca.R but i'm a bit confused why your libcurl defaults to only one minute? :-/

thanks! hope you are excellent

[dwinsemius]

On Feb 7, 2022, at 10:21 PM, Anthony Damico ***@***.***> wrote: hi! are you able to isolate the reason behind that timeout? i get > system.time( download.file( "https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT" , tempfile() , mode = 'wb' ) ) trying URL 'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT' Content type 'application/octet-stream' length 71600960 bytes (68.3 MB) downloaded 68.3 MB user system elapsed 1.10 1.64 66.29 we could surely edit https://github.com/ajdamico/lodown/blob/master/R/cachaca.R but i'm a bit confused why your libcurl defaults to only one minute? :-/ thanks! hope you are excellent — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

Here's my Mac experiment:

-- Best Regards; David.

options('timeout')

$timeout [1] 60

system.time( download.file( "https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT" , tempfile() , mode = 'wb' ) )

trying URL 'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT' Content type 'application/octet-stream' length 71600960 bytes (68.3 MB) ===================================== downloaded 51.3 MB Error in download.file("https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT", : download from 'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT' failed In addition: Warning messages: 1: In download.file("https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT", : downloaded length 53763602 != reported length 71600960 2: In download.file("https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT", : URL 'https://wwwn.cdc.gov/Nchs/Nhanes/1999-2000/DRXIFF.XPT': Timeout of 60 seconds was reached Timing stopped at: 1.455 1.659 60.01 Hope this is useful.(It's not the most recent version of RCurl since I am having some problems with gfortran versions. I've got version 10 and the source code seems to expect version 7.) #-------------> sessionInfo() R version 4.1.1 (2021-08-10) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS Big Sur 11.6 Matrix products: default BLAS: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.0.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] devtools_2.4.3 usethis_2.1.5 loaded via a namespace (and not attached): [1] magrittr_2.0.2 pkgload_1.2.4 R6_2.5.1 rlang_1.0.1 fastmap_1.1.0 [6] tools_4.1.1 pkgbuild_1.3.0 sessioninfo_1.2.2 cli_3.1.1 withr_2.4.3 [11] ellipsis_0.3.2 remotes_2.4.2 rprojroot_2.0.2 lifecycle_1.0.1 crayon_1.4.2 [16] processx_3.5.2 purrr_0.3.4 callr_3.7.0 fs_1.5.2 ps_1.6.0 [21] curl_4.3.2 testthat_3.1.1 memoise_2.0.1 glue_1.6.1 cachem_1.0.6 [26] compiler_4.1.1 desc_1.4.0 prettyunits_1.1.1

…

[ajdamico]

hi! apologies for the long delay. i've made a couple of big updates to asdfree.com that hopefully make the website a bit better, but i've decided to stop maintaining the lodown package so probably won't fix the bug you've reported. the new asdfree does have nhanes data, but only for the most current year. thanks and hope you are great