stenevang/sftp

R-package with convenience functions for SFTP operations wrapping RCurl functions

sftp's Issues

sftp_listdirs typo

Thanks for the useful package! There's a small typo in the sftp_listdirs function causing an error. I corrected it in my environment and fixing the typo fixed the issue.

sft_connection should read sftp_connection below in your function.

sftp_listdirs <- function(sftp_connection = sftp_con,
verbose = TRUE,
curlPerformVerbose = FALSE,
recurse = FALSE) {

final <- sftp_list(sftp_connection = sft_connection, ...

Thanks!

Timeout error in large list file transfer

Hi guys,

I have a list of about 600 files to transfer from multiple subfolders in an sftp server.

I can usually use sftp for this so far without issue. Today though I've been having failures to transfer with error code :

(I've obviously reducted connection info for security reasons sorry)
Error in function (type, msg, asError = TRUE) : Failed to connect to server_address_reducted port REDUCTED: Timed out

My code looks like this:

`sftp_con <- sftp_connect(server = "",
folder = "",
username = "",
password = "",
protocol = "sftp://" ,
port = "")

sftp_makedir("./____/Data/")
sftp_changedir("./ModelData/Data/")

done <- pblapply(split(upload_fileList, upload_fileList$SN, drop = T), function(filedata){

sftp_upload(file = filedata$filename, fromfolder = filedata$ParentFolder )

out <- 1
return(out)

} )
print(paste("Done"))`

Using the opportunity, I feel it'd be much faster to feed the function a list of strings the string being the complete filepath of the file like C:/myfile/myfile.csv

Any ideas how I can correct my timeout error?

Thanks in advance !

Estimated timeline for keyfile implementaiton

Any idea when you might support SSH keyfile implementations?

sftp for R 4.2.1?

Hello, wondering if there's any plans for this package to support R 4.2.1? Thanks!

Add check for sftp protocol support in curl installation

if !("sftp" %in% RCurl::curlVersion()$protocols) stop("Your systems curl installation does not have sftp support. Ask Google for the rest.")

Protocol "sftp" not supported or disabled in libcurl

Trying to upload with sftp_upload().

Get the error:

 Error in function (type, msg, asError = TRUE)  : 
     Protocol "sftp" not supported or disabled in libcurl

System: OSX 12.1 (Monterey)
R: 4.1.2
sftp: 2.0.10
curl: 4.3.2

sftp_list Error in function (type, msg, asError = TRUE)

Hi There,

I updated the sftp package and I am now getting the error message Error in function (type, msg, asError = TRUE) when trying to list files recursively in a folder on and sFTP site. This was working properly before the update.

below is the code I am running that results in the error:

sftp_con <- sftp_connect(server = server,
                         username = username ,
                         password = password ,
                         folder = 'folder')

sftp_list(sftp_con, recurse = T, verbose = F, type = 'f')

If I try to list the files in a sub directory it does work:

sftp_con <- sftp_connect(server = server,
                         username = username ,
                         password = password ,
                         folder = 'folder/subfolder')

sftp_list(sftp_con, recurse = T, verbose = F, type = 'f')

Unspecified error on upload

Trying to upload, and I get the following (this is on a different system than my other issue):

Upload from folder: xxx
SFTP destination: xxx
2 file(s) to upload.
Error in function (type, msg, asError = TRUE)  : 

There is no other error info. I enabled logging, and nothing is in the log.

Same thing happens when trying sftp_list()

Retry if download fails

Is there a way to retry to download a file if sftp_download fails the first time?

support for custom RCurl / cURL options

Extending the API to support passing of custom RCurl::curlOptions objects would be useful

upload fails when fromfolder is absolute on linux/mac due to leading slash removal

When using an absolute path starting from root on mac/linux (Ie one that starts with /), the sftp_upload() function immediately calls the internal function trim_slashes(fromfolder), which removes leading and trailing forward slashes.

This causes the upload to fail for paths starting from root, because the path without the leading slash assumes that it is starting from the current working directory.

sftp_list fails with "error/vctrs_error_incompatible_type" error

Hello!

Thank you for the excellent library.
Facing an issue when I am listing SFTP content. It seems to be happening when the last folder in the SFTP directory is empty.

Command:

sftp::sftp_list(
    sftp_connection = sftp_con_i01_v01,
    verbose = T,
    curlPerformVerbose = T,
    recurse = T
  )

Error

> rlang::last_trace()
<error/vctrs_error_incompatible_type>
Can't combine `..1$type` <character> and `..2$type` <logical>.
Backtrace:
     x
  1. \-sftp::sftp_list(...)
  2.   \-sftp::sftp_list(sftp_connection = d_con, recurse = T)
  3.     +-dplyr::bind_rows(df2, d_list) %>% dplyr::arrange(name)
  4.     | \-base::eval(lhs, parent, parent)
  5.     |   \-base::eval(lhs, parent, parent)
  6.     \-dplyr::bind_rows(df2, d_list)
  7.       \-vctrs::vec_rbind(!!!dots, .names_to = .id)
  8.         \-(function () ...
  9.           \-vctrs::vec_default_ptype2(...)
 10.             \-vctrs::stop_incompatible_type(...)
 11.               \-vctrs:::stop_incompatible(...)
 12.                 \-vctrs:::stop_vctrs(...)

sftp log uses deprecated `write_lines()`

When specifying a log file, a deprecation warning is issued:

Warning message:                                                                                                                               
The `path` argument of `write_lines()` is deprecated as of readr 1.4.0.
Please use the `file` argument instead.

Add setting for connecttimeout

Add new parameter "timeout" to the function sftp_connect().
The value of "timeout" will specify the time in seconds to set as timeout limit for RCurl when making connections in other functions of the sftp package. "timeout" will be another object in the list that is returned by sftp_connect(), so it can be referenced from the other functions and specified as an argument to the RCurl functions used.
RCurl function calls that need the connecttimeout option added:
RCurl::getBinaryURL
RCurl::getURL
RCurl::curlPerform
RCurl::ftpUpload

Issue with getting SFTP data into R

I'm having issues using the SFTP component of this package. Allow me to demonstrate.

If I enter my credential information:

sftp_con <- sftp_connect(server = "sftp.name.com",
                          username = "username",
                          password = "password",
                          protocol = "sftp://",
                          port = 2222)

sftp_download("*")

I am met immediately with the following error message:

Error in function (type, msg, asError = TRUE)  :
  Protocol "sftp" not supported or disabled in libcurl

Is there a way around this? If it helps, I'm using a MacBook to run this.

abrupt in downloading using sftp_download

Hi, Stenevang:

First, thanks for creating this powerful and easy-to-use package. I like sftp a lot. By the way, I am Sam from University of Illinois.

My question is that when I set up sftp_download to download files from a SFTP server, it abruptly stopped after successfully downloaded first 2-3 files, as shown below:

sftp_con <- sftp_connect(server = "sftp://quagmire.igb.illinois.edu/",
                         folder = "home/shared/licor_data/maize/raw/",
                         username = Sys.getenv("MY_SFTP_USER"),
                         password = Sys.getenv("MY_SFTP_PASS"),
                         protocol = "sftp://",
                         port = 22)

files_list <- 
  sftp_list(sftp_connection = sftp_con,
                               verbose = TRUE,
                               curlPerformVerbose = FALSE,
                               type = "file")

sftp_download(files_list$name, tofolder = "content/post/eddy_covariance/data/energy_farm/maize")

and here is the error:

SFTP source: sftp://quagmire.igb.illinois.edu/home/shared/licor_data/maize/raw/
Save to folder: content/post/eddy_covariance/data/energy_farm/maize
100 file(s) to download.
2019-12-07T013000_AIU-0496_maize.ghg downloaded
2019-12-07T020000_AIU-0496_maize.ghg downloaded
Error in function (type, msg, asError = TRUE) :
Failed to connect to quagmire.igb.illinois.edu port 22: Timed out

Publishing to CRAN

Hi, this package seems like it fills a hole in the existing R ecosystem. Have you considered publishing it to CRAN?