support sftp

[crazy-max]

support sftp

I have used this library in the past to do sftp in one of my projects.

https://github.com/pkg/sftp

Might be of some use.

That does indeed make it work. Thank you for taking the time to add sftp!

[crazy-max]

Hi @gregf, yes previously it was available through cURL. I completely forgot about it 😓. I will add support for SFTP on the next release and also bring support for FreeBSD ;)

Thanks for your input!

[crazy-max]

@gregf I have introduced SFTP support in the latest beta. Pubkey auth is not available for the moment. Can you give me your input? Thanks!

ftpgrab --config=/home/gregf/.ftpgrab/tv.yml       1 ↵  09:14:09 
Thu, 14 Feb 2019 14:15:34 UTC INF Starting FTPGrab dev
Thu, 14 Feb 2019 14:15:34 UTC FTL Cannot load configuration error="unable to decode into struct, yaml: unmarshal errors:\n  line 17: cannot unmarshal !!str `gregf` into int\n  line 18: cannot unmarshal !!str `gregf` into int"

server:
  type: sftp
  sftp:
    host: example.com
    port: 22
    username: gregf
    password: super_secret_password
  sources:
    - /home/gregf/downloads/tv

db:
  enable: true
  path: ~/.ftpgrab/tv.db

download:
  output: /mnt/media/tv/unsorted
  uid: gregf
  gid: gregf
  chmod_file: 0644
  chmod_dir: 0755
  include:
  exclude:
  since: 0001-01-01T00:00:00Z
  retry: 3
  hide_skipped: false
  create_basedir: false

mail:
  enable: false
  host: localhost
  port: 25
  ssl: false
  insecure_skip_verify: false
  username:
  password:
  from:
  to:

[crazy-max]

@gregf uid and gid must be an integer (root > 0) and the indentation of sources is not good either :

server:
  type: sftp
  sftp:
    host: example.com
    port: 22
    username: gregf
    password: super_secret_password
    sources:
      - /home/gregf/downloads/tv

I tested in filezilla first and it logs in fine, with ftpgrab I get this error.

https://stackoverflow.com/questions/44269142/golang-ssh-getting-must-specify-hoskeycallback-error-despite-setting-it-to-n

[crazy-max]

For the moment you must set insecure_skip_verify: true to FTPGrab configuration:

server:
  type: sftp
  sftp:
    host: example.com
    port: 22
    username: gregf
    password: super_secret_password
    sources:
      - /home/gregf/downloads/tv
    insecure_skip_verify: true

I will push a fix asap to fix that.

should update the ftpgrab.yml in the basedir so people know the new options. I just noticed the updated beta1.

[crazy-max]

@gregf Already pushed in feature/sftp branch ;) I'm updating the doc.

Thu, 14 Feb 2019 19:24:55 UTC FTL Improper configuration error="host is required"

server:
  type: sftp
  sftp:
    host: example.com
    port: 22
    username: gregf
    password:
    key: /home/gregf/.ssh/id_ed25519
    sources: /home/gregf/downloads/tv
    timeout: 30
    max_packet_size: 32768
db:
  enable: true
  path: /home/gregf/.ftpgrab/tv.db

download:
  output: /mnt/media/tv/unsorted
  uid: 1001
  gid: 1001
  chmod_file: 0644
  chmod_dir: 0755
  include:
  exclude:
  since: 0001-01-01T00:00:00Z
  retry: 3
  hide_skipped: false
  create_basedir: false

mail:
  enable: false
  host: localhost
  port: 25
  ssl: false
  insecure_skip_verify: false
  username:
  password:
  from:
  to:

[crazy-max]

@gregf Wait for 5.1.0 and keep me in touch, I think you are using outdated sources.