Giter Site home page Giter Site logo

bash-onedrive-upload's Introduction

bash-onedrive-upload

Upload files to Microsoft OneDrive via linux command line using the REST API.

Use git clone --recursive to checkout the repository including all required submodules.

If you download the contents of this repository as .zip, you also need to manually download the bash-json-parser and extract it into ./libs/json, because submodules are not included in the ZIP file.

Prerequisites

  • curl is used for accessing the API.
  • grep is used for filtering the parsed json output.
  • cut is used for value extraction from the filtered json output.
  • xargs is used for multi-threaded file uploads.
  • dd is used for chunked file uploads.
  • stat is used to determine the filesize.

Getting started (OneDrive Personal)

Before you can use this tool, create an application in the https://apps.dev.microsoft.com/#/appList/create/sapi to generate your custom Client ID and Client secret. Notice, that you need to add a new platform of type Mobile application. If you have not created Live SDK apps before you will not see menu to add it, because they are deprecated, instead use link above (enter app name & press "create application").

When you are already logged in into your Microsoft account, you will be redirected to the right form. In case you sign in first, pay attention to create your application in the Live SDK applications section, not in the Converged applications section, as the newer application type is not (yet) supported by onedrive-authorize. Please also ensure that the Application ID (Client ID) does not contain any dashes.

Afterwards your overview should show your freshly created credentials in the form

Application ID: 00000000A2B3C495
Applications secrets: qOFCYaZKjm6e13aq3fdGiNz

Please insert these values in the matching variables in onedrive.cfg:

export api_client_id="00000000A2B3C495"
export api_client_secret="88qC3kX2Cbd0tXV2sBnqYbS321abcDEF"

After the initial configuration you must authorize the app to use your OneDrive account. Run

$ ./onedrive-authorize

and follow the steps. You will need a web browser.

After the authorization process has successfully completed you can upload files.

Getting started (OneDrive for Business)

Before you can use this tool, create an application in the Microsoft Azure Management Portal to generate your custom Client ID and Client secret.

1. Navigate to Active directory
2. Select your Directory
3. Navigate to Applications tab
4. Press Add
5. Select "Add an application my organization developing"
6. Choose Name and select "Web application and/or web API"
7. In a "Sign-on URL" write https://login.microsoftonline.com/
8. In an "APP ID URI" write https://onedrive.live.com/about/business/
9. Submit form
10. Now open your new application and select Configure tab
11. Change "Reply URL" field to https://onedrive.live.com/about/business/
12. Add an application key. Select 1 year key, 2 years key will not work. Press Save and copy the generated Secret
13. Also copy your "Client ID"
14. In a "Permissions to other applications" press "Add application"
15. Select "Office 365 SharePoint Online" and "Windows Azure Active Directory"
16. Grant "Sign in and read user profile" delegated permission to "Windows Azure Active Directory"
17. Grant "Read user files" and "Read and write user files" delegated permissions to "Office 365 SharePoint Online"
18. Save changes

Afterwards you should have three components of authorization process:

Client ID: a66d1076-4c04-4a33-3bb2-2578c4891886
Secret: x73LaIRCimtjiiw/cuHA000Ozwcf4nz1Ovcpi0xUCHI=
Reply URL: https://onedrive.live.com/about/business/

Please insert these values in the matching variables in onedriveb.cfg:

export api_client_id="a66d1076-4c04-4a33-3bb2-2578c4891886"
export api_client_secret="x73LaIRCimtjiiw/cuHA000Ozwcf4nz1Ovcpi0xUCHI="
export api_reply_url="https://onedrive.live.com/about/business/"

After the initial configuration you must authorize the app to use your OneDrive for Business account. Run

$ ./onedriveb-authorize

and follow the steps. You will need a web browser.

After the authorization process has successfully completed you can upload files. If you see error AADSTS70002: Error validating credentials. AADSTS50012: Invalid client secret is provided, try to add a new key and use new secret.

Usage

To upload a single file simply type

# OneDrive Personal
$ ./onedrive-upload file1

# OneDrive for Business
$ ./onedriveb-upload file1

You can also upload multiple files, either by explicitly specifying each one

# OneDrive Personal
$ ./onedrive-upload file1 file2

# OneDrive for Business
$ ./onedriveb-upload file1 file2

or just use wildcards (globbing)

# OneDrive Personal
$ ./onedrive-upload file*.png

# OneDrive for Business
$ ./onedriveb-upload file*.png

It is also possible to recursively upload a whole folder

# OneDrive Personal
$ ./onedrive-upload /path/to/folder

# OneDrive for Business
# Not yet supported

If this folder contains hidden files (files starting with a dot) and you want to include them, just type

# OneDrive Personal
$ ./onedrive-upload --dotfiles /path/to/folder

# OneDrive for Business
# Not yet supported

You can also specify a destination folder relative to the root folder configured in onedrive.cfg:

# OneDrive Personal
$ ./onedrive-upload -f "relative/path" file

# OneDrive for Business
$ ./onedriveb-upload -f "relative/path" file

This command will automatically determine all of the needed folder ids and recursively create all subfolders that do not yet exist.

If you need your file to be uploaded with a different filename, you can activate the renaming mode:

# OneDrive Personal
$ ./onedrive-upload -r ./file1.txt renamed_file1.txt ./file2.txt renamed_file2.txt

# OneDrive for Business
$ ./onedriveb-upload -r ./file1.txt renamed_file1.txt ./file2.txt renamed_file2.txt

Be aware that for each file you specify you must provide the remote filename as the subsequent parameter. This feature can lead to an unexpected behavior when combined with wildcards (globbing) because the pathname expansion is performed by bash before the execution of the script. Also do not use this when recursively uploading folders.

Configuration

Specify an alternate root folder for uploads (OneDrive Personal)

If you want to use a folder other than the root folder of your OneDrive as your upload root folder, just enter the human readable, slash separated path into

export api_root_folder="/path/to/folder"

You can also use any of the special folders provided by the API by simply using the combination

export api_drive_resource="special/approot"
export api_root_folder="/path/relative/to/special/folder"

Specify an alternate root folder for uploads (OneDrive for Business)

If you want to use a folder other than the root folder of your OneDrive as your upload root folder, you need to retrieve its unique id by using this approach:

Upload in debug mode any file to non-existent folder, which will be used as root folder.

./onedriveb-upload -d -f Backup testfile
2016-08-26 09:33:56 Searching for 'Backup' in ''
2016-08-26 09:33:56 Creating folder 'Backup' in ''
2016-08-26 09:33:58 api_folder_id is now '01KB37ZVWEMG6F4SS6TVGZAM7ACZKOEERY'
2016-08-26 09:33:58 Size of testfile is less than or equal to 104857600 bytes, will use simple upload
2016-08-26 09:34:01 Uploading 'testfile' as 'testfile' into 01KB37ZVWEMG6F4SS6TVGZAM7ACZKOEERY

Copy `api_folder_id` and place it in your `onedriveb.cfg`

export api_root_folder="items/01KB37ZVWEMG6F4SS6TVGZAM7ACZKOEERY"

You can also specify any of the special folders provided by the API by simply using

export api_root_folder="special/approot"
export api_root_folder="special/documents"
export api_root_folder="special/photos"
...

The default is to use

export api_root_folder="root"

which identifies the root folder of your account.

Keep in mind that this will be the new root folder of your OneDrive as seen by this script. The script does not support escaping the root by using ../.

Less permissions for uploads

If you don't want the script to have access to all your folders and files you can request less permissions during the authorization process. Simply change the value

export api_permissions="onedrive.readwrite"

to

export api_permissions="onedrive.approot"

and run the authorization process. You might need to remove the apps permissions prior to that (Link) if you have already requested onedrive.readwrite permissions.

Number of simultaneous uploads

If the script does not fully utilize your bandwidth, you can maybe speed things up a little bit by increasing the value of max_upload_threads.

When you start the upload of more than one file, the script will start up to max_upload_threads parallel uploads, but only one thread per file.

Configure threshold for session based upload

For small files the script uses the simple upload of the api. For files that are larger than 100 MiB the script uses the session based upload.

If you want to use the session based upload for files smaller than 100 MiB you can change the value of max_simple_upload_size to any positive value smaller than 104857600:

export max_simple_upload_size=52428800

You can also change the value of max_chunk_size to any positive value smaller than 62914560, if you want to use smaller or larger chunks than the default:

export max_chunk_size=62914560

Contributors

Thank you all for your valuable contributions.

bash-onedrive-upload's People

Contributors

evaldsurtans avatar fkalis avatar huming2207 avatar laufi avatar rootik avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bash-onedrive-upload's Issues

New application type not supported

Hi,

To test this script out I created myself a new OneDrive account.

When creating an application (at https://account.live.com/developers/applications) I don't see how to 'switch the API Setting "Mobile or desktop client app" to "yes"'. I can however add a platform and I've added a 'Mobile application' (the other option is 'Web') and this automatically shows a Redirect URI which is greyed out and unchangeable: urn:ietf:wg:oauth:2.0:oob

When running onedrive-authorize and pasting the shown URL into a browser I get the response:

https://login.live.com/oauth20_desktop.srf?error=invalid_request&error_description=The%20provided%20value%20for%20the%20input%20parameter%20%27redirect_uri%27%20is%20not%20valid.%20The%20client%20application%20must%20be%20marked%20as%20%27mobile%27%2c%20or%20the%20value%20must%20be%20an%20absolute%20URL%20which%20matches%20the%20registered%20redirect%20URI.&lc=2057

Since this URL itself seems to be an error message I changed the URL to remove the portion from '&redirect_uri=' onwards and that errors with:
https://login.live.com/err.srf?lc=2057#error=invalid_request&error_description=The+provided+request+must+include+a+%27redirect_uri%27+input+parameter.

I then changed the URL so the redirect URL was the one shown (by default) against my application and there is no response (just see a blank page with the original URL)

Any ideas as to how to resolve. I'm also interested in uploading files from Linux to SharePoint Online (part of Office365) so any idea whether the same mechanism could be used for that.

Thanks, Nik

The provided request must include a 'client_id' input parameter.

I wrote the following Dockerfile, but got an error that a client_id parameter is required:

FROM debian:stretch-slim
RUN apt-get update \
    && apt-get upgrade -y \
    && apt-get install -y git curl

RUN git clone --recursive https://github.com/fkalis/bash-onedrive-upload.git
WORKDIR /bash-onedrive-upload

RUN ./onedrive-authorize

Output

Step 5/5 : RUN ./onedrive-authorize
 ---> Running in d19bb61d17ee
Please open the following URL in your browser and follow the steps until you see a blank page:
https://login.live.com/xxxxx

When ready, please enter the value of the code parameter (from the URL of the blank page) and press return
Something went wrong, here is the API response:
{"error":"invalid_client","error_description":"The provided request must include a 'client_id' input parameter."}
 ---> 1d51c684d675

Seems like Microsoft made some changes in their API?

Error when uploading big files

When uploading file chunks, I receave this menssage:

No refresh token received from API. Please try again or re-authorize.

I was able to upload 1 file after try 6 times. The file have 6 GB. It appears to be caused by connection or something. There is a way to fix this ?

ability to work with scope onedrive.appfolder

I created the permissions but couldnt seem to get the /drive/special/approot to work.. Guessing more api changes?

Be handy for permissions to only allow the upload script access to its own folder.

Specifying different remote filename

Using
./onedrive-upload -f "remotedir/remote filename.txt" localtemp.txt

Uploads file localtemp.txt to folder
"remotedir/remote filename.txt/localtemp.txt"

Any workarounds to specify remote filename without renaming local?

Illegal xargs error shown in Mac OS X

Hi,

Firstly, thanks for your script, which helps me a lot!
But I've got an issue when I run it under OSX, it shown me like this:

JacksonHuMac:bash-onedrive-upload Jackson$ ./onedrive-upload -d -f "temporary" /Volumes/Hu/Photos/2015/xyz/*
2016-01-02 17:21:29 Acquiring new access token
2016-01-02 17:21:39 Searching for 'temporary' in root
2016-01-02 17:21:48 api_folder_id is now 'B*************6!5****5'
xargs: illegal option -- -
usage: xargs [-0opt] [-E eofstr] [-I replstr [-R replacements]] [-J replstr]
             [-L number] [-n number [-x]] [-P maxprocs] [-s size]
             [utility [argument ...]]

But when I ran the script with the same path and same files under Debian Linux in my virtual machine, it works pretty fine. Also I checked about some articles and someone said the xargs in OSX is not exactly the same as the one in Linux. Maybe that should be the problem, but I don't know how to fix it.

See: http://www.linuxforums.org/forum/newbie/193120-linux-command-macosx-options-xargs.html

Anyway, thanks and happy new year!

Regards,
Jackson

Refresh token not found

I get the token, but with a wierd "Something went wrong" error message, and I can't upload anything.

When ready, please enter the value of the code parameter (from the URL of the blank page) and press return
M32509d39-2b15-6e4b-7d7c-c493dac159dc
./onedrive-authorize: line 14: ./libs/json/bash-json-parser: No such file or directory
Something went wrong, here is the API response:
{"token_type":"bearer","expires_in":3600,"scope":"wl.offline_access onedrive.readwrite","access_token":"EwB4Aq1DBAAUGCCXc8wU/zFu9QnLdZXy+YnElFkAAY8QGX37jfSADQfgzfYClEbWQLpK7zLHBqBCeaAKoapnP4uDzrwVDxb8Xxsh3ky8WFU/RowP+Haq2oEKXzUeFkdLxCFCfGRvh8JADs+lvWDBLPndzRMu8u2hL2hq2Cv/udyqST00CCys2q+SontZW01FBnozHizHhi6a4X8WeteSsV1zA7TTj7P++yKnPjfKxeWYKHPVdkLjQG/aBhio7TZFDLPIzoAJlbd6UI/rFEoDpRo2Dbi1ObJhC1i73zBmHbUK53qEoEDYdSZUhdCEKdPMVxsArpnPIvhuTutb+jLQUYmF89L5m0qfN/dpmW726b0PWMJshxLXRWpWnUYHWRkDZgAACGkNzZgCkIHMSAH86ul90th7y3KwDSyGk7TaNrs2KfODPylhhK/8Goue973diSNp5TGaQNM332oP/GXjCjj1bVLhUcd1jcsqxRiYuskM/HsjF3iF9Q1fIhaNST+XaVfI/fx3+VPP1T5boJ2Jxw0wAlBeFvhLhgpoFIA0gRpVZfNT1L1o1/oau+lVtI4xvejGrmDr40tLRyBVKCP1MGZ2NwfTFYIGTb1YOpllu/tMWP9kP+x4WMSZgqL6nK5hCFS5A8Jf9x1tLbJC4ha8DafiB/KrNnSsa55Sm+oT9F+dUjgcr3QQpdtRvWUOhift5dw5sM73Sfm8PjrvYVt+UdqkCN7H1wNlp5L8WTdK8pK/msQiTRP8qvSqqpJPPiYx9hBOm61BelXTNXo2sR+ZOlgI0hWzaifzrsPFMauaZ8GXu06W7ZtE1J9CWHIm5l6CM2u4DBysbgE=","refresh_token":"MCc943ClnqoydWS5_w!NAMvGPjbvadYFh_Ji2UEVdtXPj0IL_nhsvY74KU7MVZ6BE3lW2oxR9kEV8iXDS4z467Raqo1TdcYZlFp9DA8v4zVNALVww5lOW3vmRZP3daZ5CpgqblmDrVucPsyqDw2jcvO05UrKN2A8R9rdmpyfuC5bth9NIzMQREh!6b4MqjoGYXxrfKEXS438Ng7GgJ1g1ZhqNMHxx5_3TXWsZ7LfxzMrkwrxKtG37DkI7EWjLmWZy93zvmE0MDe8D7A!HcHe01X73E6Ssiz53slfCdK_JtXSRmSRu_8IGJ9gm5XvXGJP6qIJsyd72FgMPi7r!bICYvMRQpqNHpJLShN0loRYQnkCX","user_id":"b570e8ef044b76bf8fd038fdb3cfdafe"}

Diman82@Givataim-PC /cygdrive/c/cygwin64/bash-onedrive-upload-master
$ ./onedrive-
onedrive-authorize onedrive-base onedrive-upload

Diman82@Givataim-PC /cygdrive/c/cygwin64/bash-onedrive-upload-master
$ ./onedrive-upload rsload.net.TeamViewer.7z
Refresh token not found, please complete the authorization process first
Refresh token not found, please complete the authorization process first
Refresh token not found, please complete the authorization process first
An error has occurred while uploading 'rsload.net.TeamViewer.7z' (Code: )

Support a custom destination folder in plain text

The (fixed) destination folder can be configured in the onedrive.cfg.

In same cases in might be useful to override this configuration for just one file.

Solution: Support configuration parameters.

Connection problems with Raspbian Jessie

When uploading a file, it fails with error code 0

onedrive-authorize seems to return a valid token
onedrive-upload return upload error code: 0

The same setup up works on linux mint. I added some debug code to onedrive-base
and every parameter seems ok. Could it be and encoding issue or a hidden character?

Thanks

Auth problem

I got this error when I try to authenticate, what am I doing wrong?

Please open the following URL in your browser and follow the steps until you see a blank page:
https://login.live.com/oauth20_authorize.srf?client_id=8d795e77-6f14-4313-90d5-276a80f211b3&scope=wl.offline_access%20onedrive.readwrite&response_type=code&redirect_uri=https://login.live.com/oauth20_desktop.srf

When ready, please enter the value of the code parameter (from the URL of the blank page) and press return
M2-xxxx-9
Something went wrong, here is the API response:
{"error":"invalid_request","error_description":"The provided request must include a 'code' input parameter."}

I opened the link, granted and the link was:

https://login.live.com/oauth20_desktop.srf?code=M-xxx-c9&lc=yyy

I copied the code and got the error

Support parallel uploads

When uploading more than one file it might speed things up a little bit if the script would not wait for one curl call to finish before starting the next upload.

Try xargs --num-procs=${max_num_threads}

How to use in whole filesystem

I put in my env path variable the work folder of the git, but only works if the file to upload is in the same folder of the onedrive-upload.
There is a way to use across the system? I want to use to "automate" some external backup from my machines.

Recursive uploads

Hi,
Would be great if tool will be able to upload a folder and its subfolders recursively and automatically creating remote folders.
Like cp folder_name target_folder_name copies a folder with all subfolders and files in them to a new destination.

Non-issue Issue - It Works Under Windows too!

I'm sure this isn't surprising to anyone, as there is really nothing exotic to speak of (and yet it works, which should be the goal of every coder out there ๐Ÿฅ‡ ). I used the directions in the README file with "Git Bash" (which is a special roll-up of MinGW that comes with Git pre-configured and is part of the "Git for Windows" package, although I'm sure other environments would work) and had zero issues with Personal of Business accounts.

Screenshot of MinGW/Git Bash in action

Error uploading big files

Hi,

thanks for this awesome tool! I got it working with smaller files (500 MB worked fine too). Now I want to upload larger files, the one in question is 4.4 GB. But I'm getting an unauthorized response. This only happens with large files. Any idea?
I am using the larger-files version of your tool.
Log is attached.
uploader.txt

Create shareable link to file just uploaded [feature request]

Looking through https://dev.onedrive.com/items/sharing_createLink.htm it seems like a flag could be added to generate a shareable link of the file just uploaded. Seems like capturing the ID from the response during upload could be used to then create the share link.

Something like:
`./onedrive-upload [-linkview, -linkedit, -linkembed] file1
Shell Response would just have the link:
https://onedrive.live.com/?cid=cidvalue&id=idvalue&parId=paridvalue&o=OneUp [fake example]

This would be very useful for screenshots.

Onedrive for Business support

The authorize process currently supports onedrive personal only, would this work with onedrive for business as well?

secret key expiration

I have successfully implemented a key for OneDrive for Business, where it uses key which has "never expires".

Sometimes existing folders are not found and creation fails

This is a follow-up to #12.

Sometimes it happens, that the folder traversel does not return the id of an existing folder. Therefore the script tries to create the folder and fails because the folder does already exist.

The folder traversel should rely on HTTP status codes instead of curl return codes.

Idea:

Simultanous retrieval of status code and content with curl:

curl -w "%{http_code}" -o >(cat >&1) 'http://example.com' 2>/dev/null | tail -n 1
curl -qfsw "%{http_code}" 'http://example.com' | tail -n1
curl -w "###HTTP_CODE###: %{http_code}\n" -o >(cat >&1) 'http://example.com' 2>/dev/null | grep "^###HTTP_CODE###: " | cut -d\  -f2
curl -qfsw "\n###HTTP_CODE###:%{http_code}" 'http://example.com' | grep "^###HTTP_CODE###:" | cut -d: -f2

and

curl -w "%{http_code}" -o >(cat >&1) 'http://example.com' 2>/dev/null | head -n-1
curl -qfsw "%{http_code}" 'http://example.com' | head -n-1
curl -w "###HTTP_CODE###: %{http_code}\n" -o >(cat >&1) 'http://example.com' 2>/dev/null | grep -v "^###HTTP_CODE###: "
curl -qfsw "\n###HTTP_CODE###:%{http_code}" 'http://example.com' | grep -v "^###HTTP_CODE###:"

Warning: head -n-1 does not work under OS X, using grep seems to be the most compatible solution.

Retry failed uploads

If there is a temporary problem with the internet connection when uploading many backup files, the upload of some files might fail.

If this happens it might be useful if the script waited for some seconds (e.g. 5 seconds) and retries the upload (up to 3 times).

Authorization does not work

When I open the URL given to me by ./onedrive-authorize, I get redirected to this URL:

https://login.live.com/oauth20_desktop.srf?error=invalid_request&error_description=The%20provided%20value%20for%20the%20input%20parameter%20%27redirect_uri%27%20is%20not%20valid.%20The%20client%20application%20must%20be%20marked%20as%20%27mobile%27%2c%20or%20the%20value%20must%20be%20an%20absolute%20URL%20which%20matches%20the%20registered%20redirect%20URI.&lc=1033

Code 416 - unable to upload

Hi,

I've used your newest release for a few days without problems. But since 2 days I get an error code 416 after a few chunks. Every retry fails with the same message. This happens to every big file I try to upload. Has there been any chances in the api since?

2016-04-10 02:18:10 Upload of chunk 41 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' finished (Code: 202) 2016-04-10 02:18:10 Acquiring new access token 2016-04-10 02:18:13 Uploading chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' 2016-04-10 02:18:13 Content-Length: 10158080 2016-04-10 02:18:13 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:28:34 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: ) 2016-04-10 02:28:34 Content-Length: 10158080 2016-04-10 02:28:34 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:29:20 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:29:20 Content-Length: 10158080 2016-04-10 02:29:20 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:29:40 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:29:40 Content-Length: 10158080 2016-04-10 02:29:40 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:29:59 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:29:59 Content-Length: 10158080 2016-04-10 02:29:59 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:30:17 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:30:17 Content-Length: 10158080 2016-04-10 02:30:17 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:30:37 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:30:37 Content-Length: 10158080 2016-04-10 02:30:37 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:30:56 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:30:56 Content-Length: 10158080 2016-04-10 02:30:56 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:31:14 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:31:14 Content-Length: 10158080 2016-04-10 02:31:15 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:31:33 Retrying upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:31:33 Content-Length: 10158080 2016-04-10 02:31:33 Content-Range: bytes 426639360-436797439/5000000000 2016-04-10 02:31:51 Upload of chunk 42 of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' finished (Code: 416) 2016-04-10 02:31:52 Retrying full upload of '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar' (Previous code: 416) 2016-04-10 02:31:52 Size of Daten_Daten_Backups_2016-04-04.part07.rar is more than 104857600 bytes, will use chunked upload 2016-04-10 02:31:52 Acquiring new access token 2016-04-10 02:31:55 Requesting upload session for '/media/hdd/backups/Daten_Daten_Backups_2016-04-04.part07.rar'

problems with auth

I followed the install instructions, installed curl, set up a new application, and entered the code to the .cfg file
I can navigate to the link and see the new application in my microsoft account permission page.
However the auth script returns an error message:

~/bash-onedrive-upload$ ./onedrive-authorize
Please open the following URL in your browser and follow the steps until you see a blank page:
https://login.live.com/oauth20_authorize.srf?client_id=00000000XXXX&scope=wl.offline_access%20onedrive.readwrite&response_type=code&redirect_uri=https://login.live.com/oauth20_desktop.srf

When ready, please enter the value of the code parameter (from the URL of the blank page) and press return

Something went wrong, here is the API response:
{"error":"invalid_request","error_description":"The provided request must include a 'code' input parameter."}

Delete old files after a successful upload (aka backup mode)

When uploading backups on a regular basis it is useful to delete old backups after a successful upload, e.g. doing backups on a daily basis and only keeping these for a week.

Things to think about:

  • How to activate this feature via commandline
    • for example --backupmode --max-age 7d
  • How to determine which files to delete
    • Checking for items in the backup base root

A full call could look like this

$ ./bash-onedrive-upload \
--backup-mode \                      # Activate backup mode
--backup-basedir myBackups \         # Use remote folder myBackups as the backup root folder
--backup-date-pattern '+%Y-%m-%d' \  # Create a subfolder with this name in the backup base dir
--backup-max-age 7d \                # Delete all files older than 7 days after a successful upload
myBackupFiles.gpg                    # Upload these files

Cannot upload to shared folder

When I try to upload file to shared folder (to which I have read-write access) I get one of those errors:

  • if folder ID (sometething like 1234567890ABCDEF!1234) is provided in config: "An error has occurred while uploading 'test.txt' (Code: 403)"
  • if path is provided via -f flag (folder is attached to my onedrive - it's visible as regular folder in root directory and can be synced with pc): "An error has occurred while uploading 'test.txt' (Code: 501)

Debug info below:

$ ~/bash-onedrive-upload/onedrive-upload -d -f "SFI" test.txt
2016-01-28 11:08:26 Acquiring new access token
2016-01-28 11:08:27 Searching for 'SFI' in root
2016-01-28 11:08:28 api_folder_id is now 'BBF1234567890ABC!304827'
2016-01-28 11:08:28 Size of test.txt is less than or equal to 104857600 bytes, will use simple upload
2016-01-28 11:08:28 Acquiring new access token
2016-01-28 11:08:29 Uploading 'test.txt' as 'test.txt' into BBF1234567890ABC%21304827
An error has occurred while uploading 'test.txt' (Code: 501)

$ nano ~/bash-onedrive-upload/onedrive.cfg
$ ~/bash-onedrive-upload/onedrive-upload -d test.txt #with folder ID in cfg
2016-01-28 11:09:21 Size of test.txt is less than or equal to 104857600 bytes, will use simple upload
2016-01-28 11:09:21 Acquiring new access token
2016-01-28 11:09:22 Uploading 'test.txt' as 'test.txt' into 1234567890ABCDEF%212217
An error has occurred while uploading 'test.txt' (Code: 403)

Non-Issue - Confused on the instructions for setting up onedrive for business

Hey Flasik, could you explain these directions to me more clearly? Sorry for the hassle, I work mostly with just linux command line, but this azure management portal is new to me and freakishly overwhelming.

  1. Navigate to Active directory---Do you mean the Azure Active Directory? That is the only thing on the left side panel and the only thing that pops up when I do a search.
  2. Select your Directory --What does this mean?
  3. Navigate to Applications tab --In the Azure Active Directory there is an App Registration, but there is no app tab.
  4. Press Add--There is no "add" button. At this point I think I am just in the wrong place. There is a new application registration button though?

feature/28-converged-applications little issue?

When I try to upload a file named "cn_office_enterprise_2007_DVD_VL_X12-19567.iso", it will get a code:404.
After I change the name to "cn-19567.iso", it works.
Maybe the underscore "_" or long name causes this issue.

I test to upload the file named "cnaofficeaenterprisea2007aDVDaVLaX12-19567.iso", and it also get a code:404.
So I think it is long name to cause this issue.

ispring_8_1_0.msi (code:202)--OK
ispring_suite_8_1_0.msi (code:404) --error

ispring.8.1.tar.gz(code:202)--OK
ispring.x32.8.1.tar.gz (code:404) --error

An error has occurred while uploading (Code: )

I'm getting the following error since a couple of days. This happens for different files, too.
Could the internet connection be the problem, or did you see something similar already?

2015-12-11 03:46:35 Upload of chunk 241 of '/media/hdd/backups/_250_Daten_Backups_2015-12-04.part2.rar' finished (Code: 202)
2015-12-11 03:46:35 Acquiring new access token
2015-12-11 03:46:39 Uploading chunk 242 of '/media/hdd/backups/_250_Daten_Backups_2015-12-04.part2.rar'
2015-12-11 03:46:39 Content-Length: 20971520
2015-12-11 03:46:39 Content-Range: bytes 5075107840-5096079359/9000000000
2015-12-11 03:52:46 Upload of chunk 242 of '/media/hdd/backups/_250_Daten_Backups_2015-12-04.part2.rar' finished (Code: )
An error has occurred while uploading '/media/hdd/backups/_250_Daten_Backups_2015-12-04.part2.rar' (Code: )

Newly created folders with wrong name

Hi,
I'm experiencing an issue when uploading to a custom path.
For example, uploading to:

/existing/path/newfolder/filename.ext

sometimes creates new folders like:

/existing 1/path/newfolder/filename.ext
/existing/path 1/newfolder/filename.ext
/existing/path/newfolder 1/filename.ext

This happens with both normal and multipart uploads.
It happens only one time out of 10 (aproximately), but when it does, it creates new folder I have to delete manually.

Respond to 429 Throttle response in OneDrive for Business

OneDrive has limits in place to make sure that individuals and apps do not adversely affect the experience of other users. When an activity exceeds OneDrive's limits, API requests will be rejected for a period of time. OneDrive may also return a Retry-After header with the number of seconds your app should wait before sending more requests.

HTTP/1.1 429 Too Many Requests  
Retry-After: 3600

Throttling

Public clients can't send a client secret

I can view the following message.

access

But, After that

When ready, please enter the value of the code parameter (from the URL of the blank page) and press return

And I input below parameter

code=M7c53xxx7-x4xf-2867-3931-e0000000f0&lc=0000

and

Something went wrong, here is the API response:
{"error":"invalid_request","error_description":"Public clients can't send a client secret."}

Plus

App ID has always dash. So readme file need to update.

thx.

Remove folder traversel

Folder traversal is inefficient and error-prone. Setting the root folder via ID is not very user friendly.

Create empty folder (necessary for feature #25)

POST https://graph.microsoft.com/v1.0/me/drive/root:/this/path/will/be/created:/children

{
    "name": "withSubfolder",
    "folder": {}
}

or

PATCH https://graph.microsoft.com/v1.0/me/drive/root:/this/path/will/be/created/withSubfolder

{
    "folder": {}
}

Beforehand verify, that the folder does not exist yet, so it does not have to be touched

GET https://graph.microsoft.com/v1.0/me/drive/root:/this/path/will/be/created/withSubfolder

If this returns with status code 404, the folder can be created

Upload file (folder will automatically be created)

PUT https://graph.microsoft.com/v1.0/me/drive/root:/containing/folder/name/filename.txt:/content

This is the content.

An upload session can be requested via

POST https://graph.microsoft.com/v1.0/me/drive/root:/containing/folder/name/filename.txt:/createUploadSession

...

Refresh access_token between uploads if necessary

The access_token is valid 3600 seconds after receiving. When uploading multiple files and the upload process takes more than an hour, the access_token will expire and all the following uploads will fail.

Solution: Check if the access_token is still valid within the upload loop and acquire a new access_token if necessary.

Error while authorizing/granting permissions with personal account

This is the error message I got while granting permissions to the app by going to url mentioned in terminal message after running the onedrive-authorize script.

Something went wrong, here is the API response:
{
   "error":"invalid_request",
   "error_description":"Public clients can't send a client secret."
}

FIX:
Removing the client_secret=${api_client_secret} from url fixes it.

Microsoft surprisingly switched to Open Data Protocol (OData)?

In the middle of development instead of documented API response

HTTP/1.1 200 OK
Content-type: application/json

{
  "id": "0123456789abc",
  "name": "example.xlsx",
  "eTag": "123918093j1lk2jlkda",
  "cTag": "k1ml4klkasljasidj1l2j34lkaslz",
  "createdBy": { "user": { "id": "1234", "displayName": "Ryan Gregg" } },
  "createdDateTime": "string (timestamp)",
  "lastModifiedBy": { "user": { "id": "1234", "displayName": "Ryan Gregg" } },
  "lastModifiedDateTime": "string (timestamp)",
  "size": 1234,
  "webUrl": "http://onedrive.com/...",
  "parentReference": { "driveId": "12345", "id": "root", "path": "/drive/root:" },
  "folder": { "childCount": 4 }
}

I got this

@odata.context=https://contoso-my.sharepoint.com/_api/v2.0/$metadata#items/$entity
@odata.type=#oneDrive.item
@odata.id=https://contoso-my.sharepoint.com/_api/v2.0/drive/items/01KB37ZVRNDHLCZKM4QJHL56WZC6J3ASBT
@odata.editLink=drive/items/01KB37ZVRNDHLCZKM4QJHL56WZC6J3ASBT
createdBy.application.id=b66d1076-4c04-4a33-9bb2-2578c4891886
createdBy.application.displayName=Backup
createdBy.user.id=37169131-6748-2bbe-a76f-422b6ad8a79e
createdBy.user.displayName=Test
createdDateTime=2016-08-30T11:37:13Z
eTag=\"{29D6192D-9C29-4E82-BEFA-D91793B04833},1\"",
lastModifiedBy.application.id=b66d1076-4c04-4a33-9bb2-2578c4891886
lastModifiedBy.application.displayName=Backup
lastModifiedBy.user.id=37169131-6748-2bbe-a76f-422b6ad8a79e
lastModifiedBy.user.displayName=Test
lastModifiedDateTime=2016-08-30T11:37:13Z
name=test
webUrl=https://contoso-my.sharepoint.com/personal/test_contoso_com/Documents/test
cTag=\"c:{29D6192D-9C29-4E82-BEFA-D91793B04833},0\"",

Not a big issue, but I heaven't heard any announces from Microsoft about this change.
I'm going to investigate this case.

Authentication error.

Hello!
Appears this, after enter with the autenticated url on terminal:

Something went wrong, here is the API response:
{"error":"server_error"}

I have downloaded master.zip and put bash-json-parser manualy inside libs. What could be wrong?

Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.