wictorwilen/gulp-spsync

Gulp plugin for synchronizing local files with a SharePoint library

Hi and welcome!

Wictor Wilen works as Principal PM Manager in the Microsoft People System team, creating highly personalized experiences that adapt to the unique interests of our users and the relationships that they have with other people and their content. Formerly member of the Microsoft Regional Director network and a SharePoint MVP for twelve years, as well as an author and a well known international speaker.

• 💬 Ask me about anything related to people and profiles for the Microsoft 365 platform.
• 🔭 I’m the founder of the successful (but now retired) open-source project Microsoft Teams Yeoman Generator - Yo Teams
• ☑️ Creator of the free online RACI editor tool - raci.app
• ⬇️ I have over a million downloads (1,021,164 as of March 2024) of my various npm packages
• 📫 You can always reach me on Twitter or Mastodon (@wictor | @[email protected]) or read more about me on www.wictorwilen.se
• 💻 I have a fascination for IoT and especially building my own stuff (hardware & software)
• ⚡ Fun fact about me is that I sold my first software at age 14 built with Turbo Pascal, a school scheduling management system that set the foundation for future commercial software in this space.

gulp-spsync's Issues

Get existing files from library

Hi,

Pretty new to using gulp for sp dev, but just wondering if it's possible to pull files from a library with this?
e.g. sync both ways?

Thanks!

Multiple Pipes and Permissions

This is what I want to do
return gulp.src('./src/styles/style.scss') .pipe(sass().on('error', sass.logError)) .pipe(sp(spConfig)) .pipe(gulp.dest('build/css/'));

But it doesn't work, throws error

[09:04:48] Checking folder exists https://domain.sharepoint.com/sites/boston/_api/web/GetFolderByServerRelativeUrl(@FolderName)?@FolderName=''
[09:04:49] INFO: Folder '' doesn't exist and will be created
[09:04:49] Creating path
[09:04:50] ERR: StatusCodeError: 401 - "{"error":{"code":"-2147024891, System.UnauthorizedAccessException","message":{"lang":"en-US","value":"Access denied. You do not have permission to perform this action or access this resource."}}}"

Not sure why it's trying to create a blank folder. In troubleshooting, I removed the sass component and I get the same error. I'm guessing that's not the issue. Keeping sass out. I then updated to

return gulp.src('./src/**/*.*') .pipe(sp(spConfig)) .pipe(gulp.dest('build/Style Library/'));

[09:07:21] Checking folder exists https://domain.sharepoint.com/sites/boston/_api/web/GetFolderByServerRelativeUrl(@FolderName)?@FolderName='styles'
[09:07:22] INFO: Folder 'styles' doesn't exist and will be created
[09:07:22] Creating path styles
[09:07:22] ERR: StatusCodeError: 401 - "{"error":{"code":"-2147024891, System.UnauthorizedAccessException","message":{"lang":"en-US","value":"Access denied. You do not have permission to perform this action or access this resource."}}}"

I've setup the app according to your instruction, i clicked Trust it and everything. I am a SCA in the site as well. So I may be showing 2 issues here, I think the second issue is the bigger issue, I can't push anything up.

Thanks

Support for SharePoint onprem

Hi, Wictor. Are you planning to add support for on premises sharepoint?

Support targetting a folder

Great tool! Would be great to have a setting to set a "root folder" in the SharePoint site to avoid building out the full path in the src folder. Essentially mapping src in my project directory to /style library/myapp/arc in the site.

uploading a Page Layout to MasterPage Gallery

Loving the tool a lot.
However I have an issue. I am trying to use this method to upload some new publishing page layouts, with correct meta data. it all works great until i try to set the PublishingAssociatedContentType property. It fails with "Unable to update metadata of the file".

I believe that the problem is the PublishingAssociatedContentType field name. However, this is the correct internal field name for updating it in CSOM. So i have also tried different values / different combinations of values (including valid CSOM values) to no avail... (see below for the different ones i tried)

Any thoughts?
Thanks Simon

Here is the gulpfile.js (note when i take OUT the PublishingAssociatedContentType it all works but does not set the necessasy associated content type field value...)

Removed top part of GULPFILE.JS
` "verbose": true,
"update_metadata": true,
"files_metadata": fileMetadata
})).
pipe(gulp.dest('build'));
});

var fileMetadata = [
{
name: 'SimonsPL.aspx',
metadata: {
"__metadata": { type: "SP.Data.OData__x005f_catalogs_x002f_masterpageItem" },
Title: 'Simons Page Layout',
MasterPageDescription: 'This is a page layout for me',
ContentTypeId: '0x01010007FF3E057FA8AB4AA42FCB67B453FFC100E214EEE741181F4E9F7ACC43278EE811',
PublishingAssociatedContentType: "0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064 }
}
];

//PublishingAssociatedContentType: ';#Welcome Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4;#',
DEA0F50FC8C147B0B6EA0636C4A7D4"

// PublishingAssociatedContentType: ';#Welcome Page;0x010100c568db52d9d0a14d9b2fdcc96666e9f2007948130ec3db064584e219954237af390064dea0f50fc8c147b0b6ea0636c4a7d4;#'`

Upload fails when library has versioning enabled

This gulp-spsync task works awesome! One thing I noticed is when trying to upload to the Style Library where a Check-Out is required I get an error saying: "Unable to upload file, it might be checked out to someone". Also when a Library not exists due to different language for instance I do get a "Upload successful" but the file is not uploaded.

Thanks for this plugin!

SharePoint 2013 Error

Hello,

I've configured it for a new project as usual, but this time when i save a file (gulp watch), it throws this error. Any idea where i shall look into this issue ?

Thank you,

[09:59:55] Locating STS Url for
[09:59:56] STS Endpoint found https://accounts.accesscontrol.windows.net/tokens/OAuth/2
Unhandled rejection StatusCodeError: 401 - "{\"error\":\"invalid_client\",\"error_description\":\"AADSTS70002: Unable to determine the tenant identifier from the request. Client ID '5aa2595c-9be5-42b5-af80-c821665a2d29@' is not valid. AADSTS90022: Prinapp1al name format is invalid for '5aa2595c-9be5-42b5-af80-c821665a2d29@'.
Expected name[/instance][@realm].\\r\\nTrace ID: 32382b18-53e8-451d-8d04-39c33c2f5100\\r\\nCorrelation ID: 0a0e1022-1f35-4d51-a7f7-30c4f901fcbc\\r\\nTimestamp: 2018-08-21 07:59:56Z\",\"error_codes\":[70002,90022],\"timestamp\":\"2018-08-21 07:59:56Z\",\"trace_id\":\"32382b18-53e8-451d-8d04-39c33c2f5100\",\"correlation_id\":\"0a0e1022-1f35-4d51-a7f7-30c4f901fcbc\"}"
    at new StatusCodeError (C:\dev\customer.app1\node_modules\request-promise-core\lib\errors.js:32:15)
    at Request.plumbing.callback (C:\dev\customer.app1\node_modules\request-promise-core\lib\plumbing.js:104:33)
    at Request.RP$callback [as _callback] (C:\dev\customer.app1\node_modules\request-promise-core\lib\plumbing.js:46:31)
    at Request.self.callback (C:\dev\customer.app1\node_modules\request\request.js:186:22)
    at emitTwo (events.js:106:13)
    at Request.emit (events.js:191:7)
    at Request.<anonymous> (C:\dev\customer.app1\node_modules\request\request.js:1081:10)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:188:7)
    at IncomingMessage.<anonymous> (C:\dev\customer.app1\node_modules\request\request.js:1001:12)
    at IncomingMessage.g (events.js:292:16)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)

My gulpfile.js

var gulp = require('gulp')
var sp = require('gulp-spsync')
var watch = require('gulp-watch')
var settings = {
    "client_id": "5aa2595c-9be5-42b5-af80-c821665a2d29",
    "client_secret": "SUzDLo/77e242VOcqs4vt525rrylQMLgb/FHt1e9qAk=",
    "realm": "",
    "site": "https://xxxxx/sites/xxx",
    "verbose": "true",
    "watch": "true"
};

var settingsSearch = {
    "client_id": "6207fa90-2b0e-4067-b8fe-1aae6cb3926f",
    "client_secret": "ZCmjRLuiAU+HszBYnnOuwvErI+L2fTyO4aKgyccGNQ8=",
    "realm": "",
    "site": "https://xxx/sites/xx/xxx2",
    "verbose": "true",
    "watch": "true"
};

gulp.task('deploySite', function () {
    return gulp.src('src/**/*.*')
        .pipe(watch('src/**/*.*'))
        .pipe(sp(settings))
        .pipe(gulp.dest('build'))
})

gulp.task('deployToSearch', function () {
    return gulp.src('src/**/*.*')
        .pipe(watch('src/**/*.*'))
        .pipe(sp(settingsSearch))
        .pipe(gulp.dest('build'))
})

I ran : deploySite gulp task

Thank you

Error when I fetch the package from npm

I am getting an error when I fetch the packages from npm. Any idea how to get around this?

-- [email protected] +-- [email protected] | +-- [email protected] | +-- [email protected] | |-- [email protected]
| +-- UNMET PEER DEPENDENCY continuation-local-storage@~3
| -- [email protected] -- [email protected]
+-- [email protected]
`-- [email protected]

I am using npm 3.8.5

Application with identifier was not found in the directory

The app is listed in App permissions, tried a few times. Trying to upload to Site Assets.
Any idea about what could be wrong?

Replaced some ids

C:\Users\user\Source\Repos\test1\charts\Test1\test1Sp>gulp
[16:49:10] Using gulpfile ~\Source\Repos\test1\charts\Test1\test1Sp\gulpfile.js
[16:49:10] Starting 'default'...
running default
[16:49:10] Verbose logging on!
[16:49:10] Uploading SiteAssets\folder\README.txt
[16:49:10] Locating realm for https://o365test1.sharepoint.com
[16:49:10] Realm is 40d2cf2d-1b83-48ba-998c-000000000000
[16:49:10] Retreiving access token for d6c54594-f5e6-4958-8385-000000000000@40d2cf2d-1b83-48ba-998c-000000000000
[16:49:10] Locating STS Url for
[16:49:11] STS Endpoint found https://accounts.accesscontrol.windows.net/tokens/OAuth/2
{"headers":{"Content-Type":"application/x-www-form-urlencoded"},"form":{"grant_type":"client_credentials","client_id":"d6c54594-f5e6-4958-8385-000000000000@40d2cf2d-1b83-48ba-998c-000000000000","client_secret":"secret=","resource":"00000003-0000-0ff1-ce00-000000000000/o365test1.sharepoint.com@40d2cf2d-1b83-48ba-998c-000000000000"}}
Unhandled rejection StatusCodeError: 400 - {"error":"unauthorized_client","error_description":"AADSTS70001: Application with identifier 'd6c54594-f5e6-4958-8385-000000000000' was not found in the directory 40d2cf2d-1b83-48ba-998c-000000000000\r\nTrace ID: 36f197b6-2e00-47d8-a957-00\r\nCorrelation ID: 60f4380f-92ed-490e-907c-00\r\nTimestamp: 2016-09-04 20:49:11Z","error_codes":[70001],"timestamp":"2016-09-04 20:49:11Z","trace_id":"36f197b6-2e00-47d8-00","correlation_id":"60f4380f-92ed-490e-907c-00"}
at new StatusCodeError (C:\Users\user\Source\Repos\test1\charts\Test1\test1Sp\node_modules\request-promise\lib\errors.js:26:15)
at Request.RP$callback as _callback
at Request.self.callback (C:\Users\user\Source\Repos\test1\charts\Test1\test1Sp\node_modules\request\request.js:187:22)
at emitTwo (events.js:106:13)
at Request.emit (events.js:191:7)
at Request. (C:\Users\user\Source\Repos\test1\charts\Test1\test1Sp\node_modules\request\request.js:1044:10)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at IncomingMessage. (C:\Users\user\Source\Repos\test1\charts\Test1\test1Sp\node_modules\request\request.js:965:12)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:926:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9)

Ensuring that the folder is present makes gulp-spsync really slow

Now gulp-spsync checks if the folder is present before trying to upload the file and if the folder is missing it creates the folder and uploads the file into it. But checking for the folders each time a deployment is triggered is time consuming as each segment (subfolder) of the address needs to be verified with the help of a web service and confirmed.

A slightly better approach would be to -

1. Attempt checking for the presence of the folder only on upload failure, ie, the script could safely assume that the folder is present as that would be the case 99 % of the time. Obviously this would make the first run slower, but every run after that would not get bogged down by the folder presence check

2. Now in the case where more than one file is uploaded to a folder, the presence of the folder is checked for each file, triggering (number_of_files_in_folder * number_of_subfolders_in_path) web service calls for each folder. This can be simplified by keeping track of the paths verified in memeory

Happy to make these changes if you are ok with these @wictorwilen. Let me know.

Library Path usage

So if I understand correctly, there's a new libraryPath parameter that will allow sync of local assets to a different librarypath on SharePoint. What's the usage?

Running gulp upload every files (watch settings)

Hi i tried to setup the gulp watch part, though when i run gulp, it uploads every files instead of watching just the one that changed ? Did i miss smth ?

Thanks

Add ability to create sub folders if they don't exist

Can we have the ability to upload hierarchically, creating subfolders if they don't already exist. Files in subfolders that don't exist, throw an Upload failed, might be checked out error.

Thanks and great work so far Wictor

WWW-Authenticate is not returned in the header

Trying to run spsync, correctly configured, I get the following:

'debug-publish' errored after 947 ms
TypeError in plugin 'gulp-spsync'
Message:
    Cannot read property 'indexOf' of undefined

The reason is that when getting the realm from target url, WWW-Authenticate doesn't exist in the response headers.

Resolved it by changing
var clientsvc = "/vti_bin/client.svc"
to
var clientsvc = "/_vti_bin/client.svc"
in index.js (Adding an underscore).