bestander/gulp-deploy-azure-cdn

Gulp plugin for uploading files to Azure Blob Storage

gulp-deploy-azure-cdn's Issues

Set custom metadata

Thank you for this awesome module! This is more of a question than an issue but it might be helpful for others looking for the same thing.

Is there any way to set custom metadata for each file? I can see that we can set cacheControl, which forms the properties of the blob but you can also create custom metadata entries using tools like Azure Management Studio.

Here is what I mean:

I have set the cacheControl property http://puu.sh/kfyL2/3ec48d3a3f.png
I want to be able to enter a custom entry like this http://puu.sh/kfyN1/0330c86367.png

Hope this makes sense.

Thank you.

Installing Gulp in Azure website shows a failure message, but actually installs

Step 1. Install a new Azure WebSite
Step 2. Goto Tools ==> Extensions and add the Gulp extension.
Step 3. See the error message on the top right

Step 4. Refresh the Extensions screen and see Gulp is installed

The specified container does not exist.

Hi,

I'm testing this module. I've created a new Azure Storage account and I copied the name and the key and I have this configuration:

`var deployCdn = require('gulp-deploy-azure-cdn');
var gulp = require('gulp');
var gutil = require('gulp-util');

gulp.task('upload-app-to-azure', function () {
return gulp.src(['.js', '.json'], {
base: 'node_modules/deploy-azure-cdn' // optional, the base directory in which the file is located. The relative path of file to this directory is used as the destination path
}).pipe(deployCdn({
containerName: 'cdngis', // container name in blob
serviceOptions: ['MY_ACCOUNT_NAME', 'MY_ACCOUNT_KEY'], // custom arguments to azure.createBlobService
folder: '', // path within container
zip: true, // gzip files if they become smaller after zipping, content-encoding header will change if file is zipped
deleteExistingBlobs: false, // true means recursively deleting anything under folder
concurrentUploadThreads: 10, // number of concurrent uploads, choose best for your network condition
metadata: {
cacheControl: 'public, max-age=31530000', // cache in browser
cacheControlHeader: 'public, max-age=31530000' // cache in azure CDN. As this data does not change, we set it to 1 year
},
testRun: false // test run - means no blobs will be actually deleted or uploaded, see log messages for details
})).on('error', gutil.log);
});`

When I try to upload my files I get the error: "The specified container does not exist." but It seems the code creates it for me, but still doesn't work.

C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp> cmd.exe /c gulp -b "C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp" --color --gulpfile "C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp\Gulpfile.js" upload-app-to-azure
[21:14:02] Using gulpfile ~\source\repos\WebApplicationGulp\WebApplicationGulp\Gulpfile.js
[21:14:02] Starting 'upload-app-to-azure'...
[21:14:02] Uploading ....\bundleconfig.json encoding gzip
[21:14:02] Uploading ....\appsettings.Development.json encoding gzip
[21:14:02] Uploading ....\appsettings.json encoding gzip
[21:14:02] Uploading ....\gulpfile.js encoding gzip
[21:14:02] Uploading ....\bower.json encoding gzip
[21:14:02] Uploading ....\package.json encoding gzip
[21:14:02] { [Error: The specified container does not exist.
RequestId:379347d8-001e-007e-0b60-5a821d000000
Time:2017-11-10T20:14:02.6969054Z]
code: 'ContainerNotFound',
statusCode: 404,
requestId: '379347d8-001e-007e-0b60-5a821d000000',
name: 'Error',
message: 'The specified container does not exist.\nRequestId:379347d8-001e-007e-0b60-5a821d000000\nTime:2017-11-10T20:14:02.6969054Z',
stack: 'Error: The specified container does not exist.\nRequestId:379347d8-001e-007e-0b60-5a821d000000\nTime:2017-11-10T20:14:02.6969054Z\n at Function.StorageServiceClient._normalizeError (C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp\node_modules\azure-storage\lib\common\services\storageserviceclient.js:868:23)\n at BlobService.StorageServiceClient._processResponse (C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp\node_modules\azure-storage\lib\common\services\storageserviceclient.js:570:50)\n at Request.processResponseCallback [as _callback] (C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp\node_modules\azure-storage\lib\common\services\storageserviceclient.js:274:37)\n at Request.self.callback (C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp\node_modules\request\request.js:129:22)\n at emitTwo (events.js:87:13)\n at Request.emit (events.js:172:7)\n at Request. (C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp\node_modules\request\request.js:873:14)\n at emitOne (events.js:82:20)\n at Request.emit (events.js:169:7)\n at IncomingMessage. (C:\Users\gisela\source\repos\WebApplicationGulp\WebApplicationGulp\node_modules\request\request.js:824:12)',
showStack: false,
showProperties: true,
plugin: 'gulp-deploy-azure-cdn ' }
[21:14:02] 'upload-app-to-azure' errored after 675 ms
[21:14:02] Error in plugin 'gulp-deploy-azure-cdn '
Message:
The specified container does not exist.
RequestId:379347d8-001e-007e-0b60-5a821d000000
Time:2017-11-10T20:14:02.6969054Z
Details:
code: ContainerNotFound
statusCode: 404
requestId: 379347d8-001e-007e-0b60-5a821d000000
Process terminated with code 1.

I tried to create the container manually but I get the same error.

Any ideas? thanks!

Uploaded files are being created in a folders identical to my dev machine

Thanks for your work on this module, it looks like it will do what I need it to do however I am having a little bit of a problem using it correctly.

gulp.task('upload-azure', function () {
    return gulp.src(['asserts/dist/*.css', 'asserts/dist/*.js'], function () {
        cwd: 'node_modules/deploy-azure-cdn'
    })
       .pipe(deployCdn({
           containerName: 'asserts',
           serviceOptions: ['XXXX', 'XXX'],
           destinationFolderPath: '',
           folder: '',
           zip: true, 
           deleteExistingBlobs: true,
           concurrentUploadThreads: 10,
           metadata: {
               cacheControl: 'public, max-age=31530000',
               cacheControlHeader: 'public, max-age=31530000'
           },
           testRun: false
       })).on('error', gutil.log);
});

Files are uploaded to the correct container however the path to the file is wrong. It looks like its using the path from dev machine and appending it to the azure container e.g. /asserts/asserts/dist/fileabc.css /asserts/asserts/dist/fileabc.js

Ideally I dont want the path 'asserts/dist' to be applied to container I called 'asserts'
I want the files to appear in the root of the container.

What am I doing wrong here?

Missing uploaded files

When I upload loads of files (1065 files 228 folders) to the CDN the task will run properly and complete but later on I can see that not all files where uploaded properly, I see that some files are missing on the CDN, I have to execute the task several times in order to get everything uploaded properly, is there any way I can double check that all files where uploaded to guaranty 100% all files are on the CDN?
Could a high value (20) for concurrentUploadThreads be the issue?
I get random "read ECONNRESET" on uploading some files

I am getting "ContainerNotFound" errors

When I run this locally, I get...

Message:
    The specified container does not exist.
RequestId: 00000000-0000-0000-0000-000000000000
Time:2015-08-11T19:47:40.8942115Z
Details:
    code: ContainerNotFound
    statusCode: 404
    requestId: 00000000-0000-0000-0000-000000000000

I know that things are connected correctly because if I delete the container via the portal and then run this, it will create the container for me, and then still fails when trying to upload the files.

Here is my configuration.

gulp.task('azure', function () {
    return gulp.src(['cdn/**/*', 'config.js'], {
        base: 'node_modules/deploy-azure-cdn'
    }).pipe(deployCdn({
        containerName: 'statics',
        serviceOptions: ['xxx', 'yyy'],
        folder: '',
        containerOptions:  {publicAccessLevel : 'blob'},
        zip: true,
        deleteExistingBlobs: false,
        concurrentUploadThreads: 10,
        metadata: {
            cacheControl: 'public, max-age=31530000',
            cacheControlHeader: 'public, max-age=31530000'
        },
        testRun: false
    }));
});

Sometime upload doesn't work

Hey

Great plugin! But sometime the upload doesn't work without any error message when I call gulp from command line and it doesn't work at anytime when called via a npm scripts.

My gulp file:

var gulp = require('gulp')
    , deployCdn = require('gulp-deploy-azure-cdn')
    , gulpIgnore = require('gulp-ignore')
    , path = require('path')
    , rename = require('gulp-rename')
    , merge = require('merge-stream')
    , streamqueue = require('streamqueue')
    , clean = require('gulp-clean')
    , argv = require('yargs').argv;

// gulp task for publishing to Azure Blob Storage
gulp.task('publishAzure', function () {

    var renameAndMoveMainJS = gulp
        .src(['./build/static/js/**/*.js'])
        .pipe(rename('./main.js'))
        .pipe(gulp.dest('./build/dist/js/'));

    var renameAndMoveMapJS = gulp
        .src(['./build/static/js/**/*.map'])
        .pipe(rename('./main.js.map'))
        .pipe(gulp.dest('./build/dist/js/'));

    var renameAndMoveMainCSS = gulp
        .src(['./build/static/css/**/*.css'])
        .pipe(rename('./main.css'))
        .pipe(gulp.dest('./build/dist/css/'));

    var renameAndMoveMapCSS = gulp
        .src(['./build/static/css/**/*.map'])
        .pipe(rename('./main.css.map'))
        .pipe(gulp.dest('./build/dist/css/'));

    var uploadJSToBlob = gulp
        .src('./build/dist/js/**/*')
        .pipe(deployCdn({
            containerName: 'core',
            serviceOptions: [accountName, accountKey],
            folder: argv.dev === undefined ? 'stable/js' : 'dev/js',
            zip: false,
            deleteExistingBlobs: false,
            concurrentUploadThreads: 1
        }));

    var uploadCSSToBlob = gulp
        .src('./build/dist/css/**/*')
        .pipe(deployCdn({
            containerName: 'core',
            serviceOptions: [accountName, accountKey],
            folder: argv.dev === undefined ? 'stable/css' : 'dev/css',
            zip: false,
            deleteExistingBlobs: false,
            concurrentUploadThreads: 1
        }));

    return streamqueue({ objectMode: true }, renameAndMoveMainJS, renameAndMoveMainCSS, renameAndMoveMainCSS, renameAndMoveMapCSS, uploadJSToBlob, uploadCSSToBlob);
});

gulp.task('cleanUp', function () {
    return gulp
        .src('./build/dist', { read: false })
        .pipe(clean());
});

// make 'publishAzure' task the default
gulp.task('default', ['publishAzure']);

My npm script:

"publish:dev": "npm run build:remote && gulp publishAzure --dev && gulp cleanUp",

The output:

[20:56:30] Using gulpfile ~/Documents/Dev/<vendor>/webplugin/gulpfile.js
[20:56:30] Starting 'publishAzure'...
[20:56:31] Using gulpfile ~/Documents/Dev/<vendor>/webplugin/gulpfile.js
[20:56:31] Starting 'cleanUp'...
[20:56:31] Finished 'cleanUp' after 38 ms

empty folder in azure blob

Hi,

I have tried changing all the settings in my gulp file but every time i upload any assets a blank folder is created in azure

this means i can't access it through the cdn endpoint. Not sure why this is happening?

upload files in $web container

After Static Website hosting has been released (preview --> https://azure.microsoft.com/en-us/blog/azure-storage-static-web-hosting-public-preview/) would be great to upload file in $web container.

Issue with options being set to last value when making multiple requests in quick succession

Hi,

I am trying to pipe two sets of files into deployCdn() in quick succession (so two separate calls). I am sending different options with both calls but because the first call hasn't finished when the options are set the second time, both are using the second set of options.

I think this is because "var options" is declared directly inside your module.exports so it's global in the sense that the variable's scope is not local to each call. When the second call is made the global options variable is overwritten and the first call which hasn't finished yet uses these options. Basically it's a race condition that always loses.

This doesn't seem an easy thing to fix and I notice that this is quite an old project so my solution is to use a single container and one set of options for all my files for now. I just thought you might like to know.

Jon

Purge cdn

What do you think of adding feature development so that the plugin can execute the purge command on the Azure CDN?