sffc/socketio-file-upload

Uploads files to a Node.JS server using Socket.IO

socketio-file-upload's Issues

Include source mappings with minified version

It would be awesome if a source mapping could be included as well as the minified version so that issues that are thrown to the browser console can be correctly identified and traced, rather than getting stacks that just report x as a variable.

Uploaded file with non-English letter is saved in undesirable way

Uploaded files with non-English letter are saved in undesirable way.

Server and browser were set to utf-8 character set.
% locale
LANG=ko_KR.UTF-8

server side console log is as followings

session.user : { username: 'bob' }
session.user.username : bob
debug - emitting heartbeat for client 0byjQVSiXUjpmcUVYThz
debug - websocket writing 2::
debug - set heartbeat timeout for client 0byjQVSiXUjpmcUVYThz
debug - got heartbeat packet
debug - cleared heartbeat timeout for client 0byjQVSiXUjpmcUVYThz
debug - set heartbeat interval for client 0byjQVSiXUjpmcUVYThz
debug - websocket writing 5:::{"name":"siofu_ready","args":[{"id":0,"name":"-0"}]}
File saved !!
{ name: '소야도 포인트.JPG',
mtime: Tue Nov 27 2012 16:58:53 GMT+0900 (KST),
encoding: 'octet',
clientDetail: {},
id: 0,
base: '-0',
pathName: '/var/uploads/-0.JPG',
writeStream:
{ writableState:
{ highWaterMark: 16384,
objectMode: false,
needDrain: true,
ending: true,
ended: true,
finished: true,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
sync: false,
bufferProcessing: false,
onwrite: [Function],
writecb: null,
writelen: 0,
buffer: [] },
writable: true,
domain: null,
events: { open: [Function], error: [Function] },
maxListeners: 10,
path: '/var/uploads/-0.JPG',
fd: null,
flags: 'w',
mode: '0666',
start: undefined,
pos: undefined,
bytesWritten: 169291,
closed: true } }
debug - websocket writing 5:::{"name":"news","args":[{"name":"소야도 포인트.JPG","mtime":"2012-11-27T07:58:53.000Z","encoding":"octet","clientDetail":{},"id":0,"base":"-0","pathName":"/var/uploads/-0.JPG","writeStream":{"writableState":{"highWaterMark":16384,"objectMode":false,"needDrain":true,"ending":true,"ended":true,"finished":true,"decodeStrings":true,"defaultEncoding":"utf8","length":0,"writing":false,"sync":false,"bufferProcessing":false,"writecb":null,"writelen":0,"buffer":[]},"writable":true,"domain":null,"events":{},"maxListeners":10,"path":"/var/uploads/_-0.JPG","fd":null,"flags":"w","mode":"0666","bytesWritten":169291,"closed":true}}]}
debug - websocket writing 5:::{"name":"siofu_complete","args":[{"id":0,"success":true,"detail":{}}]}
debug - emitting heartbeat for client 0byjQVSiXUjpmcUVYThz

---

screen-captured browser console log

Let me know how to fix this problem, Thank you : -)

Large file uploads cause socket to disconnect and have a "transport error"

I cant seem to get this package to work with anything larger than 50MB. It essentially starts off strong and then disconnects the socket and keeps trying to transmit on a closed connection.

What is the main benefice using realtime over standard ajax to upload files ?

I i just discover your repository and I guess it's pretty cool but I don't really mean what is the main benefice using realtime over standard ajax to upload files.

Anyone can light me about that ?

Binary Data Transmission

Socket.IO 1.0 now supports binary data transmission through the WebSocket. The next release of Socket.IO File Upload will support transmitting files through this new architecture.

You can enable file transfer with buffers by setting instance.useBuffer = true on the client side.

As you use this new feature, leave feedback here. In a future release of Socket.IO File Upload, this option may default to "true" rather than to "false".

Unable to get file to transfer

I am trying to use socketio-file-upload from a context menu item click event. I am calling myinstance.prompt(), the dialog comes up, I select a file, hit Open. The server emits a "siofu_ready", then nothing else happens. The file is created in the specified server side dir but with 0 bytes. No errors in the console. Help would be appreciated.

an error when loading the module in webpack

It gives "define cannot be used indirect" error when trying to import it in webpack environment.
Is this a compatibility issue? any helpful comments I can get?

[email protected]

Uploads tangled between clients

Some images I upload on one client get uploaded on the other clients browser. I'm not sure what is causing it. I've tried putting the server lines where the documentation said to put them
var uploader = new siofu();
uploader.dir = 'images';
uploader.listen(socket);
And also I tried putting them in an event that only fires once when the client connects. Neither worked.

How would I get acces to the data without saving it to disk ?

Upload Interruption should trigger error for handling

When client socket disconnects or refresh happens on client side there should be an error on server. Is this the case? did I miss something in the documentation?
on.complete does not fire when file is upload is interrupted.

Allow overwriting files

Unless I'm not seeing the option, it seems there is no way to allow the uploader to overwrite a file if it already exists.

Undefined is not a function - client.js: 498

Appears sporadically when I'm trying to upload images, haven't been able to perfectly recreate the environment that breaks it, but it fails on this line:

// CONSTRUCTOR: Listen to the "complete", "ready", and "error" messages on the socket.
socket.on("siofu_ready", function(data){
    readyCallbacks[data.id](data.name);
});

... it makes me think that for some reason data.id is sometimes undefined. I've used the api in the same way as in the documentation so I don't know what's going on.

Cheers, Matt.

Uncaught DOMException: Failed to execute 'readAsArrayBuffer' on 'FileReader': The object is already busy reading Blobs.

A user reported that in Chrome on Windows 10 the following error is thrown which causes the progress indicator to read 100% done while network traffic continues in the background.

I'm not sure if this is fixed by #60 or not as I haven't updated our codebase to use anything past 0.5.0, but I can check on that later this week if that would help.

Uncaught DOMException: Failed to execute 'readAsArrayBuffer' on 'FileReader': The object is already busy reading Blobs.
    at x (<redacted>/js/vendor/upload/client.min.js:9:231)
    at Array.<anonymous> (<redacted>/js/vendor/upload/client.min.js:11:449)
    at n.<anonymous> (<redacted>/js/vendor/upload/client.min.js:15:179)
    at n.e.emit (<redacted>/js/vendor/socketio/socket.io.min.js:23:1606)
    at n.onevent (<redacted>/js/vendor/socketio/socket.io.min.js:22:31333)
    at n.onpacket (<redacted>/js/vendor/socketio/socket.io.min.js:22:30955)
    at n.<anonymous> (<redacted>/js/vendor/socketio/socket.io.min.js:23:2238)
    at n.e.emit (<redacted>/js/vendor/socketio/socket.io.min.js:23:1606)
    at n.ondecoded (<redacted>/js/vendor/socketio/socket.io.min.js:21:23673)
    at s.<anonymous> (<redacted>/js/vendor/socketio/socket.io.min.js:23:2238)
x @ client.min.js:9

Renaming your file before saving

maybe i don't understand how this package works very well. i have spent time to see how i can rename a file before saving it to a folder but couldn't figure it out. please if there is way to achieve this let me know. will be glad to hear your feedback ,thanks

Show File

Sorry, but i'm newbie in this.

I think my code is ok.
When i select a file, the console show the Object.

How i can show the file on html?
I need upload a image and set this on css background.

Thx for help!

Emitting progress updates question

New to node and socket.io but this looks promising :)
How would I use the progress event to send the client regular file upload progress messages?

Handle missing directory

Currently if you attempt to upload into a path that does not exist the process simply quits with an error, as would be expected. It'd be nice if there was a way to either have the process create that folder, or to allow an event where I could check for the folder, and create it before the file starts trying to upload to that location.

example code crashes server

When I try to run the example my server crashes:

 debug - websocket writing 1::
   debug - websocket writing 5:::{"name":"siofu_complete","args":[{"id":0,"success":false,"detail":{}}]}

events.js:73
        throw new Error("Uncaught, unspecified 'error' event.");
              ^
Error: Uncaught, unspecified 'error' event.
    at SocketIOFileUploadServer.EventEmitter.emit (events.js:73:15)
    at _uploadStart (/node_modules/socketio-file-upload/server.js:231:12)
    at _findFileName (/node_modules/socketio-file-upload/server.js:119:5)
    at _findFileNameWorker (h/node_modules/socketio-file-upload/server.js:94:7)
    at Object.oncomplete (fs.js:297:15)

This is happening in the browser

5:::{"name":"siofu_complete","args":[{"id":0,"success":false,"detail":{}}]}
75
20:55:33
5:::{"name":"siofu_start","args":[{"name":"test.svg","mtime":"2013-11-12T17:53:25.000Z","encoding":"octet","id":0}]}

I'm using exact example code. I think it is more a client side of problem?

destroy method

That all worked fine and dandy but then I was running into an issue where it was very difficult to remove the event listeners. To resolve this, I added the following to client.js:

/**

• Destroys the old listeners and allows me to refresh new ones.
  */
  this.destroy = function(){

callbacks = {};
uploadedFiles = [];
readyCallbacks = [];
}

This allows me to do a hard reset of siofu when I load a new page or new content by calling it from the template when it is being unloaded. This also resolved an issue where if I did several edits on a page (moved another image to the upload spot) both would appear (in my case, I only want one).

How do you submit files submitFiles() manually ?

Am I doing anything wrong ?

HTML

<input type="file" class="file-brows"
                 onchange="angular.element(this).scope().onChangeStartFileUpload(event)" >

JavaScript

   scope.onChangeStartFileUpload = function(event){
        var files = event.target.files;
        // upload.submitFiles(files[0]);
       // upload.submitFiles(files);
      };

Clear input value after upload

input does not trigger change event when the value is the same. To upload the same file twice the input value has to be cleared.

Invoke upload without Submit or Drop event

is there a way to invoke event upload with outside of ... listenOnSubmit / listenOnDrop ?

Restrict File Size Limit While Choosing File

i want to restrict the user not to select file size more than 10mb,
any suggestion to my requirment.

Suggestion to implement in this "socketio-file-upload" module
1>thumbnil for each uploaded file
2>upload only image file/ or any other by extension
3>download files in socket

Rename File on Complete of Upload ?

Is there a method to handle this on complete promise to rename file?

Can't upload file in Android Crosswalk when use plugin File or File-transfer

Hello vote539,

Have a nice day!

I have found a bug in hybrid mobile app that makes the socketio-file-upload don't work.

When I add plugin org.apache.cordova.file or org.apache.cordova.file-transfer to my app (ionic crosswalk android app), I can't upload my file to server but can choose file.

After some researchs (compared between Android and Firefox browser in Dekstop,... ), I found that the FileReader object in Android crosswalk has quite differences, it was changed by default, but you can find an attribute "_realReader"!

Yeah, from that point, I add following lines before line "// Calculate chunk size" and it works :)

if (reader._realReader)
        reader = reader._realReader;

Thank for your awesome plugin!

SocketIOFileUploadServer not accessible in mobile browser/ webview

i tried to connect socket in android, windows mobile browser , in which socket is connected but the SOFU , i using submit button event to upload files, which works well on web browser but not in mobiles,
how to make it work with mobile as well, please suggest any solution to this

code snap for it:

Want to set custom values to "event" after saving file

Actually I tried to save file with current time (say 1384637442348.jpg) as name instead of original file name(XXXXX.jpg).
So, I set file name in "start". But, while getting event in "complete" (client side) always returns event with old name(XXXXX.jpg) only. Also, I want to add some additional info to the event once file saved on server.

Here my code goes,

//Server side
uploader.on("start", function(event){
console.log(":::::::::Start::::::::");
event.file.name = new Date().getTime()+".jpg";
console.log(event);
});
uploader.on("saved", function(event){
console.log(":::::::::Saved::::::::");
event.actualName = event.file.pathName; //Custom Value
console.log(event);
});

//Client side
siofu.addEventListener("complete", function(event){
console.log(event.success);
console.log(event.file);
});

//Server Log:
:::::::::Start::::::::
{ file:
{ name: '1384637442348.jpg',
mtime: Fri May 24 2013 17:14:48 GMT+0530 (IST),
encoding: 'octet',
id: 0 } }
:::::::::Start::::::::
{ file:
{ name: '1384637442348.jpg',
mtime: Fri May 24 2013 17:14:48 GMT+0530 (IST),
encoding: 'octet',
id: 0,
base: '1384637442348',
pathName: '/Users/mymac/Node/uploadedFiles/1384637442348.jpg',
writeStream:
{ _writableState: [Object],
writable: true,
domain: null,
_events: [Object],
_maxListeners: 10,
path: '/Users/mymac/Node/uploadedFiles/1384637442348.jpg',
fd: null,
flags: 'w',
mode: '0666',
start: undefined,
pos: undefined,
bytesWritten: 50022,
closed: true } },
actualName: '/Users/mymac/Node/uploadedFiles/1384637442348.jpg' }

//Client Log:
true
File { size=256963, type="image/jpeg", name="XXXXX.jpg", more...}

Can you please help me to solve it in some other way? Correct me if i did anything wrong.

Uploaded files not saved ...

Trying to use "socketio-file-upload" in my project. Client side events fired, server side events also fired besides "progress" events ... Files created in upload directory on the server side, but uploaded files are empty. Each time a new file uploaded, server side events are multiplied.
Appreciate any help!

Thank you!

Problem solved ... Working in browsers which are able create instance of HTML5 FileReader object.

Client gets original file name even the file name has been updated

If a file with dupe file name is uploaded, it would be saved as -0., but the client is still notified with the original name.

Benchmark

This lib is very similar to mine, I have tested the who side by side to see if there is any speed difference. It seems they are near identical in speed (for me ~4 megabytes a second uploaded), in my tests I am uploading over a 10gig network, I tried disabling the server from writing to file on both libraries and found that the speed did not change, my theory is that the client browsers file.slice > file.read* is the bottle neck.

Has anyone else looked into this? Is anyone getting much better than 3-4 MB/s ?

*I use 'readAsBinaryString', this lib uses 'readAsText' and 'readAsArrayBuffer' (I found no real speed difference between the 3 methods)

*My speed calcluation is:

/**

• this get the time since the last push, scales the time and speed to per second values
  */

      var timeNow = new Date();
      if (File.lastTime) {
          var difference = timeNow.getTime() - File.lastTime.getTime();
          var seconds = Math.floor(difference * 1000);
          var scale = 1 / seconds;
          var scaledSize = CHUNK_SIZE * scale;
          File.speed = scaledSize.toFixed(2) // <-- MB/s
  
  
      }
  

  ...
  //do slice & read file, etc.

      File.lastTime = timeNow;
  

Ability to cancel an upload in progress

How difficult would it be to provide a method to cancel an in-progress file upload? Dropzone.js offers this functionality: http://www.dropzonejs.com/#config-addRemoveLinks. I'm thinking something similar.

To understand why someone might want this, "accidentally" drop a folder with a dozen 500MB video files onto your application's dropzone. It would be nice if the user could click to cancel either individual files or all files.

Cordova app

has anyone gotten this to work with a cordova app/ionic? i am getting just a bunch of errors on it right now. lastly that "addEventListener is not a function"
from
var _listenTo = function (object, eventName, callback, bubble) {
object.addEventListener(eventName, callback, bubble);
_listenedReferences.push(arguments);
};
i havent attached anything to it yet, ive just done the new instantiate of it.

"var uploader = new SocketIOFileUpload(socket);"

The second question is doing it without the document watcher if possible, not sure if i can just send a file location to it yet, but only will worry about then when it doesn't error i suppose.

Thank you

[FIX] Client to Server Meta Data do not work

Hi,

I'm trying to send same metadata from client to server and I get the following error.

TypeError: event.file.meta is undefined
event.file.meta.hello = "world";

My client-side code:

uploader.addEventListener("start", function(event){
    event.file.meta.hello = "world";
});

Everything else works just fine, beside this one.
If I set clientDetail from server to client, it works.

I have also tried defining the meta data like this:

event.file = {
            'meta' : {
                'hello': world,
            }
        };
--- or ---
event.file.meta = {
      'hello' : world;
};

...but on server side I get "undefined" when trying to access these values with event.file.meta.hello

FIX

It seems that npm install --save socketio-file-upload does not install the latest github version and all js files are smaller in size than the original github ones.
So, unless the node package is not up-to-date, it would be better to manually copy files from github repo...

Drag Events

listenOnDrop() does this method override the drop event.

Add ability to submit file not bounding to file input

It makes sense if you have some preconditions before you uploading files (validations, etc).

I can't set the quantity of files that can be uploaded per time

When I select something around 20 files, it send chunks for all the files. So if I lose internet connection, all the upload progress is aborted and I'll get no complete files in the server.

Something like this could be implemented? Setting, per example, 2 files per time (for sending chunks). So when one of these 2 files is fully uploaded, the system start sending chunks for the next file.

Microsoft Edge file object missing lastModified

This is not an issue with socketio-file-upload, but is an issue others may run into when uploading files.

I found that my Microsoft Edge users would end up with files on my server with junk timestamps. In my case, I have a process that rsyncs the files to another location and preserves timestamps. rsync would fail with a message like the one below--substitute your file name.

Time value of %s truncated on receiver.

The issue is that Edge does not follow the spec for the File Object as explained here:
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/4155006/

In short, the object is missing the "lastModified" property that normally holds the unix time value. In my own code, I was able to account for this by adding this bit of code in my function where I handle each file object:

if (!file.lastModified) { file.lastModified = file.lastModifiedDate.getTime(); }

Arguably, socketio-file-upload client could do this same thing internally.

How would I get access to the data without saving it to disk ?

There is a mention in the documentation that it is possible but i can not figure out how to do it.
if i set uploader.dir =null any way i can get the raw data?

Permission error 34 - uploader.dir = " /var/www/vhosts/chattet.de/maiwell.chattet.de/uploads";

i've tried chmod 755. chmod 777 user:usergroup www-data:www-data nothing is working, why?

on an other server it's working

0|app | Error from uploader { file:
0|app | { name: 'a.png',
0|app | mtime: Sat Sep 02 2017 13:31:44 GMT+0200 (CEST),
0|app | encoding: 'octet',
0|app | clientDetail: {},
0|app | meta: {},
0|app | id: 3,
0|app | size: 8380,
0|app | bytesLoaded: 0,
0|app | success: true },
0|app | error:
0|app | { [Error: ENOENT, open ' /var/www/vhosts/chattet.de/maiwell.chattet.de/uploads/a.png']
0|app | errno: 34,
0|app | code: 'ENOENT',
0|app | path: ' /var/www/vhosts/chattet.de/maiwell.chattet.de/uploads/a.png' },
0|app | memo: 'computing file name' }

Add ability to send some meta data with files.

Need to add ability to send some meta data with files to the server

Support for non-html5 browsers

Hi there, just a quick question.
I just read the readme file and noticed you mention html5 supporting browsers.
Is that requirement about file drag-and-drop or for this module to work?
Given socket.io works in ie7, can we assume this module works in ie7?

"nothing happens" when uploading the same file again with IE11

After uploading a file with IE11, I try to upload the same file again but it's not possible, no event is fired.
I tried to reinitialize the uploader, I delete the file before the next try, but it doesn't help.

Multiple Dropzones

I can only use ID for drop event
I need to use class so I can have multiple dope arias on the screen.
is this possible ?

submitFiles() should accept objects of type `File` or an array of `Files`, not just FileList

I'm bouncing the output from Dropzone.js and specifically react-dropzone into SIOFU's instance.submitFiles(), but it seems to only accept objects of type FileList, whilst dropzone's onDrop events pass through a File, meaning that it's not compatible. I thought perhaps I could send the File array-wrapped, or even instantiate a new FileList([File]) or something, but that seems to not be supported. Some things when sent will chuck out an error, but some things will just silently fail. On the whole it seems to be a bit confusing as to how exactly it should work.

What are your thoughts on this?

Ta!
Matt

Server crashes with: ENOENT, utime error

Hi,

We have an application running stable for several months but today we got a server crash because of an ENOENT, utime error, from the file server.js line 156

After looking at the code I can see it's related to changing the file modified time:

fs.utimes(...
    if (err) { ...
        throw err

Even tough the throw is inside a try-catch block the application crashed. Interestingly the file was properly uploaded, moved and the time stamp was actually updated.

I understand it's mysterious than the try-catch block failed to do its job, but as it seems to be a small error does it justify risking a throw?

Any clue on how to prevent it, or should I send a pull request replacing the throw with a log message?

Thanks!

File size limit on uploads?

Again, great stuff. This npm has proven to be incredibly useful to this point!

I am running into an issue that seems to be related to the socket timing out when I upload lager files - say, more than 25MB. I know I know - huge for the internet but I'm combining this with fluent-ffmpeg to convert and edit video files so in some cases they can get pretty large.

Any idea on how to force the socket to not time out? Is this a socket.io or file-upload thing - if socket, is there a way to over-ride it to stay active?

Manually invoke upload instance.submitFiles(files)

Hi
I'm trying to drag a image from element on the screen or from a remote URL into drop element of siofu. Manually invoke upload am I going down the right path or is there a different way to handle this ?

 angular.element(currentElement).bind('drop',function(event){
      var imageURL = event.dataTransfer.getData('text');
      var xhr = new XMLHttpRequest();
      xhr.open('GET', imageURL, true);
      xhr.responseType = 'blob';
      xhr.onload = function(e) {
      if (this.status === 200) {
         var myBlob = xhr.response;
         var fileObj = new File([myBlob],'image.jpg',{});
         siofu.submitFiles(fileObj);
      }
  };

Server side validation ?

How do I stop uploads if file doesn't meet file.type server side.
Giving Server ability to stop upload or proceed with upload.

Specify a custom directory for each upload

The example for uploadValidator sets the global uploader.dir property on the fly, which can have a race condition for two simultaneous uploads. A better approach would be to allow every file to have its own custom directory to override the global setting.

See #58 and #60

File mtime detected as invalid.

I'm having a strange issue where file uploads are being marked as having an invalid mtime when they are uploaded. This causes the files to use UNIX epoch as the timestamp (which is a bit undesirable).

This is an example of the output of the saved event on the server.

{ file: 
   { name: 'Microsoft Word - Lab 5 Lists.pdf',
     mtime: Invalid Date,
     encoding: 'octet',
     clientDetail: {},
     meta: { path: '', identifier: 'vqrn05pdn7m' },
     id: 0,
     size: 72118,
     bytesLoaded: 72118,
     success: true,
     base: 'Microsoft Word - Lab 5 Lists',
     pathName: '/srv/daemon-data/ptdl-mumble_e1ltb/data/Microsoft Word - Lab 5 Lists.pdf',
     writeStream: 
      WriteStream {
        _writableState: [Object],
        writable: true,
        domain: null,
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        path: '/srv/daemon-data/ptdl-mumble_e1ltb/data/Microsoft Word - Lab 5 Lists.pdf',
        fd: 20,
        flags: 'w',
        mode: '0666',
        start: undefined,
        autoClose: true,
        pos: undefined,
        bytesWritten: 72118 } } }

Here is the output of event.file during the progress step in the browser.

lastModified: 1476111825000
meta: {path: "", identifier: "vqrn05pdn7m"}
name: "Microsoft Word - Lab 5 Lists.pdf"
size: 72118
type: "application/pdf"

Same output on the complete event in the browser.

lastModified: 1476111825000
meta: {path: "", identifier: "vqrn05pdn7m"}
name: "Microsoft Word - Lab 5 Lists.pdf"
size: 72118
type: "application/pdf"

Interestingly, if you refresh the page before the file is done uploading the last modified time is correct.