Error: Invalid 'path' Argument Type in File Upload Code

[EhsanKey]

Error: Invalid 'path' Argument Type in File Upload Code

[Doc999tor]

The typings are not precise - it just says: Express.Multer.File but in fact there are two stages with different sets of fields:

1. At multer.diskStorage.filename stage the file contaons only 4 fields: fieldname, originalname, encoding, mimetype as mentioned here
   At diskStorage.on finish when the file finished to be written to the disk, the file param is updated with additional fields from here with destination, filename, path, size fields here
   So, actually we can't rely on typings in diskStorage.filename hook
2. For providing a random name you can send an updated filename in cb(null, ``${ file.originalname }-${Math.random()}``)

filename: function (req, file, cb) {
	cb(null, path.join(`${ TMP_DIR }/${ file.originalname }/${hash.digest("hex")}`))
}

So you don't need to stream the file to some additional location - by passing a correct path argument to the cb you'll get the file wherever you need

[EhsanKey]

Sorry for the late reply:

1. After the upload, I receive the file buffer

In fact, despite the downsides of using MemoryStorage, it'll work - if the file exists, you just discard the buffer; if it doesn't you can write the buffer to the disk to the relevant directory

2. upload process will be performed twice if the file does not exist

No, in this case the file will be uploaded and processed once - you upload it to MemoryStorage and then write it to the disk or discard

3. we should delete it from the temporary file directory

In this case you don't need a temp directory - you can save the buffer directly to the needed directory

You're welcome, dear friend. Thank you for the comprehensive and thorough points. I will definitely apply the changes to my code.

[EhsanKey]

The typings are not precise - it just says: Express.Multer.File but in fact there are two stages with different sets of fields:

1. At multer.diskStorage.filename stage the file contaons only 4 fields: fieldname, originalname, encoding, mimetype as mentioned here
   At diskStorage.on finish when the file finished to be written to the disk, the file param is updated with additional fields from here with destination, filename, path, size fields here
   So, actually we can't rely on typings in diskStorage.filename hook
2. For providing a random name you can send an updated filename in cb(null,${ file.originalname }-${Math.random()})

filename: function (req, file, cb) {
	cb(null, path.join(`${ TMP_DIR }/${ file.originalname }/${hash.digest("hex")}`))
}

So you don't need to stream the file to some additional location - by passing a correct path argument to the cb you'll get the file wherever you need

Thank you.
I wanted to retrieve the file buffer before it is uploaded. The approach I took to retrieve the file buffer was actually wrong. To obtain the file buffer before uploading, I needed to set the value of the variable storage to multer.memoryStorage() like this:

const storage = multer.memoryStorage();

const upload = multer({
  storage,
});

Soon, I will make this part of my project open source on GitHub and share the link below this post.

[Doc999tor]

1. For my point of view, your previous approach was correct and multer does it already: gets the file and stream it to some directory. Unlike some other languages, in Nodejs it's a bit more low-level, and you need to provide the upload destination, to ensure it's available etc.
   So you can create some temp dir, stream there the file, adjust the name, perform checks and everything you might need to do for your uploading strategy
2. MemoryStorage is a bit tricky:
   i. You'll need to have enough available memory for file uploading
   ii. It's a bit more fragile since if the process crashes the Buffer contains the file will be lost

[EhsanKey]

1. For my point of view, your previous approach was correct and multer does it already: gets the file and stream it to some directory. Unlike some other languages, in Nodejs it's a bit more low-level, and you need to provide the upload destination, to ensure it's available etc.
   So you can create some temp dir, stream there the file, adjust the name, perform checks and everything you might need to do for your uploading strategy
2. MemoryStorage is a bit tricky:
   i. You'll need to have enough available memory for file uploading
   ii. It's a bit more fragile since if the process crashes the Buffer contains the file will be lost

Thank you for your guidance.

Let me explain a scenario based on your instructions. For example, I send a file to the server and upload it to a temporary directory. After the upload, I receive the file buffer and calculate its hash, then search for the hash among the files in the directories.

If the hash exists, I delete the temporary file; otherwise, if the hash does not exist, I need to move the uploaded file to the upload directory and perform the upload there.

Do you think the file upload process will be performed twice if the file does not exist?

Also, if the file already exists, we should delete it from the temporary file directory.

[Doc999tor]

Sorry for the late reply:

1. After the upload, I receive the file buffer

In fact, despite the downsides of using MemoryStorage, it'll work - if the file exists, you just discard the buffer; if it doesn't you can write the buffer to the disk to the relevant directory

2. upload process will be performed twice if the file does not exist

No, in this case the file will be uploaded and processed once - you upload it to MemoryStorage and then write it to the disk or discard

3. we should delete it from the temporary file directory

In this case you don't need a temp directory - you can save the buffer directly to the needed directory