How to download multiple files at once?

[sindresorhus]

How to download multiple files at once?

[kbuffington]

I spent a fair amount of time looking into this tonight and it seems that there's just no good way to handle simultaneous downloads. registerListener is going to create multiple listeners for 'will-download' and then when the downloads complete multiple listeners will be getting the callback and it becomes a whole mess, and all downloads get saved under the same file name, progress gets screwed up etc. Maybe there's a solution, but I couldn't figure one out.

What I did was make it so you can't have multiple simultaneous downloads by implementing a download queue:

class DownloadQueue extends Array {
    constructor(...args) {
        super(...args);
    }
    push(item) {
        const len = super.push(item);
        if (this.length === 1) {
            this.download(item);
        }
        return len;
    }
    shift() {
        const item = super.shift();
        if (this.length > 0) {
            this.download(this[0]);
        }
        return item;
    }
    download(item) {
        item.options.onCompleted = () => {
            this.shift();
        };
        download(item.win, item.url, item.options);
    }
}

And then your 'download' listener becomes:

const downloadQueue = new DownloadQueue();

ipcMain.on('download', async (event, info) => {
    info.win = BrowserWindow.getFocusedWindow();
    downloadQueue.push(info);
}

[alexhx5]

@itzsaga @sindresorhus guys, do you have any advice on how to fix onProgress events with simultaneous downloading?

When I try to download 2nd file while 1st file is downloading, it downloads both files but it starts emitting onProgress events unexpectedly during the downloading.

Code

I modified the module a bit so it accepts other arguments as well, not just the url:

ipcMain.on('download-file', async (event, { url, directory, hash }) => {
  await download(mainWindow, { 
    url, 
    directory,
    hash,
    onProgress(progress) {
      event.sender.send('download-file-progress', progress)
    }
  })
})

Log:

• Started downloading 1st file blender-2.81.tar.xz
• After 5 seconds started downloading 2nd file blender-2.81-windows64.msi

I passed hash-1 for the 1st file and hash-2 for the 2nd into the download() with the file URL just for the log.

Compare the percentages and the hashes, look at how hashses do not correspond correctly to their files after I started downloading 2nd file:

blender-2.81.tar.xz  - 0.00% - hash-1
blender-2.81.tar.xz  - 1.88% - hash-1
blender-2.81.tar.xz  - 3.88% - hash-1
blender-2.81.tar.xz  - 5.89% - hash-1
blender-2.81.tar.xz  - 8.14% - hash-1
blender-2.81.tar.xz  - 10.40% - hash-1
blender-2.81.tar.xz  - 11.40% - hash-1
blender-2.81.tar.xz  - 13.65% - hash-1
blender-2.81.tar.xz  - 15.74% - hash-1
blender-2.81.tar.xz  - 17.79% - hash-1
blender-2.81-windows64.msi  - 0.00% - hash-1
blender-2.81-windows64.msi  - 0.00% - hash-2
blender-2.81-windows64.msi  - 0.00% - hash-1
blender-2.81-windows64.msi  - 0.00% - hash-2
blender-2.81-windows64.msi  - 19.83% - hash-1
blender-2.81-windows64.msi  - 0.50% - hash-1
blender-2.81-windows64.msi  - 0.50% - hash-2
blender-2.81-windows64.msi  - 21.92% - hash-1
blender-2.81-windows64.msi  - 1.56% - hash-1
blender-2.81-windows64.msi  - 1.56% - hash-2
blender-2.81-windows64.msi  - 23.21% - hash-1
blender-2.81-windows64.msi  - 2.77% - hash-1
blender-2.81-windows64.msi  - 2.77% - hash-2
blender-2.81-windows64.msi  - 24.51% - hash-1
blender-2.81-windows64.msi  - 4.03% - hash-1
blender-2.81-windows64.msi  - 4.03% - hash-2
blender-2.81-windows64.msi  - 26.01% - hash-1
blender-2.81-windows64.msi  - 5.33% - hash-1
blender-2.81-windows64.msi  - 5.33% - hash-2
blender-2.81-windows64.msi  - 27.30% - hash-1
blender-2.81-windows64.msi  - 6.48% - hash-1
blender-2.81-windows64.msi  - 6.48% - hash-2
blender-2.81-windows64.msi  - 28.64% - hash-1
...

[itzsaga]

We do it with this code for an array of objects where there is a url key on each object.

ipcMain.on("download-files", async (event, files) => {
  const downloadLocation = app.getPath("userData");

  const promises = files.map(file =>
    download(mainWindow, file.url, {
      saveAs: false,
      directory: downloadLocation,
      onProgress: progress => {
        event.sender.send("download-progress", { progress, file });
      }
    })
  );

  await Promise.all(promises);
});

[SushyDev]

having similar issues, downloading multiple files at the same time makes data leak into the wrong item. For example the progress of file 2 switches back and forth on file 1. So if file 1 is on 10% and file 2 on 20%, file 1 will show 10% then 20% then 10% then 20% on and on again

[lohannon]

We ran into this as well, where we needed to download two sets of files, each with a dynamic number of files. Some files from one set were getting written to the directory of the other. The way we had to get around this was to make everything synchronous. This ended up working for us:

async function downloadFile({ directory, url, filename }): Promise<string> {
  return new Promise((resolve) => {
    download(mainWindow, url, {
      directory,
      filename,
      saveAs: false,
      showBadge: false,
      onCompleted: ({ path }) => {
        resolve(path);
      },
    });
  });
}

async function downloadFiles(
  directory: string,
  files: Array<{ url: string; filename: string }>
): Promise<string[]> {
  return await files.reduce(
    async (result: Promise<Array<string>>, { url, filename }) => {
      const awaitedResults = await result;
      try {
        const downloadedFile = await downloadFile({ directory, url, filename });
        return [...awaitedResults, downloadedFile];
      } catch (err) {}
    },
    Promise.resolve([])
  );
}

[ribsies]

Running into this issue as well.

It looks like the options are being set globally and it looks like only one download can exist at a time.

Lets say I have 10 files I am downloading, and calling the download function in a foreach function.
By the end of it I only have 1 file downloaded, which has the contents of the first download in the list but the name of the last item.

I haven't had time to test but it looks like you might have to wait for a download to completely finish before doing another one. Which is not ideal.

[gihanrangana]

class DownloadManager {
    public window: any = null
    public paths: any[] = []
    public options: any = {}

    constructor (window: any, paths: any[], options: {} = {}) {
        this.window = window;
        this.paths = paths;
        this.options = options
    }

    download = async (options?: any) => {
        for (let item of this.paths) {
            const dl: any = await download(this.window, item.url, {
                ...this.options,
                filename: item.name + '.zip',
                onProgress: (progress: any) => {
                    App.window.webContents.send(options?.progressEventName || 'download-progress', { ...progress, key: item.name })
                }
            })
            if (dl.isDone) {
                App.window.webContents.send(options?.successEventName || 'download-complete', { key: item.name })
            }
        }
    }

}

[j-ribs]

@gihanrangana

Can you give some more context to your code? How and where are you calling this from? Whats the download function thats being awaited there. Missing some key points here.

[theogravity]

I wrote a library that addresses the multi-download issue and also supplies an id to the download as well:

https://www.npmjs.com/package/electron-dl-manager