Comments (4)
Thanks for reporting and diving into it!
I'll look at your PR.
I got things working by making Golden Retriever into its own file manager, which stores in memory which files are currently to be uploaded
This defeats the purpose of Golden Retriever, doesn't it? It can't be in-memory because it's meant to survive browser refreshes and restarts, which means everything is gone in memory. That's why it uses IndexedDB/Localstorage/ServiceWorkers.
from uppy.
I was going to submit a seperate feature request for how I got multiple files in recovery state working, but my method currently relies on the on.(file-added) and on.(upload-complete) listeners to set uploadStarted and uploadCompleted, so it is not something I can add into the package.
I got things working by making Golden Retriever into its own file manager, which stores in memory which files are currently to be uploaded. I am working with S3 so not sure how my solution fits into other uploaders. But in simplicity when we emit restore-confirmed Golden Retriever can check if any files in its in-memory store have an upload id and can be resumed (rather than current implementation which attempts to upload all files in recovery). (This involved moving uppy.addFile and uppy.removeFile into GR, so when I add files, I add them to golden retriever, which stores in memory and then adds to uppy)!
I only store 1 file in memory as that's what I need, but I'm sure it would work similarly for multiple files in memory too. Hope this helps for future if this is to be implemented!
handleRestoreConfirmed = () => {
this.uppy.log('[GoldenRetriever] Restore confirmed, proceeding...')
// start all uploads again when file blobs are restored
const { currentUploads } = this.uppy.getState()
if (currentUploads) {
//OLD CODE
// this.uppy.resumeAll()
// Object.keys(currentUploads).forEach((uploadId) => {
// this.uppy.restore(uploadId, currentUploads[uploadId])
// })
for (let uploadId in currentUploads) {
const files = currentUploads[uploadId].fileIDs
console.log('geloo', files)
this.uppy.resumeAll()
if (files.includes(this.currFileId)) {
console.log('geloo', currentUploads[uploadId])
this.uppy.restore(uploadId, currentUploads[uploadId])
break
}
}
}
this.uppy.setState({ recoveredState: null })
}
from uppy.
Just found an edge case where if I don't reset the file isCompleted to false on adding a file to be restored, the progress is 0 until completed. (with the new TotalProgress calculation relying on !isCompleted)
This also made me realise if the user has multiple tabs open, this will cause issues as they will alternate between setting isStarted and isCompleted between tabs and break the local storage state. Thankfully uppy keeps seperate tab state distinct which is good and means we don't need to worry about this, which is very good!
It is a really complex problem to solve I am now realising that but I got it working pretty well, let me know if I can be of any help although my example is for a 1 file multipart s3 uploader. That is all I needed!
from uppy.
This defeats the purpose of Golden Retriever, doesn't it? It can't be in-memory because it's meant to survive browser refreshes and restarts, which means everything is gone in memory. That's why it uses IndexedDB/Localstorage/ServiceWorkers.
Currently from what I've seen is that when using Golden Retriever with Uppy, the local storage is loaded into Uppy state on page load. The single source of truth for a given tab is Uppy which keeps things in memory and writes back to local storage every 0.5 seconds (using GR). In the current Dashboard implementation, if the user chooses not to re-upload the same file, uppy.remove-file is called and the recovered file is completely deleted from Uppy and local storage.
I wanted to preserve the recovery file upload state, even if the user chooses not to re-upload the same file right now. And hence I thought GR in-memory file management would be a good solution, as it keeps Uppy local storage state as the main source of truth, but uses a seperate in-memory store (in GR) to keep track of what the user is uploading in the current tab.
To make this work, I made my uppy max files 25 so we can store upto 25 files for recovery, but GR is aware of a different limit for current uploads.
My use case: (and a potential flaw I just realised)
User will be uploading 1 file in multiple tabs, I would like to detect if the user has previously tried to upload this same file and resume the upload if possible. If not, we can start a new upload, as when the user adds a new file to GR in-memory store, it internally calls uppy.add-file and I use the solution mentioned in #1122 to deactivate all other files in Uppy state, so when I call uppy.upload, only the files which have uploadStarted = false & uploadCompleted = true are uploaded. When the upload completes, I reset all the files as uploadStarted=false and uploadCompleted=true, so they are stored in local storage again for the next visit.
The flaw I just realised is testing the actual use case lol. I just started 3 concurrent uploads in different tabs and closed the window, only one of the tabs was written to local storage. This is race conditions, but I think I can get around this by changing how we write to local storage as I am reading it is atomic and thread safe across tabs!
from uppy.
Related Issues (20)
- Disable browse file / drag and drop on uppy dashboard HOT 2
- Can't resolve '@uppy/core/lib/UploaderPlugin.js HOT 10
- Errors in TypeScript definitions when installed with pnpm HOT 5
- Latest version of @uppy/xhr-upload doesn't support PATCH method HOT 11
- aws-s3-multipart + golden retriever HOT 3
- Event listener that fires before file upload process starts HOT 4
- Drag & Drop not working HOT 1
- Uppy Dashboard does not respect the size of the screen HOT 2
- Dashboard progress icon overlaps with `Remove file` button HOT 1
- Allow unique `id` for React `DragDrop` component HOT 1
- Resume failed with aws-s3-multipart and shouldUseMultipart = true option HOT 1
- New provider views + disabling GoogleDrive "shared drives"
- Wrong font in <Dashboard/> example on https://uppy.io/examples
- Support for Angular 17 HOT 2
- Adding COMPANION_SECRET does not pick up the secret value HOT 1
- use globalThis rather than window HOT 2
- Minio xml result response ends in new upload retry HOT 4
- [4.x] fa_IR locale missing export default
- Implement facebook app secret proof HOT 1
- [4.x] Dashboard doneButtonHandler resets parent state (without re-render)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uppy.