klowner/tussle

Tus daemon written in Typescript using a pluggable architecture

Tussle

Tussle is a mostly spec-compliant server implementation of the tus upload protocol.

Component based

Tussle is implemented as a collection of interchangeable modular components, allowing the developer to choose which services that they'd like to integrate with. At the center of a Tussle instance is the Tussle Core, which handles all communication between individual components.

Components

Middleware

Middleware components handle communication between clients and the Tussle instance.

• middleware-koa - mount a Tussle server as a Koa route handler.
• middleware-cloudflareworker - run Tussle server in a cloudflare worker.

State

Tracking state is tricky and often your options are limited by your deployment environment. Tussle state components are modeled after the Web Storage API

• state-memory - simple in-memory store, works anywhere.
• state-memory-ttl - in-memory store with TTL record expiration, works anywhere.
• state-namespace - wrapper to add transparent key prefixing for other stores.
• state-postgres - store upload state in a PostgreSQL database.
• state-cloudflareworkerkv - store upload state in Cloudflare Workers KV.

Storage

Uploaded files have to be stored somewhere, that's the purpose of the storage components.

• storage-b2 - Backblaze B2 cloud storage (supports large file upload)
• storage-s3 - AWS S3 compatible cloud storage (supports large file upload)
• storage-r2 - Cloudflare R2 cloud storage (supports large file upload, ^0.4.0 adds parallel support)

Request

The request component a responsible for creating outbound HTTP requests and -- when possible/appropriate -- transparently proxy upload payloads, which is determined by the storage component.

• request-axios - Use Axios, suitable for node environments.
• request-cloudflareworker - Use Cloudflare Worker's Fetch() (for use with middleware-cloudflareworker).

Support me

If you find this library useful, please consider buying me a coffee as a way of showing your support!

tussle's Issues

Use crypto.DigestStream in CF Worker environments for chunk checksums

Looks like an unofficial crypto.DigestStream has been added to the workers runtime. This should probably be utilized for chunk checksums on Cloudflare Workers environments.

https://community.cloudflare.com/t/possible-to-compute-sha256-hash-of-large-request-body/328318/5

cloudflare s3 example maybe broken

I tried to play with the CF S3 example (using iDrive E2 storage as an S3 endpoint) and always end-up with the objects named 'files' in the root of the bucket. I am not sure what config is required beyond all of the access/secret/endpoint/etc things. For now I gave up since I did not want to spend digging through the layers of code to see where the problem is. If you have a complete working example that I could learn from, that would be great. Thanks.

Store write failures should result in 500 or 503 response

Right now the create handler returns a 400: Bad Request if there's a write failure by the storage service.
Clients should not re-attempt requests that produce a 400, but in our case, the client should retry.

Consider responding to the client with a 500: Internal Server Error or 503: Service Unavailable.

(tus-js-client does not attempt to retry in the case of a 4xx response)

Handle PATCH offset mismatch with 409 Conflict

It seems this isn't handled currently. It would be nice to have this performed in core, but delegating the responsibility to the storage service and maybe formalizing the potential error responses is a better route.

How can I get all uploaded file list from storage-r2?

Hello
I want to get all file info from storage-r2
https://github.com/Klowner/tussle/blob/main/examples/cloudflare-worker-r2/src/worker.ts
How can I do this?

Cloudflare R2 example

https://github.com/Klowner/tussle/tree/main/examples/cloudflare-worker-r2 works for upload but how does one suppose to reassemble the file for reading/processing/downloading ? Despite Readme stating that files are available for download after uploading, I could not find any meaningful way to do it. Thanks.

Implement termination extension

https://tus.io/protocols/resumable-upload.html#termination

Presently core advertises termination capability but apparently I never implemented it in any of the storage services. After I implement dynamic storage service selection, the storage service itself should indicate whether it supports termination or not.

Help write unit tests

Want to help increase Tussle's test coverage?

Integration and unit tests need to be written for a few components. As of writing, there are integration tests available for state components (see packages/spec/src/stateTests.ts), but storageTests.ts and requestTests.ts are pretty much empty.

Feel free to add anything that increases test coverage.

🥳