tusdotnet/tusdotnet

.NET server implementation of the Tus protocol for resumable file uploads. Read more at https://tus.io

tusdotnet

"Our aim is to solve the problem of unreliable file uploads once and for all. tus is a new open protocol for resumable uploads built on HTTP. It offers simple, cheap and reusable stacks for clients and servers. It supports any language, any platform and any network." - https://tus.io

tusdotnet is a .NET server implementation of the tus.io protocol that runs on .NET Framework, .NET Standard, .NET6 and later.

Comments, ideas, questions and PRs are welcome!

Features

• Runs on .NET Framework, .NET Standard 1.3+ and .NET 6+ using OWIN or ASP.NET Core
• Full support for tus 1.0.0 including all major extensions (checksum, checksum-trailers, concatenation, creation, creation-with-upload, upload-defer-length, expiration and termination)
• Experimental support for IETF's Resumable Uploads For Http (see branch POC/tus2)
• Fast and reliable
• Easy to configure
• Customizable data storage
• MIT licensed

Install

Visual Studio

PM> Install-Package tusdotnet

.NET CLI

> dotnet add package tusdotnet

Configure

On .NET6 and later:

using tusdotnet;

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapTus("/files", async httpContext => new()
{
    // This method is called on each request so different configurations can be returned per user, domain, path etc.
    // Return null to disable tusdotnet for the current request.

    // Where to store data?
    Store = new tusdotnet.Stores.TusDiskStore(@"C:\tusfiles\"),
    Events = new()
    {
        // What to do when file is completely uploaded?
        OnFileCompleteAsync = async eventContext =>
        {
            tusdotnet.Interfaces.ITusFile file = await eventContext.GetFileAsync();
            Dictionary<string, tusdotnet.Models.Metadata> metadata = await file.GetMetadataAsync(eventContext.CancellationToken);
            using Stream content = await file.GetContentAsync(eventContext.CancellationToken);

            await DoSomeProcessing(content, metadata);
        }
    }
});

Depending on your infrastructure you might also need to configure Kestrel, IIS or other reverse proxies.

More options and events are available on the wiki.

On older frameworks, use the tusdotnet middelware

Create your Startup class as you would normally do. Add a using statement for tusdotnet and run UseTus on the app builder. Depending on your infrastructure you might also need to configure Kestrel, IIS or other reverse proxies. More options and events are available on the wiki.

app.UseTus(httpContext => new DefaultTusConfiguration
{
    // This method is called on each request so different configurations can be returned per user, domain, path etc.
    // Return null to disable tusdotnet for the current request.

    // c:\tusfiles is where to store files
    Store = new TusDiskStore(@"C:\tusfiles\"),
    // On what url should we listen for uploads?
    UrlPath = "/files",
    Events = new Events
    {
        OnFileCompleteAsync = async eventContext =>
        {
            ITusFile file = await eventContext.GetFileAsync();
            Dictionary<string, Metadata> metadata = await file.GetMetadataAsync(eventContext.CancellationToken);
            using Stream content = await file.GetContentAsync(eventContext.CancellationToken);

            await DoSomeProcessing(content, metadata);
        }
    }
});

Test sites

If you just want to play around with tusdotnet/the tus protocol, clone the repo and run one of the test sites. They each launch a small site running tusdotnet and the official JS client so that you can test the protocol on your own machine.

Test sites are available for:

• ASP.NET Core 6 (.NET 6.0)
• ASP.NET Core 3.1 (.NET Core 3.1)
• ASP.NET Core 3.0 (.NET Core 3.0)
• ASP.NET Core 2.2 (.NET Core 2.2)
• ASP.NET Core 2.2 (.NET Framework 4.6.2)
• ASP.NET Core 2.1 (.NET Core 2.1)
• OWIN (.NET Framework 4.5.2)

Clients

tus.io keeps a list of clients for a number of different platforms (Android, Java, JS, iOS etc). tusdotnet should work with all of them as long as they support version 1.0.0 of the protocol.

License

This project is licensed under the MIT license, see LICENSE.

Want to know more?

Check out the wiki or create an issue

tusdotnet's Issues

Exceptions in OnUploadCompleteAsync

I realized that all exception happening in OnUploadCompleteAsync are silent and the application continues to work. Is there any chance to handle them and pop higher custom responses?

Move events to its own property

Inspired by authentication in ASP.NET Core 2.0 all events shall be moved from being properties on DefaultTusConfiguration to a new property called "Events".

This will make the purpose of both the configuration object and the events object easier to understand.

Add missing tests for DELETE and PATCH to return 404 Not Found when file has expired

This currently works but there are no tests assuring that it does.

UrlPath problem

I have an app that can be accessed from two different URLs (www.example.com/app1 and www.example.com/app2). The UrlPath parameter requires including the path base. Do I have to configure 2 Tus instances? Could we have a UrlPattern instead so that all URLs that look like "/[^/]*/files/" would be handled by Tus.

The app builder extension for Owin is not chainable

The app builder extension for Owin returns void instead of itself and is thus not chainable.

Add support for Expiration

http://tus.io/protocols/resumable-upload.html#expiration

Add an optional interface to be implemented by a data store, e.g. ITusExpirationStore. TusDiskStore should implement some kind of cleanup job that runs in the background removing expired files.

Change signature of OnUploadCompleteAsync

OnUploadCompleteAsync shall be changed so that it takes a single "context" argument that contains all properties needed for the event.

This will change OnUploadCompleteAsync from
public Func<string, ITusStore, CancellationToken, Task> OnUploadCompleteAsync { get; set; }
to
public Func<FileCompleteContext, Task> OnFileCompleteAsync { get; set }

FileCompleteContext will contain the file id (the first string), the store and the cancellation token.

progress

how to response the progress to user?

Original file name in OnUploadCompleteAsync

Is there any way to get an original file name in OnUploadCompleteAsync callback?

Facing CORS error while uploading file from another domain

Hi,
While trying to implement your code, everything works with Tus JS clinet, as long I don't need CORS. By using another url for upload services and another domain for client-side app, I face 404 error on PATCH request.
How can I fix this error?

400 error?

Hi - done some work adapting this project for Azure blob storage (still in dev). Works great with included JS client but I am now trying with the Android Client and get a 400 error. I have the android client working with the official test instance at https://master.tus.io/files - so that's known working too. Any idea what's going wrong? Using the Owin netcore app btw.

Thanks for your work on this project!

VS 2017 ASP.NET Core Web Application (.NET Framework)

I am trying to install this package with VS 2017, into a new ASP.NET Core Web Application (.NET Framework) template. I get the following error:

Severity	Code	Description	Project	File	Line	Suppression State
Error	CS1929	'IApplicationBuilder' does not contain a definition for 'UseTus' and the best extension method overload 'TusAppBuilderExtensions.UseTus(IAppBuilder, Func<IOwinRequest, ITusConfiguration>)' requires a receiver of type 'IAppBuilder'	

IApplicationBuilder app resolves to Microsoft.AspNetCore.Builder.IApplicationBuilder, but it looks like your code is looking for OWIN namespace, not the Microsoft.AspNetCore namespace.

2.0 release plan

This is the release plan for tusdotnet 2.0 and what remains before it can be shipped of to nuget.

• Update tusdotnet\Source\tusdotnet\readme.txt to use the new OnFileComplete event
• Add documentation on the wiki for new events
• Updated Validate metadata before creating a file
• Update Processing a file once the file upload is completed
• Update readme.md to use the new OnFileComplete event and to link to new configuration page for events
• Create nuget package and success!

Add support for Upload-Defer-Length

http://tus.io/protocols/resumable-upload.html#upload-defer-length

As this is optional, add an additional interface, e.g. ITusCreationUploadDeferLengthStore (bad name...).
Remember to add "creation-defer-length" to Tus-Extension and to implement patch accordingly.

Can't set large maximum upload size

DefaultTusConfiguration.MaxAllowedUploadSizeInBytes should be long not int I think in order to allow for larger file sizes.

Add support for Termination

http://tus.io/protocols/resumable-upload.html#termination

Create an optional interface to be implemented in a data store, e.g. ITusTerminationStore.

Re-use client disconnect functionality in other places than inside tusdotnet

The current implementation of handling client disconnects cross pipeline resides in the Patch handler. This shall be moved to its own class so that it can be re-used from third party stores. Something like ClientDisconnectGuard.ExecuteAsync(Func<Task>...)

Update the AspNetCoreTestApp to the same standard as the OwinTestApp

AspNetCoreTestApp does not contain all the features that the OwinTestApp does.

Add support for OnCreateComplete

OnCreateCompleteAsync(CreateCompleteContext) shall be called once a file has been created. This is purely an information event (much as OnFileComplete).

Add support for X-HTTP-Method-Override

"The X-HTTP-Method-Override request header MUST be a string which MUST be interpreted as the request’s method by the Server, if the header is presented. The actual method of the request MUST be ignored. The Client SHOULD use this header if its environment does not support the PATCH or DELETE methods."

Should be implemented in TusMiddleware for all stores to enjoy :)

how i can use this with winform application

your wiki is empty please can you guide me how can i use this library with my winform application...

Add support for .NET Core

Add additional targeting for .NET Core. Even better if it could target .NET Standard.

Validation

Hi, I would like to thank you for tus project,
how i can validate the file before upload?

Client example

Is there any plans on implementing a reference client application for dotnet? I see that there are no examples for the uploading between client/server.

Thanks.

Accessing DB context from

How can I access my DB context from event handlers? For example I would like to check access rights in a different method, hand out tokens for my clients and then in OnBeforeCreateAsync connect to database and check that the valid token has been supplied or update database that file has been uploaded in OnFileCompleteAsync.

Upload-Metadata is not included in HEAD response

According to spec: "If an upload contains additional metadata, responses to HEAD requests MUST include the Upload-Metadata header and its value as specified by the Client during the creation."

Source: http://tus.io/protocols/resumable-upload.html#upload-metadata

Add helpers for setting up CORS

Currently every dev needs to add the correct exposed headers which is error prone and just bad design. We should add a helper method to get the exposed headers or the entier CORS setup object.

Uploaded files aren't accessible

I'm using just exact usage example from the main GH page and it works for upload (although it works only through debug mode of VS, doesn't when I try to make upload through IIS application, but that's not the issue). One file is half-uploaded, I refresh page, and trying to continue, but since http://locahost:12345/files/4743819885739485834798324 is 404 it starts from scratch. Same problem for fully uploaded files - I see them in the upload folder, but they're not accessible by URL.
I'm running it along with Umbraco CMS if it's important.
Do you have any thoughts why's so?

Expiration flag

Is it possible to remove expiration flag using the contract ITusExpirationStore.cs , It's good idea to upload temporary files and when the file been confirmed for any other operation file will be saved as permanent file.

any idea how to implement such kind to situation.

Add support for OnBeforeCreateAsync

Called before a file is created to allow validation of a file before it is created. The context should contain a FailRequest(string message) method that causes tusdotnet to return a 400 Bad Request to the client. BeforeCreateContext should contain the parsed metadata to make it easier to validate. Ref: #24

The context must also contain other data specific to the creation POST such as metadata, Upload-Length (or Upload-Defer-Length) etc.

Add support for Concatenation

http://tus.io/protocols/resumable-upload.html#concatenation

Add a new optional interface to be implemented by a data store, e.g. ITusConcatenationStore

Document configuration in the wiki

In #14 it was noted that in some cases you need to add context.PathBase to the UrlPath property. This, and other, configuration needs to be documented on the wiki

Proposed features for tusdotnet 2.0

tusdotnet 2.0 proposal

NOTE tusdotnet 2.0 is now feature complete. Some proposed features in this issue will be moved to later release.

These features are:

• Add support for IDisposable stores
• Add support for custom file locks
• Restructure code on disk to better conform to .NET Core standard

---

tusdotnet 2.0 will be the first release with breaking changes. This issue contains proposed features and shall focus on features that are breaking to minimize the impact of upgrading for users.

Proposed features that are referenced with an issue number are ready for development. The ones without need more refinement. This document is continually updated.

Link to all issues tagged as upcoming for the 2.0 release: https://github.com/tusdotnet/tusdotnet/milestone/1

Code structure

Move models, interfaces and some helpers into a new Nuget package called e.g. "tusdotnet.Abstractions".
This would ease the use of sharing models, interfaces etc between tusdotnet and third party stores.

Restructure code on disk to better conform to .NET Core standard, i.e. a src and a test folder. As of VS 2017 we can also probably remove our custom build scripts for publishing to Nuget.

Configuration

Remove ITusConfiguration

Issue: #26
Status: Done

This interface is an extra abstraction without a clear purpose. Modify DefaultTusConfiguration to have virtual methods and properties and remove ITusConfiguration.

Async configuration factory

Issue: #27
Status: Done

The configuration factory on app.UseTus(factory) does currently not support the async keyword. An async overload shall be added to allow higher throughput. The sync version must be kept.

Move events to its own property

Issue: #29
Status: Done

Inspired by authentication in ASP.NET Core 2.0 all events should be moved from being properties on DefaultTusConfiguration to a new property called "Events".
This would make the purpose of both the configuration object and the events object easier to understand.

Change signature of OnUploadCompleteAsync

Issue: #30
Status: Done

OnUploadCompleteAsync should be changed so that it takes a single "context" argument that contains all properties needed for the event.

This would change OnUploadCompleteAsync from
public Func<string, ITusStore, CancellationToken, Task> OnUploadCompleteAsync { get; set; }
to
public Func<UploadCompleteContext, Task> OnUploadCompleteAsync { get; set }

UploadCompleteContext would contain the file id (the first string), the store and the cancellation token.

Add more events

Properties contained in the context must be specified for each event and might differ between events. Most likely most context's will contain the file id, the store and the request's cancellation token.

New proposed events and what they shall be used for:

• OnBeforeCreateAsync(BeforeCreateContext) - Called before a file is created to allow validation of a file before it is created. The context should contain a FailRequest(string message) method that causes tusdotnet to return a 400 Bad Request to the client. BeforeCreateContext should contain the parsed metadata to make it easier to validate. Ref: #24 Issue: #33 Status: Done
• OnCreateCompleteAsync(CreateCompleteContext) - Called once a file has been created. Issue: #35 Status: Done
• OnBeforeDeleteAsync(BeforeDeleteContext) - Called before a file is deleted to allow validation that a file can be deleted. The context should contain a FailRequest(string message) method that causes tusdotnet to return 400 Bad Request to the client. Issue: #34 Status: Done
• OnDeleteCompleteAsync(DeleteCompleteContext) - Called once a file has been deleted. Issue: #36
• OnBeforeUploadAsync(BeforeUploadContext) - Called each time a upload (transmission of data) is started. Might be called multiple times for a single file if the client disconnects. NOTE: This event will not be implemented. See comment in #30.
• OnUploadCompleteAsync(UploadCompleteContext) - Called once a file has been completely uploaded. Issue: #30 Status: Done
• OnClientDisconnectAsync(ClientDisconnectContext) - Called when a client disconnects. The usefulness of this event is probably debatable and should be added at a later stage.

Stores

Add support for IDisposable stores

Make IDisposable an optional interface to implement for stores. This is not a breaking change as long as the TusDiskStore does not implement IDisposable.

Add support for custom file locks

This shall be implemented as an extra interface such as ITusFileLockStore. The store that implements this interface must then return a new lock that will be used for file locks.

tusdotnet needs this to add support for clustered environments and stores that save data externally from the server that runs tusdotnet (e.g. Azure blob storage).

Make the current file lock internal and rename it to InMemoryFileLock. tusdotnet shall use the InMemoryFileLock in cases where the store does not implement a custom lock.

For this to work with TusDiskStore it must be changed to save a lock file on disk (e.g. <fileId>.lock) instead of using the current FileLock.

Generic code fixes

Client disconnect guard

Issue: #31
Status: Done

The current implementation of handling client disconnects cross pipeline resides in the Patch handler. This should be moved to its own class so that it can be re-used from third party stores. Something like ClientDisconnectGuard.ExecuteAsync(Func<Task>...)

Run Stylecop/Roslynator on the source code

Setup common rules and practices to use and use a tool such as Stylecop or Roslynator to adhere to the rules.

OnUploadCompleteAsync Call on already uploaded file upload attempt

OnUploadCompleteAsync is firing once upload is done, although it doesn't when it's done for already uploaded files. Is there any way to make it calling in such situations as well?

Short-circuit processing if configuration factory returns null?

I've only just begun reviewing tus and this project as a potential option to replace our current upload impl, so please disregard if this is naive...

I'm considering the case where we'd want to leverage this middleware for multiple endpoints. If in my configuration factory the request doesn't match one of those urls, it appears I would currently need to return a bogus/noop configuration in order to bypass processing? I'm wondering if it makes sense to be able to short-circuit by returning a null config in that case?

While we're not committed to tus yet, I appreciate your efforts on this project as it makes it a more viable option for us!

Custom file name with extension

How can i custom filename (add extension)?
IIS can only serve static file if it has an extension.

Async configuration factory

The configuration factory on app.UseTus(factory) does currently not support the async keyword. An async overload shall be added to allow higher throughput. The sync version must be kept.

Add support for Checksum

http://tus.io/protocols/resumable-upload.html#checksum

Add a new optional interface to be implemented by a data store, e.g. ITusChecksumStore

Request finished in 120544.0557ms 400 text/plain

I send request of Creation by postman, I received 400 response , The tusdotnet site is the AspNetCore_netcoreapp2.1_TestApp testSite ,
the postman http request as follows:

the server info as follows:

Remove ITusConfiguration

This interface is an extra abstraction without a clear purpose. Modify DefaultTusConfiguration to have virtual methods and properties and remove ITusConfiguration.

Add support for OnDeleteComplete

OnDeleteCompleteAsync(DeleteCompleteContext) shall be called once a file has been deleted. This is purely an information event.

Header location

When you are in local host or standalone application on IIS Header location is correct using UrlPath configuration but the problem when you are in sub application the Header location might be different than the UrlPath i solve this problem by modify the default configuration and add my custom header location response.
https://github.com/tusdotnet/tusdotnet/blob/master/Source/tusdotnet/ProtocolHandlers/PostHandler.cs#L122

response.SetHeader(HeaderConstants.Location, $"context.Configuration.LocationPath.TrimEnd('/')}/{fileId}");

i am not sure if there is better solution such as get the Request path and add it as header or make it customizable by adding path of header response in configuration.

Make it easier to run tusdotnet on ASP.NET Core on .NET Framework

Side note: Microsoft suck at naming stuff :)

tusdotnet runs just fine on ASP.NET Core on both .NET Framework and .NET Core (the actual runtimes). The problem is that since we use build targets to expose the correct middleware (as there is no other viable way of knowing what pipeline we run) users who run ASP.NET Core on .NET Framework need to setup an extra OWIN pipeline to be able to run tusdotnet. This is bad in terms of performance while also introducing a high threshold to run tusdotnet.

Read more: https://github.com/tusdotnet/tusdotnet/wiki/Troubleshoot:-"'IApplicationBuilder'-does-not-contain-a-definition-for-'UseTus'"-in-ASP.NET-Core

We should expose the ASP.NET Core middleware even if the build target is net452.
Pros:

• No need to setup an extra OWIN pipeline to get started. Just use app.UseTus(...) and it is done.

Cons:

• OWIN and ASP.NET Core middlewares would both be exposed meaning that OWIN users would also need to install Microsoft.AspNetCore.Http.Abstractions and ASP.NET Core users would need to install Owin and Microsoft.Owin. This only applies if running on .NET Framework. For .NET Core there would be no difference.

Access metadata on OnFileCompleteAsync

It will be great if i can access Metadata OnFileCompleteAsync context to insure saving data i have sent from the front-end to back-end after the file uploaded.
there is any way to tweak this

Upload-Checksum does not work properly with chunked uploads

Spec: https://tus.io/protocols/resumable-upload.html#checksum

"A Client MAY include the Upload-Checksum header in a PATCH request. Once the entire request has been received, the Server MUST verify the uploaded chunk against the provided checksum using the specified algorithm. "

This only works correctly if the entire file is uploaded in a single chunk using streaming. If the client uploads the file chunked (and each chunk is streamed), then the verification of each chunk's checksum will always fail.

Issue with cross domain requests using the official JS client

Split from #2 as the problem is not related to CORS. Comments have been paste below in this comment:

mjpournassari commented a day ago
I used your js file and instead of 404, I get 405 Method Not Allowed on PATCH requests. OPTIONS and POST request are working by the way. 404 happens when X-HTTP-Method-Override is set to true and tus uses POST instead of PATCH.

smatsson commented 23 hours ago
@mjpournassari X-HTTP-Method-Override is not yet implemented as per the roadmap so that will not work. Are you running the site as a web application? In that case you need to update your web.config to run your code for all types of HTTP verbs:

<system.webServer>
        <validation validateIntegratedModeConfiguration="false"/>
        <handlers>
            <remove name="ExtensionlessUrlHandler-Integrated-4.0" />
            <remove name="OPTIONSVerbHandler" />
            <remove name="TRACEVerbHandler" />
            <add name="ExtensionlessUrlHandler-Integrated-4.0" path="*." verb="*" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
        </handlers>
        <modules runAllManagedModulesForAllRequests="true">
            <remove name="ApplicationInsightsWebTracking"/>
            <remove name="WebDAVModule" />
        </modules>
    </system.webServer>

mjpournassari commented 23 hours ago • edited
@smatsson Hi
My project is Web Api. In the web.config file handlers are exists.

smatsson commented 23 hours ago
@mjpournassari Can't connect to the site. Did you add runAllManagedModulesForAllRequests="true"? If so, please provide me with the code that is not working (i.e. your solution) and I'll have a look.
@mjpournassari

mjpournassari commented 22 hours ago
@smatsson Yes runAllManagedModulesForAllRequests="true.
I attached my Web Api project.

smatsson commented 21 hours ago
@mjpournassari Still can't connect but it doesn't really matter. Viewing your upload page does not give me any info that your solution won't give me. I'll have a look at it when I get the time.
@mjpournassari

mjpournassari commented 21 hours ago
@smatsson Thank you

Design questions ...

I have two questions, basically. I suspect they're a matter of opinion, but if you agree with me, I'd be willing to send some PRs ...

First, it seems it might be better if there was an abstract base class for the configuration.

I saw from the docs that you must have used an interface at one time, and I so I wondered what made you decide to make DefaultTusConfiguration not be an abstract base. That way, instead of needing to write my whole implementation inline in my Startup class, I could just subclass BaseTusConfiguration and override whatever I need to ...

I was able to treat it as a base class, and derive from it, except that because the events are in their own class, I still had to hook up all the logic in the constructor ...

Second, what's the use case for a write-only store?

Separating all of the storage interfaces individually (without a dependency chain), seems to make this awkward to use with a dependency injection model.

Specifically, I want to register the store for use in my configuration, something like this (in ASP.NET Core):

services.AddTransient<ITusStore>(provider => 
                  new TusDiskStore(Configuration.GetValue("uploadPath", @"S:\")));

However, when the uploads are done, I need to do something with them, which means I need an ITusReadableStore ... is it reasonable to register another interface of the TusDiskStore for each feature I need?

Am I missing something, or would it be reasonable that every Store must derive from ITusStore, and perhaps even require the implementation of the GetFileAsync method?

I'm honestly not sure how your actual implementation is done -- if I provide a Store that's missing interfaces, do you just disable that functionality on the middleware? Could there ever be such a thing as a write-only or read-only store?

Add support for Tus-Max-Size

http://tus.io/protocols/resumable-upload.html#tus-max-size

Add a nullable int (number of bytes allowed) to the configuration object where one can specify a Max-Upload-Length. This should be handled by TusMiddleware (and not be data store specific).

Add support for OnBeforeDeleteAsync

OnBeforeDeleteAsync(BeforeDeleteContext) shall be called before a file is deleted to allow validation that a file can be deleted. The context should contain a FailRequest(string message) method that causes tusdotnet to return 400 Bad Request to the client.

Wrap all store invokations in a try catch with TusStoreException

This is done for some operations but not all. A TusStoreException returns a 400 Bad Request to the client together with the exception message that was thrown.

Also update documentation on the wiki and add this information as it is currently missing. https://github.com/smatsson/tusdotnet/wiki/Custom-data-store