microcompiler/backblaze

Backblaze.Agent is a high-performance .NET Core implementation of the Backblaze B2 Cloud Storage API.

backblaze's Issues

Add support for the new 'b2_copy_file' endpoint

Support creating a new file by copying from an existing file.

• https://www.backblaze.com/b2/docs/b2_copy_file.html

Add support for disable checksum option using "do_not_verify" sha1

Specifying 'do_not_verify' as the checksum and letting B2 compute the checksum of the file. Option not recommended.

• https://www.backblaze.com/b2/docs/uploading.html

Issue connecting to backblaze client

Message = "Error converting value "readBuckets" to type 'Bytewizer.Backblaze.Models.Capability'. Path 'allowed.capabilities[6]', line 14, position 19."

Anyone have any idea? The API key I setup has access to 'readBuckets'

*edit
Appears the api has new return values that aren't supported by the current version. Also the UploadAsync function appears to be super slow when uploading files of any size. Anyone experiencing this?

Delete file

How do I delete a file?

Bugs in Backblaze B2 path handling

This issue is not a bug in this code, but it is a bug and it affects code that is using this library.

Based on hints in the documentation, it is suggested that the Backblaze B2 API was initially created as an exclusively HTTP POST based system, with parameters in JSON bodies or HTTP headers. More recently, a push was made to transition to idiomatic HTTP GET for get-like endpoints, which put file paths into the request URL. I have encountered problems with this, with certain characters that are explicitly allowed in filenames (per the documentation) not working with common endpoints like b2_download_file_by_name.

After some investigation, the work-around that I have come to is to transition any operation where a problematic filename is causing problems to the corresponding one using fileId. This means that an additional operation is needed to determine the fileId value that corresponds to the filename, but this operation can be done in a way that does not have these problems with common characters in filenames.

Characters that I have encountered causing this problem include:

• , (comma)
• [ (open square bracket)
• ] (close square bracket)
• & (ampersand)
• _ (underscore)

One possible course forward would be to codify the workaround in the implementation of DownloadFileByName so that that type of request just works even if an actual b2_download_file_by_name would have failed.

Another possibility is merely to document the problem so that developers know it exists.

This is, in essence, the function I wrote to facilitate translating a request from a filename to a fileId-based request:

    async Task<string?> GetFileIdByName(string remoteStorageBucketId, string fileName, bool throwIfNotFound = true)
    {
      var request = new ListFileVersionRequest(remoteStorageBucketId);

      request.StartFileName = fileName;
      request.MaxFileCount = 1;

      var response = await _b2Client.Files.ListVersionsAsync(request, cacheTTL: TimeSpan.FromSeconds(10));

      response.EnsureSuccessStatusCode();
    
      var file = response.Response.Files.SingleOrDefault(file => file.FileName == fileName);

      if (file == null)
      {
        if (throwIfNotFound)
          throw new FileNotFoundException();
        else
          return null;
      }

      return file.FileId;
    }

This function can then be used like this:

      Task<IApiResults<DownloadFileResponse>> task;

      if (fileName.IndexOfAny(B2ProblematicFileNameCharacters) < 0)
      {
        // Fast path: b2_download_file_by_name
        var request = new DownloadFileByNameRequest(FindAndCacheBucketName(), fileName);

        task = _b2Client.DownloadAsync(request, destinationStream, default, cancellationToken);
      }
      else
      {
        // Workaround: b2_list_file_versions -> b2_download_file_by_id
        var fileID = GetFileIdByName(fileName);

        var request = new DownloadFileByIdRequest(fileID);

        task = _b2Client.DownloadByIdAsync(request, destinationStream, default, cancellationToken);
      }

      var result = await task;

      if (!result.IsSuccessStatusCode)
        throw new Exception("The operation did not complete successfully.");

(NB, these functions might have some minor errors and may require a bit of work, because I had to modify the actual code from my project to remove unnecessary complexity, so this is not exactly the state of the code in my codebase.)

Hopefully this information can be useful to somebody :-)

Error converting value "writeBucketReplications" to typ

I have this capabiltiy set on my key. This happens when I try to connect

Newtonsoft.Json.JsonSerializationException: 'Error converting value "writeBucketReplications" to type 'Bytewizer.Backblaze.Models.Capability'. Path 'allowed.capabilities[1]', line 9, position 31.'

InnerException:
ArgumentException: Requested value 'writeBucketReplications' was not found.

seems to be the same reason as #19

Add file lock feature to client endpoints

Add support for the new 'b2_copy_part' endpoint

Support copies from an existing B2 file, storing it as a part of a large file which has already been started

• https://www.backblaze.com/b2/docs/b2_copy_part.html

Cannot make folder requests:

Here link it says that you can make folder requests, getting only top level folders by doing:

With a prefix of "" (meaning all files), and a delimiter of "/", you would get:

folder photos/

This doesn't work with your library.

I'm just getting an exception saying that fileId is required. FileId is not required in this case. I checked in postman and it returns top level "folders" correctly.

[Not a bug] Is there a roadmap or list of active issues? You mentioned this repo is a work in progress, i'd like to help

Hello!

Thanks for the AMAZING library, its easier than working with CLI tools, then parsing the output and boxing the JSON into objects! YOU ARE A HERO!

You mentioned this is a Work in Progress, I was wondering a few things:

.) Do you have a list of features like:L

• This is done
• This is working and unstable, testing required
• This is under-developement
• This is todo

I would love to contribute how I can. The list will help alot so that I am not blindly working with the codebase.

Thanks again!

Missing 'listAllBucketNames' application key capability name

Backblaze/b2-sdk-java#108

In asp.net core's dependency injection, do I need to connect first? The documentation doesn't seem to mention it.

In asp.net core's dependency injection, do I need to connect first? The documentation doesn't seem to mention it！

Item with the same name

Thanks for creating this it has made life much better.

I have an issue that is probably something I have done but I can't figure it out. This issue does not come up until I install this on another computer as a service.

An item with the same key has already been added. Key: large_file_sha1.

If I shutdown the service and restart the first uploads will work. Any thoughts?

[Question] Can I specify the keyId and application key after the serviceCollection is built?

I don't want to specify the keyId and application key when building the service collection but only when I upload a file for example. Is that possible? I still want to use dependency injection.

Add server-side encryption to client endpoints

UriFormatExceptionSystem thrown when calling UploadAsync

First off: Thank you for making a nice client for the Backblaze B2 API!

When calling UploadAsync in BackblazeAgent the following exception is thrown:
System.UriFormatException: 'Invalid URI: The format of the URI could not be determined.'

I believe the exception happens when UploadFileByBucketIdRequest.cs tried to convert the filename into a URI. Notice I'm trying to give only the filename - not the entire file path.

Example code:

foreach (var filepath in Directory.GetFiles(@"c:\my\directory"))
{
  using (var stream = File.OpenRead(filepath))
    await storage.UploadAsync(new UploadFileByBucketIdRequest(bucket.BucketId, new System.IO.FileInfo(filepath).Name), stream);
}

If I choose the absolute filepah the uploads succeed. Unfortunately, in Backblaze, my local full path is stored - not just the filename as wished.

Download via Stream

It shouldn't be a requirement to download a file to a local path. There should be the capability of downloading the file to any writable stream.

The best (in my opion) way to handle this would be creating a download method that returns the response stream that correlates to the file. EG, if using a Http Client, returning response.Content.ReadAsStreamAsync() might be an option.

Transfer speeds reported by progress callback incorrect

I have a project now uploading files to B2 using both the simple .Files.UploadAsync and the chunked .Parts functions, and the progress callbacks I'm getting have some really weird-looking values in BytesPerSecond. They seem to be approximately 100 times too large. For instance, an upload of a ~25 MB file takes about 2 minutes here, and the entire time I'm getting progress updates telling me it's going at 15 MB/second, give or take. Am I doing something wrong??

Optimize performance by multi-threading large file uploads and downloads

Optimize performance by multi-threading uploads and downloads by launching multiple threads simultaneously on large file parts.

Ability to initialize the client without dependency injection

As the title says, it would be nice to have a constructor of BackblazeAgent that can be used manually instead of utilizing the dependency container.

Or at least provide an example to construct the client and its dependencies manually, in the README.

Error when connecting

I am getting errors when connecting with the BackblazeClient:
Newtonsoft.Json.JsonSerializationException: 'Error converting value "readBucketReplications" to type 'Bytewizer.Backblaze.Models.Capability'. Path 'allowed.capabilities[14]', line 22, position 30.'

From a quick look: I think there are some new capabilities in the response that are not present in the Capability enum.