Giter Site home page Giter Site logo

litedb-async's Introduction

litedb-async

This library allows the use of asynchronous programming techniques with the LiteDb library. It is intended for Xamarin and WPF applications that require or would benefit a lightweight NoSQL database but also don't want to open and manage lots of threads or block the UI while database operations are occuring.

We manage the thread for you.

Available on nuget. https://www.nuget.org/packages/LiteDB.Async/

How to use

Installation

Include the nuget package in your project in the usual way

Install-Package LiteDB.Async

or

dotnet add package LiteDB.Async

Collections

Open a LiteDbAsync instance by calling the constructor in the standard way.

var db = new LiteDatabaseAsync("Filename=mydatabase.db;Connection=shared;Password=hunter2");

Collections are the equivalent of tables

var collection = _db.GetCollection<SimplePerson>();

This will give us an instance of the collection we can read and write.

We can upsert just like in the regular LiteDb

var collection = _db.GetCollection<SimplePerson>();
            var person = new SimplePerson()
            {
                Id = Guid.NewGuid(),
                FirstName = "John",
                LastName = "Smith"
            };

            var upsertResult = await collection.UpsertAsync(person);

Queries

When we want to read from the database we should use a query.

To read all SimplePerson's in the database

var collection = _db.GetCollection<SimplePerson>();
var listResult = await collection.Query().ToListAsync();

You can use Linq

var theSmiths = await collection.Query().Where(x => x.Lastname == "Smith").ToListAsync();

Transactions

Use the BeginTransaction method to start a transaction

using var asyncDb1 = new LiteDatabaseAsync(connectionString);
using var asyncDb2 = await asyncDb1.BeginTransactionAsync();

Operations run on asyncDb2 are now isolated from asyncDb1. They can also be rolled back, or if your program exits without call CommitAsync they are lost.

From version 5.0.16 of LiteDb we started to have problems with transactions on Shared connections. Therefore will be recommending Direct connections from LiteDb.Async version 0.1.6 goind forward.

To commit

await asyncDb2.CommitAsync();

Supported API

Almost all functions from LiteDb have an async replacement

From Collections

Query, CountAsync, LongCountAsync, ExistsAsync, MinAsync, MaxAsync, DeleteAsync, DeleteManyAsync, FindAsync, FindByIdAsync, FindOneAsync, FindAllAsync, Include, EnsureIndexAsync, InsertAsync, UpdateAsync, UpdateManyAsync, UpsertAsync

From Query

Include, Where, OrderBy, OrderByDescending, GroupBy, Select, Limit, Skip, Offset, ForUpdate, ToDocumentsAsync, ToEnumerableAsync, ToListAsync, ToArrayAsync, FirstAsync, FirstOrDefaultAsync, SingleAsync, SingleOrDefaultAsync, CountAsync, LongCountAsync, ExistsAsync

From Database

UtcDate, CheckpointSize, UserVersion, Timeout, Collation, LimitSize, BeginTransAsync, CommitAsync, RollbackAsync, PragmaAsync, GetCollectionNamesAsync, CollectionExistsAsync, DropCollectionAsync, RenameCollectionAsync, CheckpointAsync, RebuildAsync

From Storage

How does it work?

The constructor LiteDatabaseAsync opens and wraps a LiteDB instant. It also starts a background thread which all actions are performed in.

When a function that causes an evaluation is called it sends a message to the background thread, where the required action is performed by the LiteDb instance. The result is then return to the calling thread when the function in the background completes. If the function in the background thread causes an exception it is caught and a LiteAsyncException is raised. The original exception is preserved as the InnerException.

Each class or interface in the LiteDb library has an equivalent in the LiteDb.Async library. Each function that returns a task is named with the async suffix. You can then call LiteDb.ASync methods using similar syntax to calling async functions in EF Core.

How do transactions work?

A new database object is created and connects to the same source, the new database object also creates a new background worker thread for itself. It then calls the underlying LiteDb.BeginTrans function to use LiteDb's transaction functionality.

Building

To build the solution

dotnet build

To run the unit tests

dotnet test

To run a single specific unit test

dotnet test --filter DisplayName=LiteDB.Async.Test.SimpleDatabaseTest.TestCanUpsertAndGetList

To build for nuget dotnet pack --configuration Release

Don't forget to bump the version number in the litedbasync.csproj project file

It will build a nupkg file.

Next log in to nuget.org and upload the nupkg file which was just built.

Done the package will be available to the world within a few minutes.

Command line package build instructions

https://docs.microsoft.com/en-us/nuget/quickstart/create-and-publish-a-package-using-the-dotnet-cli

Web Upload instructions

https://docs.microsoft.com/en-us/nuget/nuget-org/publish-a-package

VS Code config

When checking out for the first time copy the .vscode/launch.json.default and .vscode/tasks.json.default to .vscode/launch.json and .vscode/tasks.json.

Example usage

The repo https://github.com/mlockett42/mvvm-cross-litedb-async gives an example of how to use this library in Xamarin Forms and MVVM.

litedb-async's People

Contributors

jensschadron avatar jmojiwat avatar mlockett42 avatar mookid8000 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

litedb-async's Issues

Methods are completed on background thread if ConfigureAwait(false) is used

We've observed that the background thread may leak to the consuming application. This is especially true in dotnet console applications – like an ordinary unittest project. UI-based platforms like WPF or Xamarin may behave differently but I haven't tested that (yet).

You can find a failing test in this commit.

The underlying flow is this:

  • application performs an async operation, like an InsertAsync, and awaits the result, using ConfigureAwait(false) (1)
  • litedb-async enqueues a lambda to perform an operation on the background thread
    • this includes a task completion source to return the result to the caller
  • the background thread performs the lambda
  • the lambda completes the task completion source (2)
  • the consuming application returns from its await (1) – without switching back to its initial synchronization context, as dictated by ConfigureAwait(false)
  • the consuming application disposes the litedb-async
    • the dispose method attempts to cancel the background thread
    • waiting for the background threads completion (by calling _backgroundThread.Join) runs into the provided timeout of 5 seconds. One can observe this with a breakpoint on the tooslow-variable introduced in this commit.
    • As a result the background thread has not finished yet
  • the consuming application moves on, maybe leaves the method and though the blocking async operation on the background thread
  • the lambda (2) completes
  • the background threads loop breaks and the thread completes

This essentially is a deadlock on the backgroundThread.Join being called from the background thread itself, which is only cancelled by the timeout.

Not using ConfigureAwait(false) does indeed fix the issue – but it's pretty common practice to use this configuration, at least in library projects.

So, what could one do?

  1. The consuming application could try to ensure to always dispose from another thread
  2. The consuming application could stop using ConfigureAwait(false)
  3. The completion of the task completion source could run on another thread, e.g. on the thread pool

Solution 1 would require proper handling in the application – thats not too complication but one has to know that. One can add a sentence to the documentation, but, meeh

Solution 2 would kind of go against common practice. It's probably not that easy to refrain from typical development patterns here, so, meeh.

Solution 3 would require a change on the libraries implementation for every operation completing its task completion source to run all waiting operations on a separate thread. Good luck this is a supported framework feature: One can simply pass TaskCreationOptions.RunContinuationsAsynchronously to the task completion sources constructor.

I would totally prefer solution 3 as it's the one with the least headaches and doesn't require special treatment from the application developer. But it's kind of a breaking change.
What do you thing? I'm willing to provide a pull request changing all the TaskCompletionSource<> calls appropriately.

Transactions rollback does not work, even explicit

I was not able to make the code work (console app on .net 8, LiteDb.Async 0.1.7)

var db = new LiteDatabaseAsync("litedbDemo.db");

try
{
	var col = db.GetCollection<MyEntity>();
	await db.BeginTransactionAsync();
	await col.UpsertAsync(new MyEntity());
	throw new Exception("something goes wrong");
}
catch (Exception)
{
	await db.RollbackAsync();
}

whether await db.RollbackAsync(); executes or not, changes made with UpsertAsync stay there in the db.

The moment I made these operations sync (original LiteDb implementation) everything starts working as expected: you do not have to run db.Rollback() even, no changes go to db till the moment you commit the transaction

Dispose takes a long time to return

Hi,

When using the LiteDatabaseAsync object in a using block, doing work then returning, it should run the dispose method. I noticed this takes a long time to complete. Anything going on in dispose ?

Thank you

Async Object Mapping

Is there any way this library can be extended to support asynchronous calls in a custom object mapper? if there is, that would be greatly appreciated if it could be added! I would be happy to contribute if needed.

[Question] Comparison with LiteDB and usage in Xamarin

Hey, finally there is async support for this amazing library! I will check it out in the nearest future. Meanwhile it would be quite cool and useful for everyone else to see the advantages of using your wrapper, not just regular LiteDB. I recommend u to put a sample out there in the Readme that demonstrates the async pros (hopefully no cons) of your wrapper - that way we, developers, can really see the advantages! Thanks!

Why transaction?

Application: Test.exe
CoreCLR Version: 8.0.424.16909
.NET Version: 8.0.4
Description: The process was terminated due to an unhandled exception.
Exception Info: Grpc.Core.RpcException: Status(StatusCode="Unknown", Detail="LiteDB encounter an error. Details in the inner exception.", DebugException="LiteDB.Async.LiteAsyncException: LiteDB encounter an error. Details in the inner exception.")
---> LiteDB.Async.LiteAsyncException: LiteDB encounter an error. Details in the inner exception.
---> LiteDB.LiteException: Maximum number of transactions reached
at LiteDB.Engine.TransactionMonitor.GetTransaction(Boolean create, Boolean queryOnly, Boolean& isNew)
at LiteDB.Engine.LiteEngine.Query(String collection, Query query)
at LiteDB.LiteQueryable1.ToDocuments()+MoveNext() at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext()
at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable1 source, Boolean& found) at LiteDB.LiteFileStream1.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Stream.CopyTo(Stream destination, Int32 bufferSize)
at LiteDB.LiteFileInfo1.SaveAs(String filename, Boolean overwritten) at LiteDB.LiteStorage1.Download(TFileId id, String filename, Boolean overwritten)
at LiteDB.Async.LiteStorageAsync1.<>c__DisplayClass16_0.<DownloadAsync>b__0() at LiteDB.Async.LiteDatabaseAsync.<>c__DisplayClass36_01.g__Function|0()
--- End of inner exception stack trace ---

builder.Services.AddSingleton<ILiteDatabaseAsync, LiteDatabaseAsync>(_ => new(builder.Configuration.GetConnectionString("Local")));

"Synchronous reads are not supported" when writing a stream

I'm using Blazor to upload files. This version of my upload method fails:

    public async Task UploadFail(IBrowserFile file)
    {
        using var db = await _dbFactory.GetDatabase();
        var stream = file.OpenReadStream(file.Size);
        await db.FileStorage.UploadAsync("test", file.Name, stream);
    }

The call stack is (trimmed):

LiteDB.Async.LiteAsyncException: LiteDb encounter an error. Details in the inner exception.
       ---> System.NotSupportedException: Synchronous reads are not supported.
         at Microsoft.AspNetCore.Components.Forms.BrowserFileStream.Read(Byte[] buffer, Int32 offset, Int32 count)
         at System.IO.Stream.CopyTo(Stream destination, Int32 bufferSize)
         at LiteDB.LiteStorage`1.Upload(TFileId id, String filename, Stream stream, BsonDocument metadata)
         at LiteDB.Async.LiteStorageAsync`1.<>c__DisplayClass11_0.<UploadAsync>b__0()
         at LiteDB.Async.LiteDatabaseAsync.<>c__DisplayClass36_0`1.<EnqueueAsync>g__Function|0()

If I change it to use an intermediary MemoryStream it works, but I have a feeling that this would buffer the entire file into memory a second time. Here's my workaround though:

var stream = new MemoryStream();
await file.OpenReadStream(file.Size).CopyToAsync(stream);
stream.Seek(0, SeekOrigin.Begin);
await db.FileStorage.UploadAsync(image.Id.ToString(), image.Filename, stream);

Improve Async Transactions

The current transaction infrastructure BeginTransAsync, CommitAsync and RollbackAsync is not very useful.

It is just a thin wrapper that starts a transaction on the underlying LiteDB instance but since every transaction runs against the underlying LiteDb instance they are all in the transaction until CommitAsync or RollbackASync is called. This will also cause strange behaviour if we await in between BeginAsync and CommitAsync/RollbackAsync which since we are targetting async program we almost certainly will.

Instead mark these functions as deprecated and add a new transaction handler. Investiage something that you give the function a callback which contains all the functions you wish to run, the framework then starts a new thread and runs all of the transactions in that thread.

[SUGGESTION] - Support a constructor that can receive an already instantiated LiteDatabase instance

Hi,

First off, kudos for putting this lib together!

While going through the constructor options, I realized for example there is not overload for connection string + BSON mapper, which happens to be there in the LiteDatabase object. Your lib currently exposes either one or the other, but not both together.

So, instead of creating yet another overload, I was thinking maybe we could simplify this and give the consumer the ability to tweak a LiteDatabase instance and feed your lib with it.

public LiteDatabaseAsync(LiteDatabase underlyingDatabase)
{
  _liteDB = underlyingDatabase ?? throw new ArgumentNullException(nameof(underlyingDatabase));
  _backgroundThread = new Thread(BackgroundLoop);
  _backgroundThread.Start();
}

The good thing about it is: Things like Pragmas and indexes would be available to the caller (I know you got async overloads for the indexes, but it may not be the case for everybody)...and you wouldn't need to create a brand new overload every single time LiteDatabase is changed.

The only con I can think of is the possibility of a external factor/leakage of the the reference in case the caller does not make a good use of it, but IMO this is all about good practices in the end.

System.ObjectDisposedException: The semaphore has been disposed

The problem is I use the hot reload in my project, and after the reloading of the project I re-create all instances of that I use, such as LiteDB, etc

simple Repro:

// On hot-reload load
protected override void Load()
{
    _liteDatabase = new LiteDatabaseAsync(databaseFilePath);
}
// On hot-reload unload
protected override void Unload()
{
    Instance = null;
    _liteDatabase?.Dispose();
}
// and at some runtime method if I'll try to use the _liteDatabase after the "reload" - I will get an exception about dispose

Expected behavior allows me to use the lite database when the new instance is created even if the previous instance was disposed

Running under load, LiteRepositoryAsync keeping file lock after Disposing

When running under load , after disposing of the ILiteRepositoryAsync, the UnderlyingDatabase is still holding onto a File lock. I think, due to not immediately disposing of the UnderlyingDatabase too.

This was my simple fix, around line 337 of LiteRepositoryAsync.cs

protected virtual void Dispose(bool disposing)
{
       if (disposing)
       {
            // BUGFIX: Ensure UnderlyingDatabase is immediately disposed of..._
            _db.UnderlyingDatabase.Dispose();
            _db.Dispose();
       }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.