jessebenson/service-fabric-queryable

Enable query support for your stateful services in Service Fabric via the OData protocol.

service-fabric-queryable's Issues

Unit testing Service Fabric Queryable

Hi Team,

We are trying to create unit test cases for Queryable Reliable Indexed Dictionary, but we are facing below error message:

_InnerException {System.InvalidProgramException: No query over the data source was specified.
at ServiceFabric.Extensions.Services.Queryable.LINQ.ReliableIndexedDictionaryQueryContext.Execute[TKey,TValue](Expression expression, IReliableStateManager stateManager, IReliableIndexedDictionary`2 dictionary, Boolean IsEnumerable)} System.Exception {System.InvalidProgramException}

This issues is observed only on unit tests, which are querying on indexed columns. We are able to run application and query all endpoints but such unit test cases are failing.

Can you please help us out on this issue?

Not supporting for Actor data

I am having a Service Fabric application with some Actor Services in it.
Can I use the same mechanism for querying all the Actor States?
Or do we have any mechanism for querying the actors?

Thanks
Prabhuji

Access records locked by another transaction in Service Fabric reliable indexed dictionary

In our application, we are using service fabric indexed dictionaries to store data. We are also using Service Fabric Queryable package to add querying capabilities on our collections.

In our case, we are having a transaction in which we do multiple updates and commit all of them at once. We have also created indexes on some properties for faster retrieval of data. Due to this indexing a record is internally created in a index dictionary with the indexed property as the key and the object as the value. This is also an AddOrUpdate operation and is included in the same transaction. And the transaction which is used to perform these update operation acquires an exclusive lock on all the newly created records.

Using the same transaction, when we try to perform a query on the queryable indexed dictionary using the property which has been indexed, we are getting a Timeout Exception.

We have gone through the source code in the Queryable extensions package and found that the query which we run on the queryable dictionary creates a new transaction to fetch the filtered records. But this transaction is unable to access the records as the previous transaction in our code has acquired an Exclusive lock on the record.

Below is some sample code which describes our scenario.

public async Task MyMethod() 
{
    using (var tx = this.StateManager.CreateTransaction())
    {
         foreach(var record in records)
         {
              // Here we perform an update operation 
              // The transaction takes an exclusive lock on the created record to perform this operation and releases only after it is committed
              var result = await indexedDictionary.AddOrUpdateAsync(tx, myKey, myValue, (key, value) => myValue);

              // Creating a Queryable dictionary
              var queryableDictionary = new QueryableReliableIndexedDictionary(indexedDictionary, this.StateManager);

              // Running a query on the queryableDictionary 
              var filteredRecords = queryableDictionary.Where(x => x.indexedProperty== propertyValue).ToList();

              // The code to filter the data exists in the Service Fabric Queryable package where a new transaction is being created.
              // Once we query with where condition on the queryable dictionary, the Execute method gets called in the package where the actual filtering takes place.
              // That code looks similar to the code snippet given below.
         }

         await tx.CommitAsync();

    }        
}

Below is the code snippet from the Service Fabric Queryable package.

internal static async Task<object> Execute<TKey, TValue>(Expression expression)
{


    // This transaction is being created which tries to access the records actual dictionary
    using (var tx = stateManager.CreateTransaction())
    {
         // This statement tries to access the dictionary.. but it can't as the previous transaction has acquired an exclusive lock on the record it is trying to access
         // It waits for 4 seconds and throws a Timeout Exception
         IEnumerable<KeyValuePair<TKey, TValue>> pairs = await indexedDictionary.GetAllAsync(tx, keys, TimeSpan.FromSeconds(4), new CancellationToken()).AsEnumerable();
         values = new KeyValueToValueEnumerable<TKey, TValue>(pairs);
         await tx.CommitAsync();
    }
}

Is there any work around for us to be able to query the queryable dictionary without having to commit the previous transaction ?

Issues in Service Fabric Queryable extensions

am working on integrating the Queryable extensions of ReliableIndexDictionary and facing few challenges on querying the Reliable Dictionary. Below are the exception details with respect to the query I am performing.

Reliable Indexed Dictionary:

var dictionary = await this.StateManager.GetOrAddIndexedAsync<Guid, File>(
                       "FileDictionaryName",                     
                      new FilterableIndex<Guid, File, string>(nameof(File.FriendlyFileName), (k, v) => v.fileName),                   
                        new FilterableIndex<Guid, File, long>(nameof(File.FileSizeInBytes), (k, v) => v.FileSizeInBytes.GetValueOrDefault()),
                       new SearchableIndex<Guid, File>(nameof(File.FileName), (k, v) => v.fileName)).ConfigureAwait(false);
Queryable Indexed Dictionary:

var queryableDictionary = new QueryableReliableIndexedDictionary<Guid, File, File >(dictionary, this.StateManager);

NOTE: We are unable to get all the records in the collection.

Can anyone please help out on this?

I have posted a question in stack over flow below is the link and issue details,
https://stackoverflow.com/questions/53482969/service-fabric-queryable-extensions-facing-issues-while-performing-linq-queri

ServiceFabric.Extensions.Services.Queryable Not supporting for .net core 2.1

Microsoft.AspNet.WebAPI.OData package not able to find in .net core 2.1 and not able to use this package in .net core statefull service fabric service.

Can anyone help me out in this?

Build warnings : use of .net framework library in .net coreclr project

Hi

I am receiving following warnings when I am building : ServiceFabric.Extensions.Services.Queryable

Build Warnings :

1>------ Build started: Project: ServiceFabric.Extensions.Services.Queryable, Configuration: Debug x64 ------
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1608: Detected package version outside of dependency constraint: Microsoft.ServiceFabric.Services 3.1.283 requires Microsoft.ServiceFabric.Data (= 3.1.283) but version Microsoft.ServiceFabric.Data 3.2.162 was resolved.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1608: Detected package version outside of dependency constraint: Microsoft.ServiceFabric.Diagnostics.Internal 3.1.283 requires Microsoft.ServiceFabric (= 6.2.283) but version Microsoft.ServiceFabric 6.3.162 was resolved.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'Microsoft.AspNet.WebApi.Client 5.2.2' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'Microsoft.AspNet.WebApi.Core 5.2.2' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'Microsoft.AspNet.WebApi.OData 5.7.0' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'System.Linq.Dynamic 1.0.7' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>ServiceFabric.Extensions.Services.Queryable -> E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\bin\x64\Debug\netstandard2.0\ServiceFabric.Extensions.Services.Queryable.dll
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1608: Detected package version outside of dependency constraint: Microsoft.ServiceFabric.Services 3.1.283 requires Microsoft.ServiceFabric.Data (= 3.1.283) but version Microsoft.ServiceFabric.Data 3.2.162 was resolved.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1608: Detected package version outside of dependency constraint: Microsoft.ServiceFabric.Diagnostics.Internal 3.1.283 requires Microsoft.ServiceFabric (= 6.2.283) but version Microsoft.ServiceFabric 6.3.162 was resolved.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'Microsoft.AspNet.WebApi.Client 5.2.2' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'Microsoft.AspNet.WebApi.Core 5.2.2' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'Microsoft.AspNet.WebApi.OData 5.7.0' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>E:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\ServiceFabric.Extensions.Services.Queryable.csproj : warning NU1701: Package 'System.Linq.Dynamic 1.0.7' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.
1>Done building project "ServiceFabric.Extensions.Services.Queryable.csproj".
1>Successfully created package 'e:\service-fabric-queryable\src\ServiceFabric.Extensions.Services.Queryable\bin\x64\Debug\ServiceFabric.Extensions.Services.Queryable.3.1.300.nupkg'.
1>Done building project "ServiceFabric.Extensions.Services.Queryable.csproj".
========== Build: 1 succeeded, 0 failed, 1 up-to-date, 0 skipped ==========

Are these build warnings known ?

.NET Core Support

I've had a stab at upgrading the library to work with .NET Core services, using the latest libs (AspNet.OData / UriParser etc) which mostly works.

Metadata is OK, querying returns data, $top works. But $filter etc I cannot get to work and am at a crossroads.

If a fork/repo were provided, would this be something you could advise on, or any suggestions on what would be next steps to get this working? From the look of the codebase, this appears to be very similar in working to what Cosmos must do behind the scenes.

Missing licence

Under what licence have you released the package?