asimmon / ephemeral-mongo Goto Github PK
View Code? Open in Web Editor NEWEphemeralMongo is a set of three NuGet packages wrapping the binaries of MongoDB 4, 5 and 6 built for .NET Standard 2.0.
License: Apache License 2.0
EphemeralMongo is a set of three NuGet packages wrapping the binaries of MongoDB 4, 5 and 6 built for .NET Standard 2.0.
License: Apache License 2.0
Hello,
This is actually a guide for those that are having issues on Fedora 38 and another Linux distro.
If you get a "Timeout" error when running you tests, maybe that are some missing dependencies needed by mongod to run.
After run dotnet restore command the mongob file will be download to your bin.
You can check the dependencies to run this file using following command:
ldd /path/to/mongod
In my case libcrypto.so.1.1 and libssl.so.1.1 were missing.
After a little search I could check that these libs are included in openssl1.1.x86_64 package.
To install openssl1.1:
sudo dnf install openssl1.1.x86_64
You can run the ldd command again and check if everything is fine.
If positive you can run your tests now!
Hope it helps!
I'm using the <PackageReference Include="EphemeralMongo6.runtime.ubuntu.22.04-x64" Version="1.1.3" Condition="$([MSBuild]::IsOSPlatform('Linux'))" />
as mentioned in #36 but my CI pipeline cant find the lib internal files. The pipeline runs on mcr.microsoft.com/dotnet/sdk:8.0
image.
Bellow its the first error message on my pipeline:
/usr/share/dotnet/sdk/8.0.300/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(284,5): warning NETSDK1206: Found version-specific or distribution-specific runtime identifier (s): ubuntu.22.04-x64. Affected libraries: EphemeralMongo6.runtime.ubuntu.22.04-x64. In .NET 8.0 and higher, assets for version-specific and distribution-specific runtime identifiers will not be found by default. See https://aka.ms/dotnet/rid-usage for details.
As far as I understand, .NET 8 won't load this library because it has a specific distribution on its name, which causes a second error message:
System.IO.FileNotFoundException : Could not find mongod in the following paths: '/github/workspace/tests/***.IntegrationTests/bin/Debug/net8.0/runtimes/linux-x64/native/mongodb/bin/ mongod'
Maybe suffixing it with "linux-x64" would make .NET 8 build include this lib correctly for linux, but im not sure. Nobody had this error like me?
Hi,
thank you for this awesome library. I was testing it locally and it was working fine. But I have found two issues:
My Fixture: https://github.com/notifo-io/notifo/blob/main/backend/tests/Notifo.Domain.Tests/Fixtures/MongoFixtureBase.cs
StandardOuputLogger => StandardOutputLogger
MongoRunner.Run()
method doesn't start the mongod
process on linux CentOS.
OS Version: Linux 3.10.0-1160.76.1.el7.x86_64
Error from std error:
mongod: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
Searching for this error brings me to this stack overflow question. Unfortunately none of the solutions posted here can be applied in my case because this is occurring on a CI server.
If a fix is not possible then the documentation should at least be updated with supported OSes and versions.
EDIT: Also I tried Mongo2Go but it has different issue.
MongoDB 7.0 just came out https://www.mongodb.com/blog/post/4-big-reasons-upgrade-mongodb-7-0. It requires the version 2.20 or higher for the driver first. They claimed massive query performance improvements.
One of my coworkers is encountering this exception when trying to use this library. It's probably something in their environment, but am flagging it just in case.
System.IO.IOException
Could not set executable bit for '{test project path}/bin/Debug/net6.0/runtimes/osx-x64/native/mongodb/bin/mongod'
at EphemeralMongo.Core.FileSystem.MakeFileExecutable(String path)
at EphemeralMongo.Core.MongoRunner.RunInternal()
at EphemeralMongo.Core.MongoRunner.Run(MongoRunnerOptions options)
at BaseMongoCollectionFixture..ctor() in {project path}/BaseMongoCollectionFixture.cs:line 49
This machine worked fine with the latest Mongo2Go. We're all using .NET 6. They upgraded to the latest version of the SDK just in case. This library is working fine for other developers and in GitHub Actions.
P.S. thanks for putting this out there! MongoDB 6 support in our tests helps us feel confident about code using newer aggregation features.
Hello ,
I have a dotnet project building on CI pipeline in linux OS. However the binaries folder does not exist in runtimes\linux-x64\native
folder which is causing the tests to fail with the below error
System.ComponentModel.Win32Exception : An error occurred trying to start process '/opt/****/workspace/Build_NetCore_MSV/source/app/test/app.Test/bin/Release/net6.0/runtimes/linux-x64/native/mongodb/bin/mongod' with working directory '/opt/****/workspace/Build_NetCore_MSV/source/app/test/app.Test/bin/Release/net6.0'. No such file or directory
I was able to see binaries in win , linux when I run locally. Please help
We have configured to use EphemeralMongo within our unit tests and now are seeing an accumulation of associated temp database files. I don't see an option to clean these up after the instance is terminated, so have I missed an option or are there any examples of automating this? Otherwise this would appear to be a problem as our server has been experiencing disk space issues and swapping to EphemeralMongo appears to have vastly increased them.
Hey,
we are currently testing it with Ubuntu 22. I have some issues that maybe https://jira.mongodb.org/browse/SERVER-62300 could fix. Would it be feasible to release an update of the EphemeralMongo6.runtime packages? From what i have seen v0.1.13 is on the following versions:
Regards and thanks for the work!
Thomas
I have a nested array called Documents in my collection.
This query works in production. While writing additional tests using ephemeral I start getting NotSupportedException
```
var projection = Builders.Projection.Expression(b => b.Documents.SingleOrDefault(i => i.Id == documentId));
return await _collection.Find(GetFilterByKey(botId)).Project(projection).FirstOrDefaultAsync();
Stack Trace:
MethodCallExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, MethodCallExpression expression)
ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
ExpressionToAggregationExpressionTranslator.TranslateLambdaBody(TranslationContext context, LambdaExpression lambdaExpression, IBsonSerializer parameterSerializer, Boolean asRoot)
LinqProviderAdapterV3.TranslateExpressionToProjectionInternal[TInput,TOutput](Expression`1 expression, IBsonSerializer`1 inputSerializer, AstSimplifier simplifier)
LinqProviderAdapterV3.TranslateExpressionToFindProjection[TSource,TProjection](Expression`1 expression, IBsonSerializer`1 sourceSerializer, IBsonSerializerRegistry serializerRegistry)
ExpressionProjectionDefinition`2.RenderForFind(IBsonSerializer`1 sourceSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
MongoCollectionImpl`1.CreateFindOperation[TProjection](FilterDefinition`1 filter, FindOptions`2 options)
MongoCollectionImpl`1.FindAsync[TProjection](IClientSessionHandle session, FilterDefinition`1 filter, FindOptions`2 options, CancellationToken cancellationToken)
<>c__DisplayClass48_0`1.<FindAsync>b__0(IClientSessionHandle session)
MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
IAsyncCursorSourceExtensions.FirstOrDefaultAsync[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken)
BotDataStore.GetBotDocument(Guid botId, Guid documentId) line 412
UpdateDocumentFromBotTestIntegration.Handler_UpdateDocument_ReturnsResponse() line 74
GenericAdapter`1.BlockUntilCompleted()
NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
AsyncToSyncAdapter.Await(Func`1 invoke)
TestMethodCommand.RunTestMethod(TestExecutionContext context)
TestMethodCommand.Execute(TestExecutionContext context)
<>c__DisplayClass1_0.<Execute>b__0()
BeforeAndAfterTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action)
I have some code that does a Limit as part of an Aggregate.
On 1.1.2 - I get
System.MissingMethodException
Method not found: 'MongoDB.Driver.IAggregateFluent`1<!0> MongoDB.Driver.IAggregateFluent`1.Limit(Int32)'.
0.1.3 -> Works
1.0.0 -> Works
1.1.0 -> Fails
1.1.2 -> Fails
Sample source code:
var customerEvents = await mongoDbContext.CustomerCollection.Aggregate()
.Match(filterLastThreeMonths)
.Sort(sortByCreated)
.Limit(500)
.ToListAsync();
I know there other ways I could have written this code, but this is a simplification from what I'm actually using.
It used to work on Mongo2Go, but I'm trying to migrate away from it.
Hi,
I'm facing an issue that might be related to the following behavior mentioned in the issue: "Windows Defender Firewall" want to block communication of testhost.exe
- EphemeralMongo also uses .NET Process class to start
mongod.exe
from your project output directory.
If I run the integration test end to end for the first time, it works perfectly. But if force the execution to end in the middle of the test (Shift + F5), the mongod.exe
is left running. Then if I make any change in the test code and try to run the test again, the IDE can't copy the mongod.exe
file to the output directory, so I can't run the tests.
This is happening because the mongod
process is still running.
The latest AWS codebuild images (that come packaged with Net6.0 SDK) are built on Ubuntu 22.04, which has deprecated libssl1.1
EphemeralMongo[0]
/codebuild/output/BroadcastServicesApi.UnitTests/bin/Release/net6.0/runtimes/linux-x64/native/mongodb/bin/mongod: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
We have been able to work around this limitation by manually installing libsssl1.1 before running our tests.
We haven't checked Ubuntu 22.04 based runners for other build systems.
The first time i launch my tests, the package works perfectly.
But then if i relaunch my tests i get:
/usr/lib/dotnet/sdk/6.0.116/Microsoft.Common.CurrentVersion.targets(4812,5): error MSB3027: Could not copy "/home/user/.nuget/packages/ephemeralmongo6.runtime.linux-x64/1.0.0/runtimes/linux-x64/native/mongodb/bin/mongod" to "bin/Debug/net6.0/runtimes/linux-x64/native/mongodb/bin/mongod". Exceeded retry count of 10. Failed. [/home/user/Desktop/Sources/Project/source/back/Project/Project.PublicBo.Api.Tests/Project.PublicBo.Api.Tests.csproj]
/usr/lib/dotnet/sdk/6.0.116/Microsoft.Common.CurrentVersion.targets(4812,5): error MSB3021: Unable to copy file "/home/user/.nuget/packages/ephemeralmongo6.runtime.linux-x64/1.0.0/runtimes/linux-x64/native/mongodb/bin/mongod" to "bin/Debug/net6.0/runtimes/linux-x64/native/mongodb/bin/mongod". Text file busy : '/home/user/Desktop/Sources/Project/source/back/Project/Project.PublicBo.Api.Tests/bin/Debug/net6.0/runtimes/linux-x64/native/mongodb/bin/mongod' [/home/user/Desktop/Sources/Project/source/back/Project/Project.PublicBo.Api.Tests/Project.PublicBo.Api.Tests.csproj]
When the service cannot start, by default there is no logs only a timeout exception with a generic message.
System.TimeoutException : MongoDB connection availability took longer than the specified timeout of 30 seconds. Consider increasing the value of 'ConnectionTimeout'.
It would be nice to capture the mongo process logs and display them in the exception message. Then, it could discard the logs once the service is ready. Also, it doesn't check if the process exited while waiting for it to start.
I think this improvement would simplify debugging.
Getting the following error when building on Ubuntu 22.04 in our Docker CI environment:
System.IO.FileNotFoundException : Could not find mongod in the following paths: '/builds/lunsjkollektivet/internal/backend/src/Tests/Tests.Common/bin/Debug/net6.0/runtimes/linux-x64/native/mongodb/bin/mongod'
I am using the package references mentioned here: #36 (comment)
The commands that i'm running when testing are as follows:
export NUGET_PACKAGES="$(realpath ./.nuget)/"
dotnet restore ./Tests/Tests.Common/Tests.Common.csproj --packages ./.nuget
dotnet test ./Tests/Tests.Common/Tests.Common.csproj --no-restore --collect:"XPlat Code Coverage" --logger "trx;LogFileName=test-result.trx;MethodFormat=Class;FailureBodyFormat=Verbose" --results-directory "./.testresults"
ARM64 will not be supported by default. The two main usecases for using EphemeralMongo are:
You can still manually download MongoDB binaries for ARM64, only use the EphemeralMongo.Core library, and specify the binaries directory path using options.BinaryDirectory
.
Adding ARM64 will increase the total size of downloaded packages. As of today, all OS/CPU architecture specific runtime packages are downloaded even though you only need one. That's because NuGet does not support conditional dependencies on both the target framework moniker (TFM) and a runtime identifier (RID). See related issue #2.
When this will be supported by NuGet, we'll be able to add new OS/CPU architecture specific runtime packages. Until then, ARM64 will not be supported by default.
After publishing the app, there are two copies of files from the 'ephemeralmongo5.runtime.win-x64' package in the output directory.
Environment:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="EphemeralMongo5" Version="1.1.3" />
</ItemGroup>
</Project>
> dotnet publish -o .\publish -r win-x64
There are two copies of attached files in the output directory.
Files structure:
Only one copy of the files in the output directory.
If publish the app for other OS, the error is happened.
dotnet publish -o publish -r linux-x64
C:\Program Files\dotnet\sdk\8.0.100\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ConflictResolution.targets(112,5): error NETSDK1152:
Found multiple publish output files with the same relative path:
C:\Users\MyUserName\.nuget\packages\ephemeralmongo5.runtime.linux-x64\1.1.3\runtimes\linux-x64\native\mongodb\community-server\THIRD-PARTY-NOTICES,
C:\Users\MyUserName\.nuget\packages\ephemeralmongo5.runtime.linux-x64\1.1.3\runtimes\linux-x64\native\mongodb\database-tools\THIRD-PARTY-NOTICES,
C:\Users\MyUserName\.nuget\packages\ephemeralmongo5.runtime.linux-x64\1.1.3\runtimes\linux-x64\native\mongodb\community-server\version.txt,
C:\Users\MyUserName\.nuget\packages\ephemeralmongo5.runtime.linux-x64\1.1.3\runtimes\linux-x64\native\mongodb\database-tools\version.txt.
OpenSSL 1.1 is going EOL in September and is being gradually removed from Linux systems:
mongod
bundled in EphemeralMongo simply doesn't start on such systems:
$ ~/.nuget/packages/ephemeralmongo6.runtime.linux-x64/1.1.0/runtimes/linux-x64/native/mongodb/bin/mongod
/home/marat/.nuget/packages/ephemeralmongo6.runtime.linux-x64/1.1.0/runtimes/linux-x64/native/mongodb/bin/mongod: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
Hi, we're just upgrading our .NET projects to 8.0 and while EphemeralMongo tests still work locally, in the CI pipeline which uses mcr.microsoft.com/dotnet/sdk:8.0 image, it fails with the following error:
Error Message:
#18 69.43 System.TimeoutException : MongoDB connection availability took longer than the specified timeout of 30 seconds. Consider increasing the value of 'ConnectionTimeout'.
#18 69.43 Stack Trace:
#18 69.43 at EphemeralMongo.MongodProcess.StartAndWaitForConnectionReadiness()
#18 69.43 at EphemeralMongo.MongodProcess.Start()
#18 69.43 at EphemeralMongo.MongoRunner.RunInternal()
#18 69.43 at EphemeralMongo.MongoRunner.Run(MongoRunnerOptions options)
Any thoughts on why that could be happening when it worked without issues with .NET 7.0?
EphemeralMongo use a dedicated runtime NuGet package for each operating system (linux-x64, osx-x64 and windows-x64).
However, when using EphemeralMongo4, EphemeralMongo5 or EphemeralMongo6, the three OS-specific runtime packages are downloaded and copied to the build output, which takes time and a lot of space. The ideal solution would to only download the relevant runtime package for the consuming operating system.
Unfortunately, there is no viable solution today to optimize the download process.
There are actually a lot of opened issues in the .NET GitHub repository about option #2, so it might be possible in a future version of .NET:
Using runtime.json is also not an option because it is deprecated, undocumented, and was created for internal .NET Core NuGet packages.
Until then, we'll stick with the RID-specific runtime package.
Hi,
i am testing the migration from Mongo2Go to EphemeralMongo. I am getting a "Windows Defender Firewall" block when using it. This does not seem to happen when using Mongo2Go. Also when i decide to ignore this message or cancel it; it does not have any negative effect on the testrun.
Any idea were this could come from?
Regards
Thomas
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.