faceonnx/faceonnx

Face recognition and analytics library based on deep neural networks and ONNX runtime

Face recognition and analytics library based on deep neural networks and ONNX runtime

A vision of what might lie across the universe...

FaceONNX

FaceONNX is a face recognition and analytics library based on ONNX runtime. It containts ready-made deep neural networks for face

• detection and landmarks extraction,
• gender and age classification,
• emotion and beauty classification,
• embeddings comparison and more.

Version

You can build FaceONNX from sources or install to your own project using nuget package manager.

Assembly	Specification	OS	Package	Algebra
FaceONNX	.NET Standard 2.0	Cross-platform	NuGet	UMapx
FaceONNX.Addons	.NET Standard 2.0	Cross-platform	NuGet	UMapx

Installation

C# interface

using FaceONNX;

To get started with FaceONNX, it is recommended to look at the folder with examples and experiments.

References

ONNX models

License

FaceONNX is released under the MIT license.

faceonnx's Issues

GPU Utilisation

Hello, first of all, thank you for your great work and sharing. I'm testing on the sample on the GPU. My graphics card is an RTX 3090 and the example uses the processor at 100% when running on the CPU. However, when running on the GPU, there is a maximum usage of 40%. What would you recommend I do to fully utilize the GPU resource?

Using SkiaSharp for imageing but not System.Drawing.Common for linux support

System.Drawing is not suit for linux, for libgdiplus has memory leak problem in linux, and have been removed since .net7.

Does not work in UWP (SoftwareBitmap)

It would be great if we could have it to work with SoftwareBitmap (or WriteableBitmap) in order to be able to use it on UWP apps too. This version compiles on UWP, but fires an "unsupported in this platform" error at runtime...

Thank you

Change facedetector model batchsize to dynamic N

Hi,
FaceONNX is an excellent lib, thanks for your works.
I'm trying to change facedetector models' batchsize to dynamic N for service inference usage, but got problem with the op "reshape", would you provide model in the orginal training framework?

Anti Face Spoofing on 2d Face Object (bitmap)

Thanks for the great works, is there any plan or anti face spoofing on 2d Face or if exist, any guide would be appreciated.

FaceONNX.Addons.Gpu runtime exception

FaceONNX.Gpu works great, but when I add the nuget package FaceONNX.Addons.Gpu, the following code throws a runtime exception:

var genderDetector = new FaceGenderClassifier()
// System.Resources.MissingManifestResourceException: 
// 'Could not find any resources appropriate for the specified culture or the neutral culture.  
// Make sure "FaceONNX.Gpu.Addons.Properties.Resources.resources" was correctly embedded or 
// linked into assembly "FaceONNX.Addons" at compile time, or that all the satellite 
// assemblies required are loadable and fully signed.'

I apologize if I'm missing something simple, but I can't figure it out.

My environment:
Win7-x64
Visual Studio 2019
WinForm .NET Framework 4.7.2

Source of face-recognition model?

We're using FaceONNX to detect faces in video images. It's working - but - we'd like to retrain the model(s) so that it recognises cartoon faces as well - we're comfortable with false positives.

Would you be able to point me at the source of the models, possibly their training sets and whatever scritps were used to (re)train them?

(I'm new to this - sorry if I'm asking the wrong questions)

ONNX model names

Can you please give information about the feature extraction model [recognition_resnet27.onnx] ? Is this model based on paper https://arxiv.org/pdf/1703.08388.pdf ?

Correct the register of labels

Can add Add Gaze model

Can add Add Gaze model?

Error when running in macOS

When I run the following code in my MacBook Pro (M1) this error occurs but, when I run the same code in Windows 10 OS the code works fine.

System.TypeInitializationException: "The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception." ---> System.DllNotFoundException: "Unable to load shared library 'onnxruntime' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable

When looking at the example codes, I noticed that a reference to windows is mentioned in .csproj

Does this mean that the library only works in Windows environments or am I forgetting to do some configuration to work on MacOS?

CPU vs GPU packages.

Hi, I've have some questions and suggestions regarding FaceOnnx package dependencies.

As I understand it, FaceOnnx uses CPU, and FaceOnnx.GPU uses Gpu (nVidia-Cuda?)

Onnx exposes more providers than just that, in fact, we use Onnx.DirectML as default.

Now, I am not asking to publish FaceOnnx for every provider, but the opposite.

My suggestion to you is that, instead of depending on "Microsoft.ML.OnnxRuntime" yo make the package depend on "Microsoft.ML.OnnxRuntime.Managed"

.Managed is just the abstract API surface and cannot run by itself because it's missing the provider, but the advantage is that you can make a package with some ONNX logic inside, and then the consumer of the package can append any provider.

The practical result would be that you only need to publish a single FaceOnnx package depending on Microsoft.ML.OnnxRuntime.Managed , and then the projects using it can choose & switch between providers.

GetEmbedding Error 【Ubuntu 18.0.4】

System.ArgumentException: Parameter is not valid.
at System.Drawing.SafeNativeMethods.Gdip.CheckStatus(Int32 status)
at System.Drawing.Bitmap.SetResolution(Single xDpi, Single yDpi)
at UMapx.Imaging.BitmapTransform.Rotate(Bitmap b, PointFloat point, Single angle)
at UMapx.Imaging.BitmapTransform.Rotate(Bitmap b, Single angle)
at FaceONNX.FaceLandmarksExtractor.Align(Bitmap image, Single angle)
at FaceONNX.FaceLandmarksExtractor.Align(Bitmap image, Rectangle rectangle, Single angle)
at KXY.AI.Services.FaceDetectionService.GetEmbedding(Bitmap image)

    public double Comparer(Stream leftImage, Stream rightImage)
    {
        var embeddings = new Embeddings();

        using var bitmap1 = new Bitmap(leftImage);
        var embedding1 = GetEmbedding(bitmap1);
        embeddings.Add(embedding1, "bitmap1");

        using var bitmap2 = new Bitmap(rightImage);
        var embedding2 = GetEmbedding(bitmap2);
        var proto = embeddings.FromSimilarity(embedding2);

        return proto.Item2;
    }

    private float[] GetEmbedding(Bitmap image)
    {
        var rectangles = faceDetector.Forward(image);
        var rectangle = rectangles.FirstOrDefault();

        if (!rectangle.IsEmpty)
        {
            // landmarks
            var points = _faceLandmarksExtractor.Forward(image, rectangle);
            var angle = points.GetRotationAngle();

            // alignment
            using var aligned = FaceLandmarksExtractor.Align(image, rectangle, angle);
            return _faceEmbedder.Forward(aligned);
        }

        return new float[512];
    }

Update UMapx NuGet packet

Update the UMapx NuGet package to the latest version.

Add scale method for the landmarks extractor class

It is not possible to download the files of the onnx models

Trying to clone de repository it gives a quota error for the files referenced with github LFS.

Could it be possible to upload the model files to some other storage such us Google Drive or some other?

Regards

GPU Support - how does it work?

I'd really like to be able to build in GPU support and have it use that if a GPU is available, but still work and fall back to CPU if no GPU is present on the system.

Is this possible?

Anti spoofing How to create simmilar images

The sample is working fine with the sample images, but can you please tell me how to generate new images with the same features as the sample ??

How well does faceONNX resist face spoofing?

Hello.
First things first, thanks for your work, looks awesome.
I was wondering if you have considered security in your implementation, or are planning to include some, to prevent face spoofing.

Fix ToBitmap method of FaceParsers

Do you have benchmark about the model?

Like LFW, megaface, mscelab? Thanks