Giter Site home page Giter Site logo

pengzhouhu / shadernn Goto Github PK

View Code? Open in Web Editor NEW

This project forked from inferenceengine/shadernn

0.0 0.0 0.0 101.78 MB

ShaderNN is a lightweight deep learning inference framework optimized for Convolutional Neural Networks on mobile platforms.

License: Other

Shell 0.46% C++ 67.53% Python 11.69% C 0.17% CMake 1.52% GLSL 18.28% Dockerfile 0.35%

shadernn's Introduction

ShaderNN logo

ShaderNN 2.0 preview release is available now!

ย 

What is ShaderNN?

ShaderNN is a lightweight deep learning inference framework optimized for Convolutional Neural Networks. It provides high-performance inference for deep learning applications in image and graphics process on mobile devices. ShaderNN workflow

Innovations of ShaderNN:

  • Use texture-based input/output, which provides an efficient, zero-copy integration with real-time graphics pipeline or image processing applications, thereby saving expensive data transfers & format conversion between CPU and GPU which is critical for real time application on mobile platforms. Inference Engine Integration

  • Leverage the fragment shader based on OpenGL backend in the neural network inference operators, which is advantageous when deploying parametrically small neural network modes.

  • Built on native OpenGL ES and Vulkan, which can be easily integrated with the graphics rendering pipeline to maximize the use of computing resources, suits for rendering, image/video and game AI applications.

  • Enable a hybrid implementation of compute and fragment shaders, with the ability to select layer-level shaders for performance optimization.

Key features of shaderNN:

  • High Performance
    • Utilize GPU Shader: Implement core operators using GPU Shader to leverage parallel computing capabilities for optimal performance.
    • Pre-built Static Computation Graph: Optimize with constant folding and operator fusion to accelerate forward operation speed.
  • Lightweight & Portability & Extensibility
    • No Third-Party Library Dependencies: Ensure independence from external libraries, reducing overhead and simplifying integration.
    • Mobile Platform Optimization: Optimize specifically for mobile platforms, enabling effortless portability, deployment, and upgrades.
    • Simple Input/Output Interface: Provide a user-friendly interface compatible with GPU processing for streamlined interactions.
  • Versatility
    • Framework & CNN network Compatibility: Support popular framework formats like TensorFlow, PyTorch, and ONNX. Support common classification, detection, segmentation, and enhancement networks.
    • User-Defined Operators: Enable easy implementation of new models by supporting user-defined operators.
    • Flexible backend configure: Select the running backend statically or dynamically according to the platform resources during model execution, dynamically adjusting kernel running parameters for minimal energy consumption at runtime.

Typical Application Scenerios:

  • ShaderNN is good at graphics and image processing pipelines, and here listed some typical scenerios, such as ray tracing denoise, deep learning super sampling, high dynamic range, super resolution and style transfer etc.
    ShaderNN Usecases

Architecture:

ShaderNN architecture

Getting Started:

Model Conversion:

  • Support conversion from TensorFlow , PyTorch and ONNX based models. Please refer to ModelConversion.md for details.

Model Zoo/Examples:

  • Provide image classification, object detection, image segmentation and image enchancement models for reference. Please refer to ModelZoo.md for details.

Operators:

  • Implement basic CNN operators by using fragment shader, computer shader or CPU. For a complete list of operators being supported, please refer to Operators.md for details.

Benchmark:

  • Benchmark models based in Model Zoo against TFLite framework. Please refer to Benchmark.md for details.

Style Transfer Demo:

  • Style Transfer example running on Android demo app using ShaderNN framework is shown below. The pretrained models are inferenced to showcase styles like Candy, Mosaic, Rain Princess and Udnie.

    Alt-text

Branching Policy:

  • For dev branches for your own use, please prefix it with "your_name/". For example, "bruce.lee/training_session_1"

License:

  • Apache License 2.0

Acknowledgement:

ShaderNN makes use of the following third party libraries:

ShaderNN makes use of models trained and converted from Tensorflow, PyTorch and ONNX, and uses Netron visualizer:

shadernn's People

Contributors

abpeak avatar computeengine avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.