Giter Site home page Giter Site logo

mmehdimousavi / aiplayground Goto Github PK

View Code? Open in Web Editor NEW
39.0 3.0 4.0 104.46 MB

A virtual environment that allows changing isolated features in the image

Python 25.62% Jupyter Notebook 74.38%
machine-learning deep-learning synthetic-data computer-vision virtualenvironment virtual-environments depth-estimation surface-normals-estimation diode-dataset

aiplayground's Introduction

AIPlayground: An Unreal Engine Based Data Ablation tool for Deep Learning

drawing

Image, GroundTruth, Prediction

Citation:

Paper

PWC

What Is Data Ablation?

Data Ablation is the study of data's effects on the performance of your network. Does your network get confused when it sees reflection in images? the only way to be sure is to test it. AIP enables you to test the effects of various aspects of data on a ML model, Image processing Algorithm or an AI Agent. Testing things like this is not feasible for natural scenes or crowd-sourced data. AIP will allow you to swap in different aspects of data without changing the other aspects, ie. isolating that exact feature that you are looking for. For example, roughness of a metal surface, reflectiveness of a tile, or the resolution of textures. With various settings for you to customize like lighting, fidelity (reflections, antialiasing, shadows, render scaling, etc), you will be able to generate vast datasets of otherwise identical images but with subtle differences to isolate and study the effects of different settings on your Models.

Watch demo video:
View Demo

AIP:

A Virtual Environment for Various Machine Learning and/or a Dataset Generation tool for Image processing.

AIP Can be used to Generate Entire Datasets for various Machine Learning purposes. It supports Depth Estimation, Surface Normal Estimation and Pixelwise Segmentation for up to 256 Classes out of the box. AIP Features Keyboard macros and an interaction Module named "Probe" that can remember and reproduce the same Image over different circumstances. AIP's code can be used to make new environments while keeping the features for other purposes.

AIP uses customizable Post-processing for generating ground-truth annotations, making it flexible for use in characters and cameras, as well as global unbound post processing volumes.

drawing

How to use:

- Using the App:

Step 1: Download Probe
Step 2: Set up Probe in Python
Step 3: Download and Open AIPlayground.exe 
Step 4: Run Probe.

Probe Will now start gathering Images based on your setup specifications. Memory module will be saved in a folder called "Probe". If you need to reproduce the images, you need to make sure to save that file.

- Using the Source Code:

Step 1: Download AIP Core
Step 2: Import AIP Core Folders into any Unreal Engine Project
Step 3: Set up your Game Mode and Default Pawn to AIP
Step 4: Import the Input settings file provided in the AIP Core
Step 5: Run the Project
Step 6: Export your project or Run it in an independent window. Probe will recognize your Window handle and start sending commands to it. 

Probe Will now start gathering Images based on your setup specifications. Memory module will be saved in a folder called "Probe". If you need to reproduce the images, you need to make sure to save that file.

AIP Character Hotkeys:

- Segmentations Glass:

Press G for a Global Class Based Pixelwise labels. Press 0-9 and Ctrl+1 to Ctrl+4 to Access Binary Class-based Pixelwise Labels.

- Surface Normals Glass:

Press R to view the Pixelwise Surface normal in 6 Axis represented by 6 Unique Colors.

- Depth Glass:

Press T to view the prespective Depth (Distance of each pixel to the camera)

- Other Glasses:

E for Greyscale Colors H, J, K, L for Very High, High, Medium, Low Graphics B for Unlit Pass Y, U, I, O for switching between maps

Map:

drawing

aiplayground's People

Contributors

mmehdimousavi avatar sraashis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

aiplayground's Issues

Executing source code

Hi, thanks for sharing this amazing work!

I'm trying to open your code on UE, but couldn't figure out how to access AIP's assets (such as rooms).
Could you please elaborate on how to run the code in UE? Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.