Giter Site home page Giter Site logo

ai-models / aimm Goto Github PK

View Code? Open in Web Editor NEW
14.0 4.0 2.0 7.42 MB

AI Model Manager: Install a model once, and use everywhere ๐Ÿคฉ Users could save huge amounts of disk space ๐Ÿ’ฝ For developers, less code to manage of downloading and installing models ๐Ÿ‘ฉ๐Ÿฝโ€๐Ÿ’ป aimodels.json is used like requirements.txt / package.json! ๐Ÿ“

Home Page: https://aimodels.org

License: Apache License 2.0

Python 82.36% Batchfile 2.56% Shell 2.08% NSIS 13.00%
cli ai ai-models registry catalog ai-model artificial-intelligence neural-network

aimm's Introduction

Project on hold

Logo Aimm (AI Model Manager)

Tool for managing AI Models that works like pipTM or npmTM

Roadmap and overview at VisionInit Github

๐Ÿ“– Table of Contents

๐Ÿ“ Description

AI Model Manager, it's like pip or npm - but for AI models!

User Benefits

  • Save disk space, potentially many gigabytes
  • No more having to hunt for a model and making sure you rename it and put it in the correct folder
  • Less effort needed to audit what models are installed and where they came from

Developer Benefits

  • Less code to manage, no longer need to handle keeping model list and download logic
  • Pickle scaning and security mechanisms will be built into app

image

๐Ÿ’ป Quickstart

First we have to initialize the aimodels.json file in the local folder

aimm init

Search for a model you want to use. For example the BSRGAN model, a super resolution model that can upscale images.

aimm search BSRGAN

Then we install and add it to a local aimodels.json file to keep track of all the models you use in your project.

aimm add BSRGAN

๐Ÿ‘ฉโ€๐Ÿ’ป GUI

The GUI is developed but not ready for release yet.

image

This will allow modules such as stable horde to be integrated.

๐Ÿ“š Documentation

Documentation

๐Ÿ“œ License

Apache License 2.0

aimm's People

Contributors

dependabot[bot] avatar dleidert avatar neural-loop avatar rayrsn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

aimm's Issues

version, check-update, licenses commands

aimm --version
Show version value from aimm.py

aimm --check-update
read github/main/aimm.py and check version #
Aimm is up to date
There are updates available https://github.com/visioninit/aimm

aimm --licenses
cat LICENSE + append: pip-licenses -f plain -u --from all
aimm --licenses -verbose
cat LICENSE + append: pip-licenses -f plain-vertical -l --no-license-path -u --from all

I experimented with adding the LICENSE with appended pip-license detail into the packages, however, different OS might have slight differences in some cases. So if this would work for showing bundled software licenses that would be nice.

Fix code scanning alert - Clear-text storage of sensitive information

Tracking issue for:

Aware of this, it was intentional to keep this information stored in a local file.
Need to also add messaging to the end user who stores the information

  • Notify that sensitive information is being stored in aimodels-lock.json
  • If .gitignore exists, check if aimodels-lock.json is included, if not, prompt user (offer to add it)
  • If .gitignore doesn't exist, we should think of a place where we could repeat the message when aimm is used

picklescan / pickle throws error in ubuntu build, circular import error

Linux build failing due to circular import error from pickle

Run echo "***************************************"
  echo "***************************************"
  dist/aimm --version
  echo "***************************************"
  echo "DEBIAN_PACKAGE_VERSION=$(dist/aimm --version | awk -F: '{print $[2](https://github.com/visioninit/aimm/actions/runs/3627020816/jobs/6116521619#step:8:2)}' | tr -d ' ')+$(date +%Y%m%d%H%M)" >> $GITHUB_ENV
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/[3](https://github.com/visioninit/aimm/actions/runs/3627020816/jobs/6116521619#step:8:3).9.15/x6[4](https://github.com/visioninit/aimm/actions/runs/3627020816/jobs/6116521619#step:8:4)
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.9.1[5](https://github.com/visioninit/aimm/actions/runs/3627020816/jobs/6116521619#step:8:5)/x[6](https://github.com/visioninit/aimm/actions/runs/3627020816/jobs/6116521619#step:8:6)4/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.[9](https://github.com/visioninit/aimm/actions/runs/3627020816/jobs/6116521619#step:8:9).[15](https://github.com/visioninit/aimm/actions/runs/3627020816/jobs/6116521619#step:8:16)/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.15/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.15/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.15/x64/lib
***************************************
Traceback (most recent call last):
  File "aimm.py", line 2, in <module>
    from cli import aimmApp
  File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module
  File "cli/__init__.py", line 2, in <module>
    from cli import (base_funcs,
ImportError: cannot import name 'scan' from partially initialized module 'cli' (most likely due to a circular import) (/tmp/_MEIgr09wC/cli/__init__.pyc)
[2453] Failed to execute script 'aimm' due to unhandled exception!
Error: Process completed with exit code 1.

I've tried many things to try to fix this but haven't been able to sort it out

Update auth handling, auth storage will be 'project local' instead of user_config

  • I've added auth_required at the field level

image

  • the credential commands will save/delete/list from the aimodels-lock.json
  • credential will need auth_domain, user, and password
    • examples (let me know if another format works better)
  • Precheck scans through all files that will be downloaded by action (install, add) and does md5 value check and checks if auth_required is set.
  • If auth_required is set,
    • check aimodels-lock.json
    • interactive prompt(?) what if there are 2 domains that require auth
  • Add aimodels-lock.json in .gitignore

Tests

  • P: auth_required = True StableDiffusion [email protected] (signed terms)
  • F: auth_required = True StableDiffusion [email protected] (didn't sign terms)
  • F: auth_required = True StableDiffusion [email protected] (not registered)
  • F: auth_required = True StableDiffusion (no auth provided)
  • F: auth_required = False StableDiffusion-noauth (auth actually required but not checked)
  • P: auth_required = False spider-verse-diffusion

Add model path information into aimodels-lock.json

{
  "file_paths": {
    "trinart_stable_diffusion:2": {
      "path": " C:\\ProgramData\\aimm\\aimm\trinart_stable_dif\nfusion\\2",
      "files": [
        " C:\\ProgramData\\aimm\\aimm\trinart_stable_dif\nfusion\\2\file.ckpt",
        " C:\\ProgramData\\aimm\\aimm\trinart_stable_dif\nfusion\\2\\other.ckpt"
      ]
    }
  }
}

A model lookup function in the shared lib

I think the code complexity would be greatly reduced by having one single function which takes the modelname:version arguments and returns the needed data

model_lookup(model,version):
check lowercase to lowercase
returns installed modelname & model version internal values

Then replace all of the different occurrences where there is custom checking against these arguments

terminalizer

Rayrsn, would you be interested in taking the task to create maybe a script that would generate the terminalizer going through the different commands of the app that I could put in the documentation?

I was thinking something that I would break into different steps / paragraphs

aimm init
aimm search coqui-language

aimm add modelname
cat aimodels.json
cat aimodels-lock.json

aimm install othermodelname
aimm list

aimm cat aimodels.json (See, it's not there)
aimm add othermodelname (And it's added from the local storage)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.