jungin500/nn-meter-backend-ssh

SSH backend plugin of Microsoft's nn-Meter for Hardware-aware NAS - effective on Raspberry Pi, Jetson Nano and other SBCs

nn-Meter SSH-TFLite backend

Description

• nn-Meter is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices.
• While building per-device CNN kernel latency predictor, nn-Meter requires Backend which is responsible of running latency benchmarking on specific device
• Android-specific latency benchmark backend tflite_cpu was already present on official nn-Meter, but formal SSH backend has not been implemented.

Features

• TFLite backend which will run over remote SSH
• Minizes connection overhead by reusing single SSH connection on latency benchmark pipeline
• Supports

Installation

1. Install nni==2.7.1, jungin500/nn-Meter (Use my repo with fixes - Highly recommended)
2. Clone this repository to any directory and remember as $DIRECTORY
3. Run pip3 install -r requirements.txt (Edit file before run if you want to use your speicific tensorflow version)
4. Edit ssh_tflite_cpu.yml and replace <CURRENT_DIRECTORY_HERE> to $DIRECTORY
5. Run nn-meter register --predictor ssh_tflite_cpu.yml
6. Create workspace using ssh_tflite_cpu backend as described on this guide
7. Open newly created configs/backend_config.yaml to match your own environment
   • Password and pubkey-based auth supported (Be aware to check permission of private key!)
8. You can use this backend within latency predictor: Official nn-meter guide