cbasavaraj/deepfloorplan

TF2 Deep FloorPlan Recognition using a Multi-task Network with Room-boundary-Guided Attention. Enable tensorboard, quantization, flask, tflite, docker, github actions and google colab.

TF2DeepFloorplan

This repo contains a basic procedure to train and deploy the DNN model suggested by the paper 'Deep Floor Plan Recognition using a Multi-task Network with Room-boundary-Guided Attention'. It rewrites the original codes from zlzeng/DeepFloorplan into newer versions of Tensorflow and Python.
Network Architectures from the paper,

Requirements

Install the packages stated in requirements.txt, including matplotlib,numpy,opencv-python,pdbpp, tensorflow-gpu and tensorboard.
The code has been tested under the environment of Python 3.7.4 with tensorflow-gpu==2.3.0, cudnn==7.6.5 and cuda10.1_0. Used Nvidia RTX2080-Ti eGPU, 60 epochs take approximately 1 hour to complete.

How to run?

1. Install packages via pip and requirements.txt.

python -m venv venv
source venv/bin/activate
pip install --upgrade pip setuptools wheel
pip install -r requirements.txt

2. According to the original repo, please download r3d dataset and transform it to tfrecords r3d.tfrecords. Friendly reminder: there is another dataset r2v used to train their original repo's model, I did not use it here cos of limited access. Please see the link here zlzeng/DeepFloorplan#17.
3. Run the train.py file to initiate the training, model checkpoint is stored as log/store/G and weight is in model/store,

python dfp/train.py [--batchsize 2][--lr 1e-4][--epochs 1000]
[--logdir 'log/store'][--modeldir 'model/store']
[--saveTensorInterval 10][--saveModelInterval 20]

• for example,

python dfp/train.py --batchsize=8 --lr=1e-4 --epochs=60
--logdir=log/store --modeldir=model/store

4. Run Tensorboard to view the progress of loss and images via,

tensorboard --logdir=log/store

5. Convert model to tflite via convert2tflite.py.

python dfp/convert2tflite.py [--modeldir model/store]
[--tflitedir model/store/model.tflite]
[--quantize]

6. Download and unzip model from google drive,

gdown https://drive.google.com/uc?id=1czUSFvk6Z49H-zRikTc67g2HUUz4imON # log files 112.5mb
unzip log.zip
gdown https://drive.google.com/uc?id=1tuqUPbiZnuubPFHMQqCo1_kFNKq4hU8i # pb files 107.3mb
unzip model.zip
gdown https://drive.google.com/uc?id=1B-Fw-zgufEqiLm00ec2WCMUo5E6RY2eO # tfilte file 37.1mb
unzip tflite.zip

7. Deploy the model via deploy.py, please be aware that load method parameter should match with weight input.

python dfp/deploy.py [--image 'path/to/image']
[--postprocess][--colorize][--save 'path/to/output_image']
[--loadmethod 'log'/'pb'/'tflite']
[--weight 'log/store/G'/'model/store'/'model/store/model.tflite']

• for example,

python dfp/deploy.py --image floorplan.jpg --weight log/store/G
--postprocess --colorize --save output.jpg --loadmethod log

Docker for API

1. Build and run docker container. (Please train your weight, google drive does not work currently due to its update.)

docker build -t tf_docker -f Dockerfile .
docker run -d -p 1111:1111 tf_docker:latest
docker run --gpus all -d -p 1111:1111 tf_docker:latest

2. Call the api for output.

curl -H "Content-Type: application/json" --request POST  \
  -d '{"uri":"https://cdn.cnn.com/cnnnext/dam/assets/200212132008-04-london-rental-market-intl-exlarge-169.jpg","colorize":1,"postprocess":0, "output":"/tmp"}' \
  http://0.0.0.0:1111/process --output /tmp/tmp.jpg


curl --request POST -F "file=@resources/30939153.jpg;type=image/jpeg" \
  -F "postprocess=0" -F "colorize=0" -F "output=/tmp" http://0.0.0.0:1111/process --output out.jpg

3. If you run app.py without docker, the second curl for file upload will not work.

Google Colab

1. Click on and authorize access.
2. Run the first code cell for installation.
3. Go to Runtime Tab, click on Restart runtime. This ensures the packages installed are enabled.
4. Run the rest of the notebook.

Deep Floorplan package

1. Install as a package.

pip install -e .
python setup.py test
coverage run ./setup.py test

2. Import as a package.

import dfp
from dfp import net, data
model = net.deepfloorplanModel()

3. Uninstall package. pip uninstall Deep_floorplan

How to Contribute?

1. Git clone this repo.
2. Install required packages and pre-commit-hooks.

pip install -r requirements.txt
pip install pre-commit
pre-commit install
pre-commit run
pre-commit run --all-files
# pre-commit uninstall/ pip uninstall pre-commit

3. Create issues. Maintainer will decide if it requires branch. If so,

git fetch origin
git checkout xx-features

4. Stage your files, Commit and Push to branch.
5. After pull and merge requests, the issue is solved and the branch is deleted. You can,

git checkout main
git pull
git remote prune origin
git branch -d xx-features

Results

• From train.py and tensorboard.

Compare Ground Truth (top) against Outputs (bottom)	Total Loss
Boundary Loss	Room Loss

• From deploy.py and utils/legend.py.

Input	Legend	Output
--colorize	--postprocess	--colorize --postprocess