dbetm/handwritten-flowchart-with-cnn

Recognition of handwritten flowcharts using convolutional neural networks to generate C source code and reconstructed digital flowchart.

handwritten-flowchart-with-cnn's Issues

Good job! but still have some bug

Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\Lenovo.conda\envs\rrett\lib\tkinter_init_.py", line 1892, in call
return self.func(*args)
File "F:/hhx/pythontest/handwritten1013/handler.py", line 461, in
background="green",command = lambda:self.__continue_process(text_nodes,shape_nodes,image_path,window))
File "F:/hhx/pythontest/handwritten1013/handler.py", line 427, in __continue_process
fg.generate_flowchart()
File "F:\hhx\pythontest\handwritten1013\flowchart_generator\flowchart_generator.py", line 36, in generate_flowchart
node = self.flow[key]
TypeError: string indices must be integers

are some faults in this bug?

urllib.error.HTTPError: HTTP Error 404: Not Found

Exception in Tkinter callback Traceback (most recent call last): File "D:\Programs\Anaconda\envs\flowchart\lib\tkinter\__init__.py", line 1705, in __call__ return self.func(*args) File "handler.py", line 417, in <lambda> ],window File "handler.py", line 533, in predict self.tc = TextClassifier() File "E:\Projects\flowchart\handwritten-flowchart-with-cnn\text_model\text_classifier.py", line 43, in __init__ self.recognizer = keras_ocr.recognition.Recognizer(alphabet = alphabet_detection) File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\recognition.py", line 329, in __init__ sha256=weights_dict['weights']['top']['sha256'])) File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\tools.py", line 453, in download_and_verify urllib.request.urlretrieve(url, filepath) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 248, in urlretrieve with contextlib.closing(urlopen(url, data)) as fp: File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 223, in urlopen return opener.open(url, data, timeout) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 532, in open response = meth(req, response) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 564, in error result = self._call_chain(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 504, in _call_chain result = func(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 756, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 532, in open response = meth(req, response) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 570, in error return self._call_chain(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 504, in _call_chain result = func(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found

You should replace this code in keras_ocr\recognition.py.

PRETRAINED_WEIGHTS = {
    'kurapan': {
        'alphabet': DEFAULT_ALPHABET,
        'build_params': DEFAULT_BUILD_PARAMS,
        'weights': {
            # 'notop': {
            #     'url': 'https://www.mediafire.com/file/n9yfn5wueu82rgf/crnn_kurapan_notop.h5/file',
            #     'filename': 'crnn_kurapan_notop.h5',
            #     'sha256': '027fd2cced3cbea0c4f5894bb8e9e85bac04f11daf96b8fdcf1e4ee95dcf51b9'
            # },
            # 'top': {
            #     'url': 'https://www.mediafire.com/file/pkj2p29b1f6fpil/crnn_kurapan.h5/file',
            #     'filename': 'crnn_kurapan.h5',
            #     'sha256': 'a7d8086ac8f5c3d6a0a828f7d6fbabcaf815415dd125c32533013f85603be46d'
            # }
            "notop": {
                "url": "https://github.com/faustomorales/keras-ocr/releases/download/v0.8.4/crnn_kurapan_notop.h5",
                "filename": "crnn_kurapan_notop.h5",
                "sha256": "027fd2cced3cbea0c4f5894bb8e9e85bac04f11daf96b8fdcf1e4ee95dcf51b9",
            },
            "top": {
                "url": "https://github.com/faustomorales/keras-ocr/releases/download/v0.8.4/crnn_kurapan.h5",
                "filename": "crnn_kurapan.h5",
                "sha256": "a7d8086ac8f5c3d6a0a828f7d6fbabcaf815415dd125c32533013f85603be46d",
            }
        }
    }
}

urllib.error.HTTPError: HTTP Error 404: Not Found

Exception in Tkinter callback Traceback (most recent call last): File "D:\Programs\Anaconda\envs\flowchart\lib\tkinter\__init__.py", line 1705, in __call__ return self.func(*args) File "handler.py", line 417, in <lambda> ],window File "handler.py", line 533, in predict self.tc = TextClassifier() File "E:\Projects\flowchart\handwritten-flowchart-with-cnn\text_model\text_classifier.py", line 45, in __init__ self.pipeline = keras_ocr.pipeline.Pipeline(recognizer = self.recognizer) File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\pipeline.py", line 19, in __init__ detector = detection.Detector() File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\detection.py", line 605, in __init__ sha256=weights_config['sha256']) File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\tools.py", line 453, in download_and_verify urllib.request.urlretrieve(url, filepath) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 248, in urlretrieve with contextlib.closing(urlopen(url, data)) as fp: File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 223, in urlopen return opener.open(url, data, timeout) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 532, in open response = meth(req, response) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 564, in error result = self._call_chain(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 504, in _call_chain result = func(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 756, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 532, in open response = meth(req, response) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 570, in error return self._call_chain(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 504, in _call_chain result = func(*args) File "D:\Programs\Anaconda\envs\flowchart\lib\urllib\request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found

replace the code in keras_ocr\detection.py

PRETRAINED_WEIGHTS = {
    ('clovaai_general', True): {
        # 'url': 'https://www.mediafire.com/file/qh2ullnnywi320s/craft_mlt_25k.pth/file',
        # 'filename': 'craft_mlt_25k.pth',
        # 'sha256': '4a5efbfb48b4081100544e75e1e2b57f8de3d84f213004b14b85fd4b3748db17'
        "url": "https://github.com/faustomorales/keras-ocr/releases/download/v0.8.4/craft_mlt_25k.pth",
        "filename": "craft_mlt_25k.pth",
        "sha256": "4a5efbfb48b4081100544e75e1e2b57f8de3d84f213004b14b85fd4b3748db17",
    },
    ('clovaai_general', False): {
        # 'url': 'https://www.mediafire.com/file/mepzf3sq7u7nve9/craft_mlt_25k.h5/file',
        # 'filename': 'craft_mlt_25k.h5',
        # 'sha256': '7283ce2ff05a0617e9740c316175ff3bacdd7215dbdf1a726890d5099431f899'
        "url": "https://github.com/faustomorales/keras-ocr/releases/download/v0.8.4/craft_mlt_25k.h5",
        "filename": "craft_mlt_25k.h5",
        "sha256": "7283ce2ff05a0617e9740c316175ff3bacdd7215dbdf1a726890d5099431f899",
    }
}

OCR Flowchart

Light refactor plan

• Improve English and instructions on README.md
• Delete # -*- coding: utf-8 -*- lines
• Order imports using PEP8
• Improve code style of handler.py, preprocessor.py, node.py,
• Rename codeGenerator.py to code_generator.py
• Upload dataset to Kaggle instead of the GDrive reference - I got an error trying to upload a new version.

Could not find a version that satisfies the requirement opt-einsum==0+untagged.56.g2664021.dirty

When i run pip install -r requirements.txt, i have this error :

ERROR: No matching distribution found for opt-einsum==0+untagged.56.g2664021.dirty```

Training

I am able to successfully train the model by running the shape_model.py file.
A new folder gets created inside the training_results folder.

But when I run the report.py file with the dataset path and model path, The mAP calculated always comes out to be 1.0.

I need some help in this part. Please help
Below is a reason I think the model is not getting trained. But I need help to fix it. Could you please suggest some way to correct this error?

Shape model: Improve data augmentation

In order to improve data augmentation for shape model v2, it's necessary implement data augmentation with rotations (-+1°) for all symbols (It does not affect the bboxes much so as not to update them), flips only works for shapes [start_end, process, decision] so it is not convenient to leave it.

Others ways to do data augmentation:

• Change brightness.
• Change contrast.
• Hue/saturation changes.
• Umbralization.
• Random gamma.

Good job！thank you for your great work !however a big bug in this project can you help me ,thankyou

Traceback (most recent call last):
File "C:\Users\Lenovo.conda\envs\rrett\lib\tkinter_init_.py", line 1892, in call
return self.func(*args)
File "F:/hhx/pythontest/handwritten1013/handler.py", line 461, in
background="green",command = lambda:self.__continue_process(text_nodes,shape_nodes,image_path,window))
File "F:/hhx/pythontest/handwritten1013/handler.py", line 420, in __continue_process
flow = graph.generate_graph()
File "F:\hhx\pythontest\handwritten1013\graph.py", line 227, in generate_graph
self.nodes = self.__collapse_nodes()
File "F:\hhx\pythontest\handwritten1013\graph.py", line 91, in __collapse_nodes
shape_nodes[min_node].set_text(text_nodes[i].get_text())
TypeError: list indices must be integers or slices, not NoneType

this bug how can i deal with this

can not download frcnn model in google driver, need permission

how ?

OSError: [WinError 126] 找不到指定的模块。

I suffer this problem.
Using TensorFlow backend. Traceback (most recent call last): File "handler.py", line 16, in <module> from text_model.text_classifier import TextClassifier File "E:\Projects\flowchart\handwritten-flowchart-with-cnn\text_model\text_classifier.py", line 6, in <module> import keras_ocr File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\__init__.py", line 1, in <module> from . import (detection, recognition, tools, data_generation, pipeline, evaluation, datasets) File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\detection.py", line 31, in <module> from . import tools File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\keras_ocr\tools.py", line 14, in <module> from shapely import geometry File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module> from .base import CAP_STYLE, JOIN_STYLE File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\shapely\geometry\base.py", line 18, in <module> from shapely.coords import CoordinateSequence File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\shapely\coords.py", line 8, in <module> from shapely.geos import lgeos File "D:\Programs\Anaconda\envs\flowchart\lib\site-packages\shapely\geos.py", line 145, in <module> _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll')) File "D:\Programs\Anaconda\envs\flowchart\lib\ctypes\__init__.py", line 348, in __init__ self._handle = _dlopen(self._name, mode) OSError: [WinError 126] 找不到指定的模块。

You can find the solution here
https://blog.csdn.net/s_l_k/article/details/115256397

Exception: Weight count mismatch for layer #20 (named time_distributed_1). Layer expects 2 weight(s). Received 0 saved weight(s)

Hello and thanks for this software!

I could not easily install it simply by the requirements file, but I had to struggle some hours, finally getting it to work (almost).

In the working configuration I am using python 3.9 (no chance with 3.7 and 3.8, or 3.10 and 3.11) in miniconda, ubuntu 20.04 in wsl2.
After installing python, I used the requirements file but I had then to:

1. make some installations manually (details available, of course, if someone is interested)
2. change in cnn.py:
   #from keras.engine.topology import get_source_inputs
   from keras.utils.layer_utils import get_source_inputs

Now the GUI starts.

I can:

• go to the "recognize flowchart" menu,
• select the "9" folder for the traning results,
• load an input image,
• click on "predict"

but I get:

Using Conda env: handwritten-flowchart-recog
Config loaded successful!!
Loading weights from model/training_results/9/flowchart_3b_model.hdf5
Exception: Weight count mismatch for layer #20 (named time_distributed_1). Layer expects 2 weight(s). Received 0 saved weight(s)
Couldn't load pretrained model weights!

What may the problem be? Too tired now to try anything myself...
Thanks!
Best
Giorgio

python version problem

I found the version of python which can successfully work is 3.6.
And before you use pip install -r requirement.txt, you should not install the protobuf and pygpu.
After you finish the pip, you can use conda install pygpu and conda install protobuf==3.11.4

This is a very helpful repo.

TypeError: create_target_machine() got an unexpected keyword argument 'jitdebug'

I get this error when running handler.py after following installation instructions.

File "/home/ew/anaconda3/envs/tt/lib/python3.9/site-packages/numba/targets/codegen.py", line 621, in _init
    tm = target.create_target_machine(**tm_options)
TypeError: create_target_machine() got an unexpected keyword argument 'jitdebug'