Giter Site home page Giter Site logo

tearitco / sudoku-scanner Goto Github PK

View Code? Open in Web Editor NEW

This project forked from taylorjg/sudoku-scanner

0.0 0.0 0.0 24.99 MB

Use deep learning to recognise a Sudoku puzzle from a scanned image

Home Page: https://sudoku-scanner.herokuapp.com/

JavaScript 92.75% HTML 5.27% CSS 1.86% Shell 0.12%

sudoku-scanner's Introduction

Description

I want to create a web app to scan and solve a Sudoku puzzle. Apps already exist for Android and iOS but I haven't seen this done as a web app before. The hard part is scanning the puzzle which is what I am attempting to do in this repo. My plan is to use TensorFlow.js to:

  • Train a model to find the bounding box of a Sudoku puzzle
  • Calculate the grid squares from the bounding box
  • Train a model to distinguish between blank grid squares and digits
  • Train a model to recognise digits 1-9

UPDATE I have been unable to train a model to find the bounding box of a Sudoko puzzle. Therefore, I have reluctantly decided to follow the OpenCV route. I say reluctantly because OpenCV.js is massive. I borrowed heavily from the following links to get something working using OpenCV.js:

Training Failure

TODO: describe the problem that I encountered

Further Work

TODO: describe plans to have another go at training a model to find the bounding box

Instructions

  • Open https://sudoku-scanner.herokuapp.com/
  • Click 'Load' under 'Training - Blanks' to load a pre-trained model to distinguish between blank grid squares and digits
    • Or, click 'Train' to train a new model
  • Click 'Load' under 'Training - Digits' to load a pre-trained model to recognise digits 1-9
    • Or, click 'Train' to train a new model
  • Click 'Start Camera'
  • Click 'Capture Camera' to take a photo of a Sudoku puzzle from a newpaper or similar
  • Try to get the puzzle grid to:
    • roughly fill the guide lines
    • be in focus
    • not be wonky
  • Click 'Predict Capture' to scan and solve the puzzle

TODO

These are some of my major TODO items:

  • Instead of using a trial and error process of capturing images and trying to scan them, automatically try to scan an image from the video stream and stop the camera once a convincing match has been found
  • Take the results of this repo and feed them into sudoku-buster
    • The idea is for sudoku-buster to be a polished Sudoku scanning/solving web app whereas this repo is a laboratory experiment
  • Persevere with trying to train a model to find the bounding box of a Sudoku puzzle so that everything is done via TensorFlow.js and I no longer need to bring in OpenCV.js (which is massive)

Links

sudoku-scanner's People

Contributors

taylorjg avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.