lakshyakhatri/bookshelf-reader-api

A browsable REST API for recognizing book spines in an image.

Bookshelf-Reader-API

A browsable REST API built using Django REST Framework for recognizing book spines in an image.

Uploaded Image	Result

Installation

• To run this project locally, clone or download this repository.
• Install requirements using:

  pip install -r requirements.txt
  

• Then run the migrations using:

  python3 manage.py makemigrations  
  python3 manage.py migrate
  

• Run the application:

  python3 manage.py runserver
  

Usage

Add these URLs after your landing URL

Function	url	Return
Upload Bookshelf Image	/api/create-bookshelf/	ID for referring the uploaded image (Inside the Response Header)
Spine Line Drawn Image	/api/bookshelf/<bookshelf-id>/	Spine line drawn image
Cropped Spines	/api/spines/<bookshelf-id>/	URLS of the cropped spine images

Further Implementation

This project contains scrappers to scrap the information of all the books recognized in the spine image. Recognized spine can be sent for text recognition and then the recieved text can be uploaded to below URL's for scrapping the book's information.

NOTE 1: It's okay if the recognized text is not accurate. The scrapper will automatically find the correct book.
NOTE 2: The uploaded text is expected to be the book title [and author name].

Function	url	Return
Upload book title text for scrapping information	/api/add-book/	ID of the created Book object (Inside the Response Header)
Get Book Information	/api/book/<book-id>/	Scrapped book information.

Client Side Application

If you want to see how this REST API can be used at client side then checkout Bookshelf Reader Android Application

Liscence

MIT License

Copyright (c) 2020 Lakshya Khatri

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

bookshelf-reader-api's Issues

no such table: detect_spines_bookshelf

I got below errors while post image

Request Method:	POST
http://127.0.0.1:8000/api/create-bookshelf/
3.1.5
OperationalError
no such table: detect_spines_bookshelf

Improve book/spine recognition?

I've been playing around with the code to try to improve recognizing books.

These are issues I've noticed:

1. Sometimes two or more books are grouped. This seems to be due to various reasons such as the congratulation of the edge detection, lighting issues, and the fact that gray scale is used.
2. A lot of times there are slivers captured as books when they are clearly not. I think some could be removed simply by checking the width of the sliver and if it is a few pixels or so it could be removed.
3. Sometimes a spine is split in two. I've noticed this when the spine has a horizontal line in it or if there is a strong specular highlight. Basically anything that will produce a horizontal line.

I'm wondering if there can be a much more improved way to get the books and maybe even books in general such as using AI. If a NN was setup then it would just be a matter of training it and this might be auto generated by combining spines images in to "shelves".

I'm not sure how we can improve the current algorithm to make it more robust. I get a lot of false hits and strange results that make it a bit unusable. Not saying it can't get the data but it's too inaccurate to have it fully automatable.

There are some AI models that already have the ability to detect things like books so I wonder if it could be used as plug and play where one gets the books then clips them out of the image and feeds them in to the OCR to get info about them.

E.g., https://www.freecodecamp.org/news/how-to-detect-objects-in-images-using-yolov8/

https://cocodataset.org/#explore

You can see it is pretty good at recognizing books.

Missing migrations for detect_spines app?

Hi, I apologize in advance if I'm doing something silly but I can't seem to run (or find) the migrations for this app.

Running makemigrate and migrate only installs migrations for the admin, auth, contenttypes, sessions packages but none related to the detect_spines app. All of the api calls fail because the database tables don't exist.

Internal Server Error: /api/create-bookshelf/
Traceback (most recent call last):
File "/Users/natitessema/workspace/Bookshelf-Reader-API-master/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/Users/natitessema/workspace/Bookshelf-Reader-API-master/venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 477, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: detect_spines_bookshelf

Is there a missing migrations/ directory for the detect_spines app or am I doing something wrong? Thanks in advance and thanks for making this project @LakshyaKhatri 🙂

no such table: detect_spines_bookshelf

I got below errors while post image

Request Method:	POST
http://127.0.0.1:8000/api/create-bookshelf/
3.1.5
OperationalError
no such table: detect_spines_bookshelf

Add ability to scan barcodes and covers and export/track

It shouldn't be too hard to extend it to be able to scan covers and bar codes and then keep a list.

There are some book cataloging programs out there but most seem to have some issues with some books(older at least). It would be nice if your app could scan the covers and/or barcodes to try to get as much info as possible for older books and then store that info such as the images and scraped info in to a database/list which can then potentially be imported in to another app.

E.g., if one scans a cover then there should be just "one spine" so to speak. I would think that the code would essentially work almost unchanged. The only other thing may be to recognize the barcodes in the image which probably would not be that difficult.

So I would imagine very little actually has to be done to get that functionality all working which would allow making the app more useful for cataloging. The idea is to make it fast, accurate, and informative.

The problem I'm having now is every book catalog app I've tried is

1. the barcode scanner will not correctly detect some books.
2. Some books seem to have the wrong bar code(yeah, I know, something isn't right but it's returning the wrong book because it seems there are some books with the same barcode or there was a misprint on the book). Hence also searching for title, author, etc would help resolve that.
3. Some books are too old and do not have barcodes, e.g., old sheet music. Hence it would be nice to get some info about them and maybe any updated versions that have the barcode info to act as a replacement.
4. Usually the apps are a bit slow. If one has hundreds of books one doesn't want to scan, edit, add... just scan scan scan. This is what makes your app nice is that it can do the spines so at least it is fast in those cases.

I haven't used your app yet but I will try it when I get a chance.

no such table error

I got below errors while post image

Request Method: | POST

http://127.0.0.1:8000/api/create-bookshelf/
3.1.5
OperationalError
no such table: detect_spines_bookshelf

Great job! Thank you. I wish to know why you decided to resize the images to 500 pixels.

Thank you for your great code here.
When I tried to use Google Vision API - the results wasn't accurate because of the resize.
Is it possible to disable the resize?
Changing the width by hand cause the algorithm to be less effective.
Thanks.