ideonate / jupyter-innotater Goto Github PK
View Code? Open in Web Editor NEWInline data annotator for Jupyter notebooks
Home Page: https://jupyter-innotater.rtfd.io/
License: MIT License
Inline data annotator for Jupyter notebooks
Home Page: https://jupyter-innotater.rtfd.io/
License: MIT License
Is there any option to format the view of the tool. Like to shift the label boxes below the image viewer. Since using the default format gives very little space for the image.
Pandas is a common library used for data management. It would be very useful to support dataframes on top of numpy arrays.
I assume, based on the install instructions, that this project does not support JupyterLab? (also, trying to test the first example from the README doesn't yield any useful output; see below). Do you plan to introduce such support or is that outside of the scope of this project?
Running this code
import os
import numpy as np
import matplotlib.pyplot as plt
from jupyter_innotater import Innotater, ImageInnotation, BoundingBoxInnotation
img = plt.imread("roadrunner.png") # any test image
targets = np.zeros((1, 4))
Innotater(ImageInnotation([img]), BoundingBoxInnotation(targets))
gives me the following
Innotater(children=(HBox(children=(VBox(children=(ImagePad(value=b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00…
but no widgets or images displayed. I just tested in a classic Notebook, and it seems to work fine there.
Hi! I'm trying to produce a few simple bounding boxes for a set of images. Unfortunately, I run into the following error using the basic example from the project readme:
Clicking the box expands the following message:
[Open Browser Console for more detailed log - Double click to close this message]
Failed to load model class 'InnotaterModel' from module 'jupyter-innotater'
Error: No version of module jupyter-innotater is registered
at f.loadClass (http://localhost:8888/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/134.083e6b37f2f7b2f04b5e.js?v=083e6b37f2f7b2f04b5e:1:74855)
at f.loadModelClass (http://localhost:8888/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/150.467514c324d2bcc23502.js?v=467514c324d2bcc23502:1:10721)
at f._make_model (http://localhost:8888/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/150.467514c324d2bcc23502.js?v=467514c324d2bcc23502:1:7517)
at f.new_model (http://localhost:8888/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/150.467514c324d2bcc23502.js?v=467514c324d2bcc23502:1:5137)
at f.handle_comm_open (http://localhost:8888/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/150.467514c324d2bcc23502.js?v=467514c324d2bcc23502:1:3894)
at _handleCommOpen (http://localhost:8888/lab/extensions/@jupyter-widgets/jupyterlab-manager/static/134.083e6b37f2f7b2f04b5e.js?v=083e6b37f2f7b2f04b5e:1:73392)
at b._handleCommOpen (http://localhost:8888/static/lab/jlab_core.da7da3bf6a92a54183c1.js?v=da7da3bf6a92a54183c1:2:994423)
at async b._handleMessage (http://localhost:8888/static/lab/jlab_core.da7da3bf6a92a54183c1.js?v=da7da3bf6a92a54183c1:2:996413)
I'm trying to run jupyter-innotater inside Jupyter Lab, having installed it via conda install -c conda-forge jupyter_innotater
.
Here's the output of jupyter --version
, just in case. If more input is helpful to debug this issue, please let me know.
IPython : 8.4.0
ipykernel : 6.15.1
ipywidgets : 8.0.1
jupyter_client : 7.3.5
jupyter_core : 4.11.1
jupyter_server : 1.18.1
jupyterlab : 3.4.5
nbclient : 0.6.7
nbconvert : 7.0.0
nbformat : 5.4.0
notebook : 6.4.12
qtconsole : not installed
traitlets : 5.3.0
Hi!
First of all, thanks for what seems like a great plugin!
I am having some issues with getting the innotater to work.
I am running a fresh local installation of JupyterLab, with only the jupyter-innotater
installed, and when trying to run the example I get the output:
Error displaying widget: model not found
On further inspection of the JS-console I see the following errors:
manager.js:319 Uncaught (in promise) Error: Module jupyter-innotater, semver range ~0.2.1 is not registered as a widget module
at C.loadClass (manager.js:319)
at C.<anonymous> (manager-base.js:264)
at l (manager-base.js:45)
at Object.next (manager-base.js:26)
at manager-base.js:20
at new Promise (<anonymous>)
at Rtm6.S (manager-base.js:16)
at C.e._make_model (manager-base.js:258)
at C.<anonymous> (manager-base.js:247)
at l (manager-base.js:45)
utils.js:119 Error: Could not create a model.
at n (utils.js:119)
at async C._handleCommOpen (manager.js:62)
at async g._handleCommOpen (default.js:985)
at async g._handleMessage (default.js:1138)
What am I doing wrong? Thanks in advance for any help.
Depending on the image, it may be hard to see the bounding box. It would be nice to be able to set basic parameters for how the box is drawn (line width and color, and possible style (e.g. dashed, dotted, solid), should suffice). Here's an example where the bounding box around the roadrunner (in red) is hard to see.
Use Case: A widget for annotating multiple Bounding Boxes within a single image, each of which can belong to one of many classes.
Problem: If you draw a bounding box, but leave the dropdown on its default value, the one-hot labelling of the bounding box will be all 0 (e.g. no label assigned). If you change the assigned class in the drop down to any other class and reselect the default (first) item, then the labelling will be recorded properly.
Example Innotater
Call
# setup all the other variables, e.g. image list etc before this.
annotation_labels = ["default", "a", "b"]
innotater_args = {
"inputs": [
ImageInnotation(image_files, path=image_path, width=image_width),
TextInnotation(image_files, multiline=False)
],
"targets": [
RepeatInnotation(
(BoundingBoxInnotation, inno_bboxes),
(MultiClassInnotation, inno_labels,
{
"name": "ROI Type",
"classes": annotation_labels,
"dropdown": True
}
),
max_repeats=50,
min_repats=1
)
]
}
Innotater(**innotater_args)
I'm getting the above error when trying to do the example in the README.md. I'm using a jupyter notebook in a binder with the versions for ipywidgets, notebook, widgetsnbextension, jupyter_innotater and pillow from the requirements.txt (other package versions may differ because of additional package requirements, but those probably shouldn't cause the error...?)
Any ideas what might cause this?
Currently the width and height property in the ImageInnotation subclass are effective only if the canvas size is smaller than source image.
I would like to enlarge source image if there are no problems.
In imagewidget.js
if (wantwidth !== undefined && wantwidth > 0) {
self.usewidth = wantwidth;
self.canvas.setAttribute('width', wantwidth.toString());
// Disabling zoom if canvas width is larger than source image width.
// Therefore only shrinking is possible.
if (wantwidth > self.imgel.width) {
self.usewidth = self.imgel.width;
self.zoom = 1.0;
}
else {
self.usewidth = wantwidth;
self.zoom = wantwidth / self.imgel.width;
}
}
i tried using display, and without, @interact is working, and other wigdets
When adding the DataMixin
class (PR: #14), the inheritence of most of the classes in the repo have not been set properly.
An failing example:
import ipywidgets
from jupyter_innotater.data import Innotation, TextInnotation, BoundingBoxInnotation
class ButtonInnotationExtended(Innotation):
requires_data = False
def __init__(self, icon, description, on_click=None, **kwargs):
self.icon = icon
self.description = description
self.on_click_callback = on_click
super().__init__(**kwargs)
def on_click(self, callback, remove=False):
self.button.on_click(callback, remove)
def _create_widget(self):
if not hasattr(self, "button"):
self.button = ipywidgets.Button(description=self.description, layout=self.layout, icon=self.icon)
if self.on_click_callback is not None:
self.button.on_click(self.on_click_callback)
return self.button
return self.button
def update_ui(self, uindex):
pass
def update_data(self, uindex):
pass
Calling ButtonInnotationExtended(description='save', on_click=None, icon='save')
would fail with the exception TypeError: object.__init__() takes exactly one argument (the instance to initialize)
.
After reading this very good blog posts: Python’s super() considered super! and Cooperative multiple-inheritance paradigm in python, it seems that there is a better way to propagate unknown arguments through the class inheritance tree.
In the documentation, it says that images passed directly as numpy
arrays should be in RGB format. However, this doesn't seem to be the case. I've loaded an image in RGB format using matplotlib
's imread
and displayed it using imshow
(which uses RGB format for 3-channel images), and here's the comparison with the image shown by Innotator
:
import os
import numpy as np
import matplotlib.pyplot as plt
from jupyter_innotater import Innotater, ImageInnotation, BoundingBoxInnotation
# image is RGBA; drop alpha channel
img = plt.imread("roadrunner.png")[..., :-1]
plt.figure(figsize=(8,8))
plt.imshow(img)
plt.axis("off")
targets = np.zeros((1, 4))
Innotater(ImageInnotation([img]), BoundingBoxInnotation(targets))
When labelling one image to n > min_repeats elements, the following images to be labelled keep the same n rows (instead of displaying min_repeats).
or potentially add support for python 3.5
Looking into see if this is possible
a HxW image didn't work for me, but a HxWx1 did. Its no big deal really.
this is what I'm using on my end to get around it:
if len(img.shape) == 2:
img = img[..., None]
In BoundingBoxInnotation
once a bbox is created it's not possible to be edited. It would be nice if the edges and corners could be active and be moved for fine adjustment of existing bboxes.
Hi, I am wondering if functionality for annotating time series or signal data might be something that could be implemented in this project? The ability to iterate over a bunch of numpy arrays (ex. each array represents a snippet of ECG data containing N different rhythms to be labelled) would be especially helpful for my projects. Thanks for the great work!
I found the error while I was running the bellow like code with jupyter-innotater==0.2.2.
import io
import png
import numpy as np
print(png.__version__, np.__version__)
npim = np.zeros((256, 256, 3), dtype=np.uint8)
npim[:, :, 0] = 255
npim[:, ::8, 1] = 255
npim[:, 1::8, 1] = 255
npim[:, 2::8, 1] = 255
pngbytes = io.BytesIO()
pngmode = 'L' # Greyscale
if len(npim.shape) == 3:
if npim.shape[2] > 4:
raise Exception("Image numpy array appears to have more than 4 channels")
pngmode = ('L','LA','RGB','RGBA')[npim.shape[2]-1]
else:
npim = np.expand_dims(npim, axis=-1) # Need a third axis for channel
pngim = png.from_array(npim, mode=pngmode) # Don't have BGR available so flipped to RGB above
pngim.write(pngbytes) if hasattr(pngim, 'write') else pngim.save(pngbytes) # PyPNG API due to change after v0.0.19
pixel_value = pngbytes.getvalue()
pngbytes.close()
png.__version__='0.20220715.0' np.__version__='1.23.4'
---------------------------------------------------------------------------
ProtocolError Traceback (most recent call last)
Cell In [11], line 20
16 # h, w, ch = npim.shape
17 # npim = npim.reshape(h, w * ch)
19 pngim = png.from_array(npim, mode=pngmode) # Don't have BGR available so flipped to RGB above
---> 20 pngim.write(pngbytes) if hasattr(pngim, 'write') else pngim.save(pngbytes) # PyPNG API due to change after v0.0.19
21 pixel_value = pngbytes.getvalue()
22 pngbytes.close()
File /app/venv/lib/python3.9/site-packages/png.py:1320, in Image.write(self, file)
1309 """Write the image to the open file object.
1310
1311 See `.save()` if you have a filename.
(...)
1316 cannot be streamed again.
1317 """
1319 w = Writer(**self.info)
-> 1320 w.write(file, self.rows)
File /app/venv/lib/python3.9/site-packages/png.py:668, in Writer.write(self, outfile, rows)
665 a = array(fmt, itertools.chain(*check_rows(rows)))
666 return self.write_array(outfile, a)
--> 668 nrows = self.write_passes(outfile, check_rows(rows))
669 if nrows != self.height:
670 raise ProtocolError(
671 "rows supplied (%d) does not match height (%d)" %
672 (nrows, self.height))
File /app/venv/lib/python3.9/site-packages/png.py:703, in Writer.write_passes(self, outfile, rows)
700 elif self.bitdepth == 16:
701 rows = unpack_rows(rows)
--> 703 return self.write_packed(outfile, rows)
File /app/venv/lib/python3.9/site-packages/png.py:738, in Writer.write_packed(self, outfile, rows)
735 # raise i scope out of the for loop. set to -1, because the for loop
736 # sets i to 0 on the first pass
737 i = -1
--> 738 for i, row in enumerate(rows):
739 # Add "None" filter type.
740 # Currently, it's essential that this filter type be used
741 # for every scanline as
742 # we do not mark the first row of a reduced pass image;
743 # that means we could accidentally compute
744 # the wrong filtered scanline if we used
745 # "up", "average", or "paeth" on such a line.
746 data.append(0)
747 data.extend(row)
File /app/venv/lib/python3.9/site-packages/png.py:658, in Writer.write.<locals>.check_rows(rows)
655 wrong_length = False
656 if wrong_length:
657 # Note: row numbers start at 0.
--> 658 raise ProtocolError(
659 "Expected %d values but got %d values, in row %d" %
660 (vpr, len(row), i))
661 yield row
ProtocolError: ProtocolError: Expected 255 values but got 256 values, in row 0
I checked that latest pypng.from_array is unsupported 3D ndarray.
https://gitlab.com/drj11/pypng/#release-0020
So I made a simple patch code and check that it work.
print(f"{png.__version__=} {np.__version__=}")
npim = np.zeros((256, 256, 3), dtype=np.uint8)
npim[:, :, 0] = 255
npim[:, ::8, 1] = 255
npim[:, 1::8, 1] = 255
npim[:, 2::8, 1] = 255
pngbytes = io.BytesIO()
pngmode = 'L' # Greyscale
if len(npim.shape) == 3:
if npim.shape[2] > 4:
raise Exception("Image numpy array appears to have more than 4 channels")
pngmode = ('L','LA','RGB','RGBA')[npim.shape[2]-1]
else:
npim = np.expand_dims(npim, axis=-1) # Need a third axis for channel
h, w, ch = npim.shape
npim = npim.reshape(h, w * ch)
npim = np.ascontiguousarray(npim)
pngim = png.from_array(npim, mode=pngmode) # Don't have BGR available so flipped to RGB above
pngim.write(pngbytes) if hasattr(pngim, 'write') else pngim.save(pngbytes) # PyPNG API due to change after v0.0.19
pixel_value = pngbytes.getvalue()
pngbytes.close()
This error only occurs when opencv-python was not installed.
If you want to fix the error, I think that you need to fix the pypng version to 0.0.19 or fix the jupyter_innotater/data.py .
Thanks for releasing this inspiring project!
I'm trying to use voila[0] to turn jupyter-innotater into a stand alone app. Currently I'm facing a range of error messages in the browser console that indicate that innotater is currently
not compatible with voila.
I believe fixing this has huge benefits especially when non technical people are involved into the labeling process.
Voila [0] is popular project that allows to turn interactive jupyter notebooks into standalone apps just by serving them with voila example.ipynb
.
[tool.poetry.dependencies]
python = "^3.7"
jupyter_innotater = { path = "./jupyter-innotater/jupyter-innotater" }
jupyter = "^1.0.0"
jupyterlab = "^1.1.4"
nbconvert = ">=5.5"
ipywidgets = "^7.5.1"
pandas = "^0.25.1"
opencv-python = "^4.1.1"
voila = "^0.1.17"
[tool.poetry.dev-dependencies]
Ensure that current setup is properly configured by running example from
voila repo.
voila notebooks/interactive.ipynb
Running the innotator example with the same setup
voila jupyter-innotater/Example/FlatImages.ipynb
doesn't work:
The following voila discussions might be helpful to track down why innotator isn't compatible with
voila.
It would be nice to have a Done button widget that could be included in the layout and pressed to, e.g., call IPython.display.clear_output()
. This would just make the notebook look nicer after you're finished with annotations.
Hi there,
I really value your tool, thanks for making it. I've used it on jupyter notebook, and it works perfectly for me.
A few students I work with use the colab notebook, and for them the innotator widgets don't display (at all).
The chrome developer console shows an error:
"Error has occurred while trying to update output."
"Error: Widgets module jupyter-innotater is not supported"
I'm not sure if you can think of an easy work around for this, it would be helpful.
Other ipywidgets do work fine.
I copied one of your examples into a colab notebook to reproduce the issue in case it helps.
https://colab.research.google.com/drive/1ANFSGUyX3K_6DaZJCgZWPHGYyGOnOb0T?usp=sharing
On a potentially related note, I noticed when I installed jupyter_innotator on the notebook that I need to install the package both on the kernel env and on the notebook env (I use a single notebook for multiple kernels, so they are actually in different python environments). This makes some sense, because there is both browser and python logic involved. Before I did that, I found a similar error when I only installed the package on the kernel environment, and not on the notebook one.
Hello,
I have been using your innotator for a while and it is great for me to annotate some data on my remote server.
I have changed some code in one of the .js files but I didn't see any changes in the widget.
Could you point me to a better direction?
Thanks
I tried another library called ipyannotations but it's currently broken. I saw this lib, jupyter-innotater, but it doesn't seem to support segmentation annotation.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.