ValueError: could not broadcast input array from shape

[bgr33r]

ValueError: could not broadcast input array from shape

[bgr33r]

Update: The above problem was unrelated to loss_shape. I figured out through some trial and error that the format of the segmentation data must have a color index between 1-256 instead of RGB. Just updating this here in case someone is putting their own training data together. Though, it would be useful if there was a better descriptor of how this code uses loss_shape/how the loss function is working in here. Thanks!

[ahundt]

Could you elaborate on exactly the formats you're discussing here?

[bgr33r]

Sure. I created training data using RGB .jpg format images that I downloaded from the web. I took the raw .jpg files, and using GIMP, created .jpg segmented images that were RGB format. I converted these to .png to match the format that the code was written to utilize. I then banged my head on my desk for a while until I thought to examine the image properties of the VOC2012 segmentation data and compare it to my own data. I found that while my segmentation data was RGB, the VOC segmentation data was Indexed color (256 colors). I wrote a short python script that read in the jpgs and converted to 256 colors .png files (below). I am currently training and on epoch 59/250. Not sure about success yet - but there it is. I do have concerns that when I converted from RGB to 256 colors I accidentally matched a color in the VOC data because I picked colors using RGB - I will be checking that next.

#script to batch convert to RGB .jpg images to .png 256 color images.
WD = "~/SegmentationObject/"
os.chdir(WD)
files = glob.glob ('*.jpg')
for infile in files:
file, ext = os.path.splitext(infile)
im = Image.open(infile).convert('P',colors=256)
im.save(file + ".png")

[ahundt]

Oh yeah that's a tricky one! For others reading, the png values are actually single channel ids, which are reused in Pascal VOC as class ids. What you see when you look at a png files are the values resulting from a lookup into what's known as a color palette which is a map from id to a color.

jpg is a bit risky to use for segmentation data because it may change colors depending on compression settings, and thus identify a different class. I think python pillow can be used for format conversion and creating the palette.

[bgr33r]

Yup! And, somewhat ironically, the .jpgs files I initially created were maximum quality and histograms of the colors showed my 3 distinct classes perfectly - but converting from .jpg to .png using PIL introduced new colors (for example, band 84 was cut into bands 83 and 84). As a work around, I managed to use quantize in PIL to convert my .jpgs to .pngs without loss (i.e. new colors) - but I have very little control over the class IDs and for palette I can only use an "adaptive" or "web" palette (see PIL readme). One major problem I also had was that two of my color classes were different enough in RGB that quantize converted my green and purple classes differently. I don't have any examples I can share, but with my green colored class, quantize combined the green colors with the VOID outline from VOC, but with my purple class, quantize kept the classes separate. In the end, I went back and manually recolored my images using the color select and bucket tools in GIMP. Another option would have been to convert into numpy format - but I wasn't sure how the COCO data was handled in the main code, and I didn't want to open a new can of worms.

Below is my new code I used to convert .jpg to .png:

`#convert to 3 color palette index

from PIL import Image
import glob, os
WD = "~/SegmentationObject/"
os.chdir(WD)
files = glob.glob ('*.jpg')
for infile in files:
file, ext = os.path.splitext(infile)
im = Image.open(infile).quantize(colors=3)
im.save(file + ".png")`

[ahundt]

Wow, thanks for the detailed explanation! I'm sure that will help someone out in the future.

[zwlshine]

@bgr33r
Thank you very much for sharing you knowledge, I met this problem when I use this code for training 2 classes images.