Comments (4)
Hi Romain,
Yes our main interest is obtaining the masks and keypoints by training only on raw images.
I am not sure I understand your request. Our model can be directly applied if you simply resize the 256x256 images to 128x128.
If you want to re-train on 256x256 images, you can
- delete the hard coded architecture in line 149-225 in generator.py to allow it consume 256x256 images..
- copy line 41 to line 42 in discriminator.py to allow it consume 256x256 images.
If you have further questions, don't hesitate to reach me.
Bests,
Xingzhe
from ganseg.
Hi,
I'm sorry i didn't go straight to the point : we run your model successfully on celeba but we failed to apply it on this toy example, where the raw images we train the model on are plain blue rectangles of various sizes and position on a plain grey background :
The model succeeds to generate such shapes but fail into segmenting the blue rectangle from the background.
- here are two example with 3 keypoints :
(the segmentation mask is a tiny point on this one)
- here is an other examples with 8 keypoints :
---------------------- more info
- the number of processing epochs for these failed experiments was similar (and actually higher) than the one for the succeeding runs on celeba.
- we used a the same batch size of 4 for all experiments (celeba and toy samples)
---------------------- why we asked about the preprocessing first
Since we produced the model input h5 file by applying the celeba processing file on our toy samples, our previous question aimed to verify that the preprocessing step was not in fault. Meanwhile, we actually wrote a generic preprocessing file on our own, that only requires a folder with raw images in it (without any segmentation information). It's available here if required : https://github.com/romain-rsr/colab/blob/main/uprocess.py
Bests,
Romain
from ganseg.
Hi Romain,
Thanks for this experiment! It is indeed very very interesting! It illustrates some insights I have never thought about.
I also tested by myself on GANSeg, and it didn't work. I think it is due to the (positional encoding or part embedding or background embedding) overfit this too simple dataset.
In parallel, I also tested on two different unsupervised keypoint detection methods, which use keypoint embeddings. They also fail although slightly slightly better than GANSeg, probably because they don't have background embeddings and positional encoding. I doubt if all methods using embeddings without additional care could lead to this problem to some extend.
Finally I tested on a method without using embeddings: https://xingzhehe.github.io/autolink/
and it finally works:
Although it only detect keypoints and their linkages, masks can be easily extracted.
Bests,
Xingzhe
from ganseg.
Hi,
Thanks a lot for these complementary works on our examples. I focus my experiments on getting the segmentation encoding for a private dataset whom characteristics are halfway between those of the toy dataset and those of highly structured datasets (celeba, flower etc). Since we can't communicate publicly on it, I'll try to find a dataset that share the same problematics (high details and diversity with very low overall structuration)
Bests,
Romain
from ganseg.
Related Issues (4)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ganseg.