I have a question about phase2

[clovaai]

I have a question about phase2

[clscy]

[8uos]

Hi, I have checked the code and tested it, but there was no error.

A possible mistake is calling dataset directly in multiple times, like this:

dset = FactTrainDataset(...)
in_style_ids = dset[0][0]
trg_style_ids = dset[0][3]
...

Because the output of dataset.__getitem__ changes every time you call it(because of random sampling), please modify the code like this:

dset = FactTrainDataset(...)
batch = dset[0]
in_style_ids = batch[0]
trg_style_ids = batch[3]
...

or use the Dataloader defined in datasets/__init__.py

Sorry for the inconvenience.

[clscy]

Thank you for your reply。I mean that in the p2dataset.py,
first, through the code on 122 line(self.sample_input(self.n_in_chars))，in_fonts will have three styles fonts(because self.n_in_chars = 3),
then in self.check_and_sample (...), the code on 113 line(trg_fonts = self.get_available_fonts(trg_unis, in_fonts)), for trg_unis, trg_fonts are random chosed based code on 55 line (avail_font = random.choice(avail_fonts)),.I think the operation will cause that trg_imgs possibly don't match to in_fonts.
for example:
in_fonts are ["fontA", "fontB", "fontC"], fontA include a component compA, fontB include a component compB, fontC include a component compC,
but trg_imgs(s.t compnent imgs) possibly occur that compA represent the style of fontB, not represent the style of fontA.
is it right? Thank you.

[8uos]

I understood. It is right, not a bug.
In phase 2, we aim to train the model to build new component-styles not included in input images("compA represent the style of fontB" of your example) through factorizing and combining the component-styles included in input images("compA represent the style of fontA ..." of your example).
So, what you are worrying about is what we intended to.
The detailed process is described in Section 3.3 of our paper.

Thanks for the question.

[clscy]

OK， thank you. In other words, in phase1 trg_imgs match to in_fonts, in phase2 may not match to in_fonts.
In your paper, the generated Style interpolation graphs is good, how is it generated?

[SanghyukChun]

@clscy
I presume you mean the style interpolation figure (Fig 10, below)

We followed many previous style transfer literature for the style interpolation (e.g., AdaIN https://arxiv.org/abs/1703.06868)
More precisely, we generate the interpolated samples by

• Decoder(content_features, content_factors, interpolated_style_factors) where
• content_features = ContentEncoder(content_glyph)
• content_factors = StyleEncoder_ContentFactorDecomposed(content_glyph, target_components)
• style_factors = StyleEncoder_StyleFactorDecomposed(reference_glyphs, reference_components)
• interpolated_style_factors = lambda * style_factors_1 + (1 - lambda) style_factors_2, here lambda is between 0 and 1 (the figure shows the uniform lambda range), and style_factors_1 and style_factors_2 are from the most left / right fonts, respectively.

[clscy]

Ok, thank you.