batermj/scan

SCAN: Learning Abstract Hierarchical Compositional Visual Concepts

SCAN

About

Replicating SCAN algorithm described in Google DeepMind's paper "SCAN: Learning Abstract Hierarchical Compositional Visual Concepts"

Image datasets are created with Rodent envrironment.

Requirements

• Tensorflow 1.2 or later
• Python2 or 3

How to train

First extract dataset, and then run main.py

$ tar xvf data.tar.gz
$ python main.py

Result

Sym2img

Symbol to image conversion result.

1. Generated images when wall_color=white is specified.

2. Generated images when wall_color=white, floor_color=white are specified.

3. Generated images when wall_color=white, floor_color=white, obj_color=white are specified.

4. Generated images when wall_color=white, floor_color=white, obj_color=white, obj_id=ice_lolly are specified.

Img2sym

Input	Output
	obj_color=white, wall_color=white, floor_color=white, obj_id=ice_lolly

(All of the outputs are correct.)

Input	Output
	obj_color=purple, wall_color=dark_yellow, obj_id=hat

(Correct obj_color was red, but confused as purple. floor_color was not specifiled in the output.)

Concept recombination

Concept reombination result as image outputs.

1. Recombination result of (obj_color=white) AND (wall_color=white). (Correct result should have white object and white wall.)

2. Recombination result of (obj_color=white, obj_id=ice_lolly) IN_COMMON (obj_color=white, wall_color=white. (Correct result should have white object.)

3. Recombination result of (obj_color=white, wall_color=white) IGNORE (obj_color=white). (Correct result should have white wall.)

βVAE disentanglement

Disentanglement result for latent variables for object parameters. (Wall color, Object color, Floor Color, Object Type, Object position).

Wall color

Obj color

Floor color

Obj Type

Obj pos (and Obj Type)