flamato/awesome-image-colorization

:books: A collection of Deep Learning based Image Colorization and Video Colorization papers.

Awesome-Image-Colorization

A collection of Deep Learning based Image Colorization papers and corresponding source code/demo program, including Automatic and User Guided (i.e. with User Interaction) colorization, as well as video colorization.

Feel free to create a PR or an issue. (Pull Request is preferred)

Outline

• Automatic Image Colorization
• User Guided Image Colorization
  • Based on color strokes
  • Based on reference color image
  • Based on color palette
  • Based on language(text)
• Techniques of Improving Image Colorization
• Video Colorization
  • Automatically
  • Based on reference

---

1. Automatic Image Colorization

Paper	Source	Code/Project Link
Learning Large-Scale Automatic Image Colorization	ICCV 2015	[project] [code]
Deep Colorization	ICCV 2015
Learning Representations for Automatic Colorization	ECCV 2016	[project] [code]
Colorful Image Colorization	ECCV 2016	[project] [code]
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification	SIGGRAPH 2016	[project] [code]
Learning Diverse Image Colorization	CVPR 2017	[code]
Structural Consistency and Controllability for Diverse Colorization	ECCV 2018
Coloring With Limited Data: Few-Shot Colorization via Memory Augmented Networks	CVPR 2019	[project] Unofficial: [code (PyTorch)]
ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution	WACV 2020	[code]
Instance-aware Image Colorization	CVPR 2020	[code] [project]
Pixelated Semantic Colorization	IJCV 2020
Colorization Transformer	ICLR 2021	[code]
Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies	ACM MM 2021	[code]
Towards Vivid and Diverse Image Colorization with Generative Color Prior	ICCV 2021
SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also reference based)	AAAI 2022
DeOldify	Software	[code]

2. User Guided Image Colorization

2.1 Based on color strokes

Image Type	Paper	Source	Code/Project Link
Manga	Manga colorization	SIGGRAPH 2006	[Project Home] Unofficial: [code]
Line art / Sketch / Manga	LazyBrush: Flexible Painting Tool for Hand-drawn Cartoons	Eurographics 2009	[Project Home & Software] Unofficial: [code1 (Matlab)] [code2 (C#)] [code3]
Line art / Sketch	Outline Colorization through Tandem Adversarial Networks	1704.08834	[code]
Line art / Sketch	Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks	1705.01908	[code]
Natural Gray-Scale	Real-Time User-Guided Image Colorization with Learned Deep Priors	SIGGRAPH 2017	[project] [code1] [code2]
Sketch	Scribbler: Controlling Deep Image Synthesis with Sketch and Color	CVPR 2017
Line art	User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks	ACM MM 2018	[code]
Line art	Style2paints V3 : Two-stage Sketch Colorization	SIGGRAPH Asia 2018	[Project] [Paper's code(V3)] [Portable software(V4.5)] Unofficial: [code1] [code2] [PaintsTensorFlow]
Natural Gray-Scale	Interactive Deep Colorization Using Simultaneous Global and Local Inputs (also palette based)	ICASSP 2019
Line art	PaintsChainer (Petalica Paint)	Online Demo (by Preferred Networks, Inc.)	[V3 Demo] [V1 code]
Line art	User-Guided Line Art Flat Filling with Split Filling Mechanism	CVPR 2021	[Project] [code]
Line art	Dual Color Space Guided Sketch Colorization	TIP 2021

2.2 Based on reference color image

Image Type	Paper	Source	Code/Project Link
Line art	Style2paints V1 : Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN	ACPR 2017	[Code] Unofficial: [code]
Manga	Comicolorization: Semi-Automatic Manga Colorization (also palette based)	SIGGRAPH Asia 2017	[code]
Sketch	TextureGAN: Controlling Deep Image Synthesis with Texture Patches	CVPR 2018	[code]
Natural Gray-Scale	Deep Exemplar-based Colorization	SIGGRAPH 2018	[code]
Natural Gray-Scale	Example-Based Colourization Via Dense Encoding Pyramids (also palette based)	Pacific Graphics 2018	[code]
Natural Gray-Scale	A Superpixel-based Variational Model for Image Colorization	TVCG 2019
Natural Gray-Scale	Automatic Example-based Image Colourisation using Location-Aware Cross-Scale Matching	TIP 2019
Line art / Sketch	Adversarial Colorization Of Icons Based On Structure And Color Conditions	ACM MM 2019	[Code]
Line art / Sketch	Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence	CVPR 2020	[project] Unofficial: [code]
Natural Gray-Scale	Stylization-Based Architecture for Fast Deep Exemplar Colorization	CVPR 2020	[code]
Manga	Manga Filling Style Conversion with Screentone Variational Autoencoder (also palette based)	SIGGRAPH Asia 2020	[project]
Line art / Sketch	Colorization of Line Drawings with Empty Pupils	PG 2020
Line art / Sketch	Active Colorization for Cartoon Line Drawings	TVCG 2020
Natural Gray-Scale	Gray2ColorNet: Transfer More Colors from Reference Image	ACM MM 2020
Line art / Sketch	Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization	WACV 2021
Natural Gray-Scale	Globally and Locally Semantic Colorization via Exemplar-Based Broad-GAN	TIP 2021
Natural Gray-Scale	Yes, "Attention Is All You Need", for Exemplar based Colorization	ACM MM 2021
Natural Gray-Scale	SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also automatic)	AAAI 2022

2.3 Based on color palette

Image Type	Paper	Source	Code/Project Link
Natural Image	Palette-based Photo Recoloring	SIGGRAPH 2015	[project]
Manga	Comicolorization: Semi-Automatic Manga Colorization (also reference based)	SIGGRAPH Asia 2017	[code]
Natural Gray-Scale	Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation (also text based)	ECCV 2018	[code]
Natural Gray-Scale	Example-Based Colourization Via Dense Encoding Pyramids (also reference based)	Pacific Graphics 2018	[code]
Natural Gray-Scale	Interactive Deep Colorization Using Simultaneous Global and Local Inputs (also strokes based)	ICASSP 2019

2.4 Based on language or text

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale / Sketch	Language-Based Image Editing with Recurrent Attentive Models	CVPR 2018	[code]
Natural Gray-Scale	Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation (also palette based)	ECCV 2018	[code]
Natural Gray-Scale	Learning to Color from Language	NAACL 2018	[code]
Line art	Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss	ICCV 2019	[code] [code2 (GUI)]
Scene Sketch	Language-based Colorization of Scene Sketches	SIGGRAPH Asia 2019	[code] [project]
Natural Gray-Scale	L-CoDe: Language-based Colorization using Color-object Decoupled Conditions	AAAI 2022

3. Techniques of Improving Image Colorization

Paper	Source	Code/Project Link
Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects	ICCV 2021	[project] [code(metric)]
Line Art Colorization Based on Explicit Region Segmentation	Pacific Graphics 2021	[code]

4. Video Colorization

4.1 Automatically

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	Fully Automatic Video Colorization with Self-Regularization and Diversity	CVPR 2019	[code]

4.2 Based on reference

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	Switchable Temporal Propagation Network	ECCV 2018
Natural Gray-Scale	Tracking Emerges by Colorizing Videos	ECCV 2018	[code]
Natural Gray-Scale	Deep Exemplar-based Video Colorization	CVPR 2019	[code]
Natural Gray-Scale	DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement	SIGGRAPH Asia 2019	[code] [project]
Natural Gray-Scale	Reference-Based Video Colorization with Spatiotemporal Correspondence	2011.12528
Line art	The Animation Transformer: Visual Correspondence via Segment Matching	ICCV 2021	[App]
Line art	Reference-Based Deep Line Art Video Colorization	TVCG 2022