nocotan/geodesical_skew_divergence

PyTorch implementation of α-geodesical skew divergence

α-Geodesical Skew Divergence

Official PyTorch Implementation of "α-Geodesical Skew Divergence".

• [Entropy]
• [arXiv]

The asymmetric skew divergence smooths one of the distributions by mixing it, to a degree determined by the parameter λ, with the other distribution. Such divergence is an approximation of the KL divergence that does not require the target distribution to be absolutely continuous with respect to the source distribution. In this paper, an information geometric generalization of the skew divergence called the α-geodesical skew divergence is proposed, and its properties are studied.

Installation

From PyPI

Install gs_divergence from PyPI via

$ pip install gs_divergence

From GitHub

Alternatively, you can install the latest version directly from the GitHub repository with the following command.

$ git clone https://github.com/nocotan/geodesical_skew_divergence
$ python setup.py install

Usage

Compute divergence from two Tensors

import torch
from gs_divergence import gs_div

a = torch.Tensor([0.1, 0.2, 0.3, 0.4])
b = torch.Tensor([0.2, 0.2, 0.4, 0.2])

div = gs_div(a, b, alpha=-1, lmd=0.5)

Compute gradients

import torch
from gs_divergence import gs_div

a = torch.tensor([0.1, 0.2, 0.3, 0.4], requires_grad=True)
b = torch.tensor([0.2, 0.2, 0.4, 0.2])

div = gs_div(a, b, alpha=-1, lmd=0.5)
dif.backward()

parameter	description
input	Tensor of arbitrary shape
target	Tensor of the same shape as input
alpha	Specifies the coordinate systems which equiped the geodesical skew divergence (default=-1)
lmd	Specifies the position on the geodesic (default=0.5)
reduction	Specifies the reduction to apply to the output: 'none' | 'batchmean' | 'sum' | 'mean'. 'none': no reduction will be applied 'batchmean': the sum of the output will be divided by the batchsize 'sum': the output will be summed 'mean': the output will be divided by the number of elements in the output default='sum'

Definition of α-Geodesical Skew Divergence

Visualizations of the α-Geodesical Skew Divergence

Monotonicity of the α-geodesical skew divergence with respect to α

Continuity of the α-geodesical skew divergence with respect to α and λ.

Citation

@Article{e23050528,
AUTHOR = {Kimura, Masanari and Hino, Hideitsu},
TITLE = {alpha-Geodesical Skew Divergence},
JOURNAL = {Entropy},
VOLUME = {23},
YEAR = {2021},
NUMBER = {5},
ARTICLE-NUMBER = {528},
URL = {https://www.mdpi.com/1099-4300/23/5/528},
ISSN = {1099-4300},
ABSTRACT = {The asymmetric skew divergence smooths one of the distributions by mixing it, to a degree determined by the parameter λ, with the other distribution. Such divergence is an approximation of the KL divergence that does not require the target distribution to be absolutely continuous with respect to the source distribution. In this paper, an information geometric generalization of the skew divergence called the α-geodesical skew divergence is proposed, and its properties are studied.},
DOI = {10.3390/e23050528}
}

geodesical_skew_divergence's Issues

add Repo size badge

add test badge

add DOI

GSDivLoss class

add LICENSE badge

test case: alpha=-1

test case: GSDivLoss

add unit tests

add citation

test case: alpha=0

add downloads badge

add visualizer

add star badge

add issue badge

implementation of the symmetrized geodesical skew divergence

add github actions

test case: alpha=1

test case: lower bound

test case: alpha=2

test case: special case of geodesical skew divergence

register pypi

Blush up README.md

import GSDivLoss (__init__.py)

test case: upper bound