wesselb/mlkernels

Kernels, the machine learning ones

mlkernels's Issues

Categorical kernel can not use stretch()

Hi @wesselb,

I was implementing CategoricalKernel() where I think we can not make use of the stretch() method. The reason is, stretch() directly divides inputs by length scales, which is not suitable for CategoricalKernel(). As of now, I could think of the following version. Can you suggest a better idea or am I missing something here?

import lab as B
from matrix import Dense

from . import _dispatch
from .. import Kernel

__all__ = ["Categorical"]


class Categorical(Kernel):
    """Categorical kernel based on Hamming distance."""
    
    def __init__(self, scale):
        self.scale = scale

    def _compute(self, dists):
        return B.exp(-1. * dists/self.scale ** 2)

    @property
    def _stationary(self):
        return True

    @_dispatch
    def __eq__(self, other: "Categorical"):
        return True

# After implementing pw_ne and ew_ne in LAB.

@_dispatch
def pairwise(k: Categorical, x: B.Numeric, y: B.Numeric):
    return Dense(k._compute(B.pw_ne(x, y)))


@_dispatch
def elwise(k: Categorical, x: B.Numeric, y: B.Numeric):
    return k._compute(B.ew_ne(x, y))

Also, I think this problem could be generalized for all kernels that can not scale the inputs directly (Gibbs kernel in 4.32).

Decaying kernel not positive definite in ND

I was trying the decaying kernel (not with mlkernel itself because I'm lazy), and it results not pos def when used in more than 1 dimensions:

import numpy as np
from matplotlib import pyplot as plt

nd = 2
alpha = 1
beta = np.ones(nd)
nx = 5

def kernel(x, y, alpha, beta):
    bn2 = np.sum(beta ** 2)
    xyb = x + y + beta
    xybn2 = np.sum(xyb ** 2, axis=-1)
    return (bn2 / xybn2) ** (alpha / 2)

gen = np.random.default_rng(202206241555)
x = gen.uniform(0, 5, (nx, nd))
c = kernel(x[None, :, :], x[:, None, :], alpha, beta)
w = np.linalg.eigvalsh(c)
error = len(c) * np.finfo(float).eps * np.max(w)
if np.min(w) < -error:
    print('NOT POSITIVE!')

fig, ax = plt.subplots(num='decaying', clear=True)

ax.plot(w)

fig.show()

Not working with JAX 0.4

Hi,

Firstly thanks for creating this package. I have been finding it very useful in my research.

I recently updated JAX to version 0.4.1. The JAX backend for mlkernels no longer seems to work. Running the following code yields the following error:

Code:

from mlkernels.jax import EQ
import jax.numpy as np

x = np.array([1.0,2.0])

k = EQ()

k(x,x)

Error:

RuntimeError: For kernel "EQ()", could not resolve arguments "[1. 2.]" and "[1. 2.]".

Would it be possible to support the newer version of JAX? I am happy to help with this but unsure of where to start...

I did notice that arrays in JAX now seem to have type jaxlib.xla_extension.Array instead of jaxlib.xla_extension.DeviceArray. I am not sure how to add this type to mlkernels. Any help would be appreciated :)

Non-stationary kernel implementations

I am working on a project where I am planning to implement some useful generalized non-stationary kernel papers like this one. I saw Stheno a few days back and thought of using it as a potential platform for our implementations. Would you be interested in full-time collaborating with us or maybe helping us occasionally with the project?