keras-team/keras-core

A multi-backend implementation of the Keras API, with support for TensorFlow, JAX, and PyTorch.

Keras Core is becoming Keras 3 and has moved to keras-team/keras

Multi-backend Keras has a new repo: keras-team/keras. Open any issues / PRs there. keras-team/keras-core is no longer in use.

Keras Core was the codename of the multi-backend Keras project throughout its initial development (April 2023 - July 2023) and its public beta test (July 2023 - September 2023). Now, Keras Core is gearing up to become Keras 3, to be released under the keras name. As such, we've moved development back to keras-team/keras.

Meanwhile, the legacy tf.keras codebase is now available at keras-team/tf-keras.

If you were a contributor to Keras Core -- thank you! Your commits have been reapplied in keras-team/keras under your own authorship, so that your contribution record is fully preserved.

Likewise, if you were a contributor to keras-team/keras before the swap, your contribution history has been fully preserved: the new commits have been reapplied on top of the old git tree.

keras-core's Issues

pip build fails

I am trying to generate the whl file locally to do a local install of the package but I get the No module named build error. FYI, I am running the build py file in a conda env

Unexpected values returned by MeanMetricWrapper subclass

I have a very interesting test case that I am unable to debug. I implemented the Hinge loss (check #31 ), and imported the hinge(...) function for the corresponding metric implementation. The values that are coming out of metric are way off than the actual values but the values coming out of the loss function are as expected. Here is the test case:

y_true = tf.constant([[-1., 1., -1., 1.], [-1., -1., 1., 1.]])
y_pred = tf.constant([[-0.3, 0.2, -0.1, 1.6], [-0.25, -1.0, 0.5, 0.6]])
sample_weight = tf.constant([1.5, 2.0])

##### Hinge loss #########################

# TF implementation
tf.keras.losses.Hinge()(y_true, y_pred, sample_weight)
# output <tf.Tensor: shape=(), dtype=float32, numpy=0.8625>

# My implemetation
keras_core.losses.Hinge()(y_true, y_pred, sample_weight)
# ouput <tf.Tensor: shape=(), dtype=float32, numpy=0.8625>

##### Hinge metric #########################

# TF metric implementation
tf.keras.metrics.Hinge()(y_true, y_pred, sample_weight)
# output <tf.Tensor: shape=(), dtype=float32, numpy=0.49285716> 

# My metric implementation
keras_core.metrics.Hinge()(y_true, y_pred, sample_weight=sample_weight)
# output <tf.Tensor: shape=(), dtype=float32, numpy=0.575>

Here is how the the class implementation looks like:

@keras_core_export("keras_core.metrics.Hinge")
class Hinge(reduction_metrics.MeanMetricWrapper):
    def __init__(self, name="hinge", dtype=None):
        super().__init__(fn=keras_core.losses.hinge, name=name, dtype=dtype)

    def get_config(self):
        return {"name": self.name, "dtype": self.dtype}

Expected behavior:

Given that the loss function is working as expected, the metric implementation should work as we are just importing the same function in the corresponding metric class

Image classification with EANet (External Attention Transformer)

Tracking issue for Probabilistic Metrics & Losses

Probabilistic Metrics

• Cross Entropy Group
• Poisson
• KL Divergence

Training stuck for `text_classification_from_scratch` example

First non-trivial issue encountered in keras.io example conversion: text_classification_from_scratch doesn't train. Loss / accuracy can't get above random baseline. It reliably trains in tf.keras.

Main suspect: embedding layer?

Error fitting custom model if its build() method was already called on jax backend only

Still playing with custom models. Encountered this issue on the jax backend (works fine on other backends):

import keras_core as keras
import numpy as np

class MyCustomModel(keras.Model):
    def __init__(self, units, **kwargs):
        super().__init__(**kwargs)
        self.units = units
        self.hidden1 = keras.layers.Dense(self.units)
    
    def call(self, inputs):
        return self.hidden1(inputs)

    def get_config(self):
        return {"units": self.units}

    @classmethod
    def from_config(cls, config):
        return cls(**config)

X = np.random.rand(100, 3)
Y = np.random.rand(100, 2)

model = MyCustomModel(2)
model.build(X.shape)  # issue disappears if you comment out this line
model.compile(loss="mse", optimizer="nadam")
history = model.fit(X, Y)  #!!! IndexError: list index out of range

The issue disappears if you comment out the model.build(X.shape) line.

Below is the full stacktrace:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[1], line 26
     24 model.build(X.shape)
     25 model.compile(loss="mse", optimizer="nadam")
---> 26 history = model.fit(X, Y)

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/utils/traceback_utils.py:122, in filter_traceback.<locals>.error_handler(*args, **kwargs)
    119     filtered_tb = _process_traceback_frames(e.__traceback__)
    120     # To get the full stack trace, call:
    121     # `keras_core.config.disable_traceback_filtering()`
--> 122     raise e.with_traceback(filtered_tb) from None
    123 finally:
    124     del filtered_tb

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/optimizers/nadam.py:82, in Nadam.build(self, var_list)
     80 self._momentums = []
     81 self._velocities = []
---> 82 self._u_product = backend.Variable(1.0, dtype=var_list[0].dtype)
     84 for var in var_list:
     85     self._momentums.append(
     86         self.add_variable_from_reference(
     87             reference_variable=var, name="m"
     88         )
     89     )

IndexError: list index out of range

GAN overriding Model.train_step: @fchollet

The backend functions are not available when outside the keras-core directory

I used conda, I created and activated an empty environment with just Python 3.8, I cloned the keras-core repo and followed the installation instructions, including pip install -r requirements and python pip_build.py --install.

Now when I open a Python shell from the root keras-core directory, everything works fine.

However, when I open a Python shell anywhere else, I'm missing most functions in the keras_core.backend package. Here's what I get:

>>> import keras_core.backend as K
Using TensorFlow backend
>>> dir(K)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__',
'__spec__', 'backend', 'clear_session', 'epsilon', 'floatx', 'get_uid', 'image_data_format', 'set_epsilon',
'set_floatx', 'set_image_data_format']

As you can see, I'm missing K.shape(), K.numpy and almost everything else.

For comparison, this is what I get when I launch the shell from the keras-core root directory:

>>> import keras_core.backend as K
Using TensorFlow backend
>>> dir(K)
['AutocastScope', 'DYNAMIC_SHAPES_OK', 'KerasTensor', 'StatelessScope', 'Variable', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'any_symbolic_tensors', 'backend', 'cast', 'common', 'compute_output_spec', 'cond', 'config', 'convert_to_numpy', 'convert_to_tensor', 'core', 'epsilon', 'floatx', 'get_autocast_scope', 'get_stateless_scope', 'gru', 'image', 'image_data_format', 'in_stateless_scope', 'is_float_dtype', 'is_keras_tensor', 'is_tensor', 'lstm', 'math', 'name_scope', 'nn', 'numpy', 'print_msg', 'random', 'rnn', 'scatter', 'set_epsilon', 'set_floatx', 'set_image_data_format', 'shape', 'standardize_data_format', 'standardize_dtype', 'standardize_shape', 'stop_gradient', 'tensorboard', 'tensorflow', 'vectorized_map']

I tried running pip uninstall keras-core, confirmed it was no longer available, and reinstalled it using python pip_build.py --install, but I still get the same issue.

Installation instructions on README.md are missing a step or two

I tried cloning the repo and following the instructions on the README.md, i.e.:
python3 pip_build.py --install

but this failed because namex was not installed. So perhaps we're missing a step like:
pip install -r requirements.txt

We may want to also add the git clone step, although that may be obvious.

Lastly, adding a tiny example on the home page would be nice. I made the mistake of importing keras instead of keras_core, it may seem stupid but I suspect this will be a common error, so seeing an example with import keras_core would help. Moreover, it would be nice to indicate whether or not import keras_core as keras is recommended (to port code easily) or frowned upon (because of the risk of conflict with the original keras library).

Perhaps something like this:

import keras_core as keras
import numpy as np

X = np.random.rand(1000, 5)
Y = np.random.rand(1000, 2)

model = keras.Sequential([
    keras.layers.Dense(10, activation="relu"),
    keras.layers.Dense(2),
])
model.compile(loss="mse", optimizer="nadam")
history = model.fit(X, Y)
Y_pred = model.predict(X)

The export KERAS_BACKEND="jax" may confuse Windows users. Perhaps give instructions for them as well.

model.predict(...) fails on a multi-input model only with the jax backend

Running some tests with multi-input models, I came across the following issue:

import keras_core as keras
import numpy as np

class MultiInputModel(keras.Model):
    def call(self, inputs):
        a, b = inputs
        return a + b

model = MultiInputModel()
x1, x2 = np.random.rand(2, 3, 4)
y_pred1 = model((x1, x2)) #works fine
y_pred2 = model.predict((x1, x2)) #!!! ValueError (only with jax backend)

The error is:

ValueError: Exception encountered when calling MultiInputModel.call().
not enough values to unpack (expected 2, got 1)

Arguments received by MultiInputModel.call():
  • inputs=(('jnp.ndarray(shape=(3, 2), dtype=float32)', 'jnp.ndarray(shape=(3, 2), dtype=float32)'),)

Full stack trace:

ValueError                                Traceback (most recent call last)
Cell In[1], line 11
      9 model = MultiInputModel()
     10 x1, x2 = np.random.rand(2, 3, 2)
---> 11 model.predict((x1, x2))

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/backend/jax/trainer.py:560, in JAXTrainer.predict(self, x, batch_size, verbose, steps, callbacks)
    557     for _, data in epoch_iterator.enumerate_epoch(return_type="np"):
    558         # Build model
    559         with backend.StatelessScope():
--> 560             self(data[0])
    561         break
    563 # Container that configures and calls callbacks.

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/utils/traceback_utils.py:122, in filter_traceback.<locals>.error_handler(*args, **kwargs)
    119     filtered_tb = _process_traceback_frames(e.__traceback__)
    120     # To get the full stack trace, call:
    121     # `keras_core.config.disable_traceback_filtering()`
--> 122     raise e.with_traceback(filtered_tb) from None
    123 finally:
    124     del filtered_tb

Cell In[1], line 6, in MultiInputModel.call(self, inputs)
      5 def call(self, inputs):
----> 6     a, b = inputs
      7     return a + b

ValueError: Exception encountered when calling MultiInputModel.call().

not enough values to unpack (expected 2, got 1)

Arguments received by MultiInputModel.call():
  • inputs=(('jnp.ndarray(shape=(3, 2), dtype=float32)', 'jnp.ndarray(shape=(3, 2), dtype=float32)'),)

is `keras_core.backend.clear_session` available?

Looks like the clear_session public API is exported, but its not accessible via keras_core.backend.clear_session.

https://github.com/keras-team/keras-core/blob/main/keras_core/backend/common/global_state.py#L36

Image classification with Vision Transformer

Cannot load a custom model with layer constructed in build() method

Playing with custom models, I ran into the following issue, while loading the custom model (same error on all 3 backends):

import keras_core as keras
import numpy as np

class MyCustomModel(keras.Model):
    def __init__(self, units=30, **kwargs):
        super().__init__(**kwargs)
        self.units = units

    def build(self, batch_input_shape):
        self.hidden1 = keras.layers.Dense(self.units)    

    def call(self, inputs):
        return self.hidden1(inputs)

    def get_config(self):
        return {"units": self.units}

    @classmethod
    def from_config(cls, config):
        return cls(**config)

model = MyCustomModel(10)
model.compile(loss="mse", optimizer="nadam")
X = np.random.rand(100, 5)
Y = np.random.rand(100, 10)
history = model.fit(X, Y)
model.save("my_custom_model.keras")
loaded_model = keras.models.load_model( #!!! IndexError: list index out of range
    "my_custom_model.keras",
    custom_objects={"MyCustomModel": MyCustomModel}
)

Below is the full stacktrace:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[1], line 28
     26 history = model.fit(X, Y)
     27 model.save("my_custom_model.keras")
---> 28 loaded_model = keras.models.load_model(
     29     "my_custom_model.keras",
     30     custom_objects={"MyCustomModel": MyCustomModel}
     31 )

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/saving/saving_api.py:80, in load_model(filepath, custom_objects, compile, safe_mode)
     77         is_keras_zip = True
     79 if is_keras_zip:
---> 80     return saving_lib.load_model(
     81         filepath,
     82         custom_objects=custom_objects,
     83         compile=compile,
     84         safe_mode=safe_mode,
     85     )
     86 else:
     87     raise ValueError(
     88         "The file you are trying to load does not appear to "
     89         "be a valid `.keras` model file. If it is a legacy "
     90         "`.h5` or SavedModel file, do note that these formats "
     91         "are not supported in Keras Core."
     92     )

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/saving/saving_lib.py:184, in load_model(filepath, custom_objects, compile, safe_mode)
    182 # Construct the model from the configuration file in the archive.
    183 with ObjectSharingScope():
--> 184     model = deserialize_keras_object(
    185         config_dict, custom_objects, safe_mode=safe_mode
    186     )
    188 all_filenames = zf.namelist()
    189 if _VARS_FNAME + ".h5" in all_filenames:

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/saving/serialization_lib.py:678, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
    676     compile_config = config.get("compile_config", None)
    677     if compile_config:
--> 678         instance.compile_from_config(compile_config)
    679         instance.compiled = True
    681 if "shared_object_id" in config:

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/trainers/trainer.py:684, in Trainer.compile_from_config(self, config)
    681 self.compile(**config)
    682 if hasattr(self, "optimizer") and self.built:
    683     # Create optimizer variables.
--> 684     self.optimizer.build(self.trainable_variables)

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/optimizers/nadam.py:82, in Nadam.build(self, var_list)
     80 self._momentums = []
     81 self._velocities = []
---> 82 self._u_product = backend.Variable(1.0, dtype=var_list[0].dtype)
     84 for var in var_list:
     85     self._momentums.append(
     86         self.add_variable_from_reference(
     87             reference_variable=var, name="m"
     88         )
     89     )

IndexError: list index out of range

If the Dense layer is created in the constructor instead of the build() method, then everything works fine.

Dropout layer doesn't work with jax backend when using jit + StatelessScope in training mode

The Dropout layer throws an error 'list' object has no attribute 'dtype' when run with jit and StatelessScope enabled. It looks like the seed's state value isn't getting initialized properly.

Here's an MRE:

import keras_core
from keras_core import backend
from keras_core.operations import numpy as knp
from jax import numpy as jnp
import jax

x = knp.array([0.1, 0.2, 0.3])

@jax.jit
def train_step(x):
    with backend.StatelessScope():
        x = keras_core.layers.Dropout(rate=0.1)(x, training=True)
    return x

keras_core.utils.traceback_utils.disable_traceback_filtering()
x = train_step(x)
assert isinstance(x, jnp.ndarray)

Setup numerical test for model correctness verification

We should have some golden number comparison for the major workflow like model.fit/eval/predict.

The baseline can be an application model from keras, with fixed weights eg, imagenet pretrained weights.

[Cleanup] Move the tensor/variable related implementation

Currently all the variable class and tensor related utils are living under keras_core/backend/tensorflow/__init__.py (same for all the backend). We should have a proper file/package for them.

My proposal:

1. Variable should move to a new file backend/tensorflow/variable.py
2. tensor related util move to a new file backend/tensorflow/tensor_util.py
3. Apply this to all the backends.

Image classification with modern MLP models

Image classification via fine-tuning with EfficientNet

Hack week planning

Participants:

• @haifeng-jin
• @nkovela1
• @grasskin
• @hertschuh
• @mattdangerw
• @rchao
• @jbischof
• @divyashreepathihalli
• @ianstenbit
• @qlzh727
• @sampathweb
• @chenmoneygithub

See proposed assignments below.

Some of the assignment buckets may not last the week -- we will reshuffle assignments as we make progress. If we reach a sufficient degree of feature completeness mid-week, we can move on to backwards compatibility testing.

@fchollet

• plot_model / model_to_dot
• explore modernization of plot_model
• stack trace filtering
• add support for legacy h5 weights loading
• JAX rnn function
• FeatureSpace

Layers:

• Wrapper
• Normalization
• Rescaling
• CenterCrop
• Resizing
• RNN / RNNCell
• StackedRNNCells
• SimpleRNN
• LSTM
• GRU
• Lambda
• ConvLSTM1D/2D/3D
• Bidirectional
• TimeDistributed

Wrapped layers:

• Discretization
• TextVectorization
• Hashing
• StringLookup
• IntegerLookup

@chenmoneygithub

• Conv1D/2D/3D
• SeparableConv1D/2D
• DepthwiseConv1D/2D
• Conv1DTranspose/2D/3D

@sampathweb

• Set up code coverage CI test (e.g. codecov)
• Identify missing tests and major TODOs

Callbacks:

• RemoteMonitor
• TerminateOnNaN
• CSVLogger
• ModelCheckpoint
• BackupAndRestore

@qlzh727

• #96
• keras-team/keras#18473

@ianstenbit

• Confusion metrics
• Learning rate schedules (keras_core/optimizers/schedules/)

Callbacks:

• LearningRateScheduler
• ReduceLROnPlateau

@divyashreepathihalli

Layers:

• Elu
• ReLU
• LeakyReLU
• PReLU
• Softmax

Wrapped layers:

• RandomCrop
• RandomFlip
• RandomTranslation
• RandomRotation
• RandomZoom

@jbischof

• Implement keras_core/backend/pytorch/numpy.py

@rchao

Layers:

• UpSampling1D/2D/3D

@mattdangerw

Layers:

• Attention
• AdditiveAttention
• MultiHeadAttention

@hertschuh

Layers:

• Flatten
• Reshape
• RepeatVector
• Permute
• Cropping1D/2D/3D
• ZeroPadding1D/2D/3D

@grasskin

Metrics

• R2Score
• F1Score
• FBetaScore
• IoU metrics

Callbacks:

• TensorBoard

@nkovela1

Layers:

• UnitNormalization
• LayerNormalization
• GroupNormalization
• SpectralNormalization
• CategoryEncoding
• RandomContrast
• RandomBrightness

@haifeng-jin

• Implement step fusing in TF Trainer and JAX Trainer

Image captioning

`model.summary()` table formatting issue

The current table produced by model.summary() needs reformatting. Here is how it looks like right now:

PS: Can we try to use Rich tables?

Implement `rnn` in torch backend

For reference, contributors can look at:

• tensorflow rnn: link
• Jax rnn: link

The rnn part is necessary, gru and lstm could be a followup.

To test the implementation works, please use the layer tests, e.g.,

pytest keras_core/layers/rnn_test.py

Tracking issue for all layers

Core layers -> @fchollet

• Dense
• Dropout
• Activation
• EinsumDense
• Embedding
• Identity
• Masking

Regularization layers -> @fchollet

• ActivityRegularization
• Dropout
• GaussianDropout
• GaussianNoise
• SpatialDropout1D/2D/3D

Attention -> @mattdangerw

• Attention
• AdditiveAttention
• MultiHeadAttention

Activations

• Elu
• ReLU
• LeakyReLU
• PReLU
• Softmax

Merging -> @AakashKumarNain

• Add
• Subtract
• Average
• Concatenate
• Maximum
• Minumum
• Dot
• Multiply

Reshaping -> @hertschuh

• Flatten
• Reshape
• RepeatVector
• Permute
• Cropping1D/2D/3D
• ZeroPadding1D/2D/3D
• UpSampling1D/2D/3D

Convolutional -> @chenmoneygithub

• Conv1D/2D/3D
• SeparableConv1D/2D
• DepthwiseConv1D/2D
• Conv1DTranspose/2D/3D

Pooling -> @chenmoneygithub

• AveragePooling1D/2D/3D
• MaxPooling1D/2D/3D
• GlobalAveragePooling1D/2D/3D
• GlobalMaxPooling1D/2D/3D

Normalization

• BatchNormalization
• UnitNormalization
• LayerNormalization
• GroupNormalization
• SpectralNormalization

RNN -> @fchollet + @LukeWood

• RNN / RNNCell
• SimpleRNN
• LSTM
• GRU
• ConvLSTM1D/2D/3D
• TimeDistributed
• Bidirectional

Preprocessing layers -> @fchollet

TBD

Image classification with ConvMixer

Guidelines regarding code formatting

Can we put up simple instructions on how to configure the dev environment to properly align with the code formatting expected? I have both flake8 and black installed in VSCode but somehow the settings aren't aligned even if I point it to the correct settings file. A simple workaround is to run both flake8 and black separately but looking for better guidelines

Metrics doesn't take string names in `model.compile`

import numpy as np
from tensorflow import data as tf_data
import keras_core


x = np.random.randn(8, 2).astype("float32")
y = np.random.randint(0, 1, (8, 1)).astype("uint8")
dataset = tf_data.Dataset.from_tensor_slices((x, y)).batch(2)

model = keras_core.Sequential([
    keras_core.layers.Input(shape=(2,)),
    keras_core.layers.Dense(1, activation="sigmoid")
    ])
model.summary()

model.compile(optimizer="sgd", loss="mse", 
              metrics=["categorical_accuracy"] # Fails
              # metrics=[keras_core.metrics.CategoricalAccuracy()]  # Works fine when giving class names
)
model.fit(dataset, epochs=2)

[keras_core/metrics/__init__.py](https://localhost:8080/#) in get(identifier)
    188         return obj
    189     else:
--> 190         raise ValueError(f"Could not interpret metric identifier: {identifier}")

ValueError: Could not interpret metric identifier: categorical_accuracy

When specifying the metric via class name keras_core.metrics.CategoricalAccuracy(), it works fine. This is true for other metrics as well.

BatchNorm layer appears broken in multi-backend Keras

classic MNIST training:

• Keras multi-backend with BN on JAX. Final accuracy 19%!!!
• Keras multi-backend w/o BN on JAX. Final accuracy: 99%
• Keras multi-backend with BN on TF. Final accuracy: 14%!!!
• Keras multi-backend w/o BN on TF. Final accuracy: 99%
• Keras baseline (regular tf.keras on TF, with BN). Final accuracy: 99%

The common factors triggering the issue seem to be: using the BatchNorm layer, on Keras multi-backend. The choice of backend does not seem to change the behavior.

Keras Applications progress tracker

• VGG*: @fchollet
• Xception: @fchollet
• ResNet*: @AakashKumarNain
• ResNet*V2: @AakashKumarNain
• InceptionV3: @fchollet
• InceptionResNetV2: @fchollet
• MobileNet: @fchollet
• MobileNetV2: @fchollet
• DenseNet*: @fchollet
• NASNet*: @fchollet
• EfficientNet*: @fchollet
• ConvNeXt*: @fchollet

Deadlock when running `pip_build.py`

On a CPU only python environment, I have hit a deadlock issue running python pip_build.py --install.

Tracked this down as far as the torch import, which is never resolving when in the tf backend.

Re-running the script with the torch backend selected fixes the deadlock fully.

BatchNorm is slow

Especially when jit_compile=False, Keras Core BN is significantly slower than tf.keras BN. Need to fix.

Torch numpy ops tracker

Most Torch numpy ops have been successfully completed. However, we need full implementations of these ops to be functionally equivalent to the numpy ops.

• (@nkovela1) torch.full and torch.full_like --> implement support for an array being passed in fill_value arg, maybe use np.tile?
• (@nkovela1) torch.linspace and torch.logspace --> implement support for arrays being passed in start and stop args
• (@nkovela1 ) torch.take --> resolve the incompatibility in dimensions between the result of np.take and torch.take
• (@nkovela1) torch.pad --> implement padding with non-constant padding for the symmetric and reflect modes
• (@nkovela1) torch.split --> resolve the arg divergence between np.split and torch.split that produces different results
• (@nkovela1) torch.trace --> implement torch.trace support for arguments offset, axis1, axis2
• (@nkovela1) torch.eye --> implement support for k diagonal arg that exists in np.eye but not torch.eye
• (@nkovela1) torch.tri --> create an implementation for scratch based on the logic of np.tri, as the torch.tri API does not exist

To start implementing these:

1. Add your name to the task.
2. Remove the corresponding pytest skipif decorator in numpy_test.py. This will let you see the error to debug.
3. After implementing, if applicable, merge the test with the part of the numpy test without torch.
4. Delete any corresponding NotImplementedError, if it exists.

Thank you and happy hacking!

TODO: binary crossentropy stability in JAX and torch

We implement BCE as follows:

def binary_crossentropy(target, output, from_logits=False):
    target = jnp.array(target)
    output = jnp.array(output)

    if target.shape != output.shape:
        raise ValueError(
            "Arguments `target` and `output` must have the same shape. "
            "Received: "
            f"target.shape={target.shape}, output.shape={output.shape}"
        )

    if from_logits:
        output = jnn.sigmoid(output)

    output = jnp.clip(output, epsilon(), 1.0 - epsilon())
    bce = target * jnp.log(output)
    bce += (1.0 - target) * jnp.log(1.0 - output)
    return -bce

An issue here is the fact that, provided logits, we take the sigmoid, and then the log. This can lead to numerical instability in fp16.

Can we come up with an implementation that goes through log_sigmoid or something?

`python demo_functional.py` Error

Running python demo_functional.py fails with error -

    File "keras-core/keras_core/losses/loss.py", line 69, in squeeze_to_same_rank  *
        x2_rank = len(x2.shape)

    ValueError: Cannot take the length of shape with unknown rank.

Example `nlp/lstm_seq2seq.py` doesn't train with JAX backend

The model trains fine with TF. With JAX, the code runs but trains rather poorly. Need to find out root cause.

In addition, in this example, a model trained with TF cannot be reloaded with JAX.

Trainer should work for torch tensors

          Should work for torch tensors as well -- we should start adding data adapter tests targeting torch tensors (in a new PR)

Originally posted by @fchollet in #247 (comment)

keras.layers.CenterCrop raises AttributeError when passed a list of images

With tf.keras, the following code works, but not in Keras-Core:

import keras_core as keras
import numpy as np

images = [
    np.random.rand(100, 100, 3),
    np.random.rand(100, 100, 3),
]
keras.layers.CenterCrop(height=60, width=50)(images)  #!!! AttributeError

Full stacktrace below:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[21], line 8
      2 import numpy as np
      4 images = [
      5     np.random.rand(100, 100, 3),
      6     np.random.rand(100, 100, 3),
      7 ]
----> 8 keras.layers.CenterCrop(height=60, width=50)(images)

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/layers/preprocessing/tf_data_layer.py:36, in TFDataLayer.__call__(self, inputs, **kwargs)
     34             self._convert_input_args = True
     35     return outputs
---> 36 return super().__call__(inputs, **kwargs)

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/utils/traceback_utils.py:122, in filter_traceback.<locals>.error_handler(*args, **kwargs)
    119     filtered_tb = _process_traceback_frames(e.__traceback__)
    120     # To get the full stack trace, call:
    121     # `keras_core.config.disable_traceback_filtering()`
--> 122     raise e.with_traceback(filtered_tb) from None
    123 finally:
    124     del filtered_tb

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/layers/preprocessing/center_crop.py:59, in CenterCrop.call(self, inputs)
     57     init_width = inputs.shape[-1]
     58 else:
---> 59     init_height = inputs.shape[-3]
     60     init_width = inputs.shape[-2]
     62 if init_height is None or init_width is None:
     63     # Dynamic size case. TODO.

AttributeError: Exception encountered when calling CenterCrop.call().

'list' object has no attribute 'shape'

Arguments received by CenterCrop.call():
  • inputs=['jnp.ndarray(shape=(100, 100, 3), dtype=float32)', 'jnp.ndarray(shape=(100, 100, 3), dtype=float32)']

A simple workaround is to stack the images:

keras.layers.CenterCrop(height=60, width=50)(np.stack(images))

Not sure this can be considered a bug, but it's one of those little differences that may porting code from Keras 2.x to 3.0 a bit harder.

`acc` or `accuracy` string in `metrics` error

import numpy as np
from keras_core import layers
from keras_core import metrics
from keras_core.models import Sequential

    def make_model():
        model = Sequential(
            [
                layers.Dense(4, activation="relu"),
                layers.Dense(2, activation="softmax"),
            ]
        )
        model.compile(
            loss="categorical_crossentropy",
            optimizer="sgd",
            # metrics=[metrics.Accuracy()]  # Works
            # metrics=["acc"]  # Fails
            metrics=["accuracy"]  # Fails
        )
        return model

  x_train = np.random.random((10, 3))
  y_train = np.random.random((10, 2))
  x_test = np.random.random((10, 3))
  y_test = np.random.random((10, 2))

  model = make_model()
  model.fit(
      x_train,
      y_train,
      batch_size=4,
      validation_data=(x_test, y_test),
      epochs=1,
  )

When using acc or accuracy with categorical_crossentropy it fails with error that categorical_accuracy not found. But works fine when using Accuracy() metric -

    File "keras-core/keras_core/backend/tensorflow/trainer.py", line 107, in one_step_on_iterator  *
        outputs = self.distribute_strategy.run(
    File "keras-core/keras_core/backend/tensorflow/trainer.py", line 97, in one_step_on_data  *
        return self.train_step(data)
    File "keras-core/keras_core/backend/tensorflow/trainer.py", line 68, in train_step  *
        return self.compute_metrics(x, y, y_pred, sample_weight=sample_weight)
    File "keras-core/keras_core/trainers/trainer.py", line 201, in compute_metrics  *
        self._compile_metrics.update_state(y, y_pred, sample_weight)
    File "keras-core/keras_core/trainers/compile_utils.py", line 301, in update_state  *
        self.build(y_true, y_pred)
    File "keras-core/keras_core/trainers/compile_utils.py", line 170, in build  *
        self._flat_metrics = self._build_metrics_set(
    File "keras-core/keras_core/trainers/compile_utils.py", line 209, in _build_metrics_set  *
        flat_metrics.append(
    File "/keras-core/keras_core/trainers/compile_utils.py", line 74, in get_metric  *
        metric_obj = metrics_module.categorical_accuracy

    AttributeError: module 'keras_core.metrics' has no attribute 'categorical_accuracy'

ModelCheckpoint callback won't take directories for path

Issue with https://keras.io/examples/vision/object_detection_using_vision_transformer/ conversion.

Model checkpoint filepath used to be able to take a directory like this:

checkpoint_filepath = "logs/"
checkpoint_callback = keras.callbacks.ModelCheckpoint(
    checkpoint_filepath,
    monitor="val_loss",
    save_best_only=True,
    save_weights_only=True,
)

Currently it requires a filepath that must end in .weights.h5.

Possible bug in MSE

Shouldn't the mean_squared_error return the mean values? The loss function currently returns (y_true - y_pred) ** 2 but shouldn't it be ops.mean((y_true - y_pred) ** 2) instead? An example to illustrate the same

y_true = np.array([[0., 1.], [0., 0.]])
y_pred = np.array([[1., 1.], [1., 0.]])

loss = MeanSquaredError()
loss(y_true, y_pred) # Returns 1.0 but should return 0.5

`reduce_*` operation in ops

Are we going to implement reduce_ operations in ops? This includes the following:

1. all
2. any
3. min
4. max
5. mean
6. sum
7. variance
8. std

Learning rate schedules do not work with JAX backend

repro colab: https://colab.corp.google.com/drive/15AFfhJIG9p3kgxjZQBxpns7wmwsOMzDi?resourcekey=0-ZgBZ6c0U2RjfstKTmOrO-g&usp=sharing

The Colab displays the learning rate from the optimizer through a custom metric.

Repro steps:

• select JAX backend in first cell, run notebook. Reported LR is:

Epoch 1/5 val_lr_metric: 0.0100
Epoch 2/5 val_lr_metric: 0.0100
Epoch 3/5 val_lr_metric: 0.0100

• restart from scratch (use "disconnect and delete runtime"), then select tensorflow backend in the first cell. Reported LR is:

Epoch 1/5 val_lr_metric: 0.0060
Epoch 2/5 val_lr_metric: 0.0036
Epoch 3/5 val_lr_metric: 0.0022

Error when training a multi-output model: AttributeError: 'dict' object has no attribute 'dtype'

I've built a simple dual-output model using the functional API, but it's failing at training time with an AttributeError:

import numpy as np
import keras_core as keras

X = np.random.rand(1000, 5)
Y1 = np.random.rand(1000, 2)
Y2 = np.random.rand(1000, 3)

input_ = keras.layers.Input(shape=X.shape[1:])
dense1 = keras.layers.Dense(10, activation="relu")(input_)
output1 = keras.layers.Dense(Y1.shape[1])(dense1)
output2 = keras.layers.Dense(Y2.shape[1])(dense1)
model = keras.Model(inputs=(input_,), outputs={"out1": output1, "out2": output2})
model.compile(loss="mse", optimizer="nadam")
history = model.fit(X, {"out1": Y1, "out2": Y2})   #!!! AttributeError: 'dict' object has no attribute 'dtype'
Y_pred = model.predict(X)

This fails with all 3 backends. I'm running the latest code (just checked out and built).
Below is the full stacktrace.

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], line 14
     12 model = keras.Model(inputs=[input_], outputs=[output1, output2])
     13 model.compile(loss="mse", optimizer="nadam")
---> 14 model.fit(X, (Y1, Y2))
     15 model.predict(X)

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/utils/traceback_utils.py:122, in filter_traceback.<locals>.error_handler(*args, **kwargs)
    119     filtered_tb = _process_traceback_frames(e.__traceback__)
    120     # To get the full stack trace, call:
    121     # `keras_core.config.disable_traceback_filtering()`
--> 122     raise e.with_traceback(filtered_tb) from None
    123 finally:
    124     del filtered_tb

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/losses/loss.py:41, in Loss.__call__.<locals>.<lambda>(x)
     36 dtype = backend.floatx()
     37 y_pred = nest.map_structure(
     38     lambda x: ops.convert_to_tensor(x, dtype=dtype), y_pred
     39 )
     40 y_true = nest.map_structure(
---> 41     lambda x: ops.convert_to_tensor(x, dtype=y_pred.dtype), y_true
     42 )
     44 losses = self.call(y_true, y_pred)
     45 out_mask = getattr(losses, "_keras_mask", None)

AttributeError: 'ListWrapper' object has no attribute 'dtype'

tf.distribute support in TF backend

`demo_custom_tf_workflow.py` fails on GPU

I get following error when running on GPU -

2023-05-03 11:14:08.026060: W tensorflow/core/framework/op_kernel.cc:1830] 
OP_REQUIRES failed at xla_ops.cc:347 : INVALID_ARGUMENT: 
Trying to access resource Resource-0-at-0x1ef56e60 (defined @ keras-core/keras_core/backend/tensorflow/__init__.py:22) 
located in device /job:localhost/replica:0/task:0/device:CPU:0 from device /job:localhost/replica:0/task:0/device:GPU:0
 Cf. https://www.tensorflow.org/xla/known_issues#tfvariable_on_a_different_device
Traceback (most recent call last):
  File "keras-core/demo_custom_tf_workflow.py", line 86, in <module>
    loss = train_step(data)
  File "python3.9/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "python3.9/site-packages/tensorflow/python/eager/execute.py", line 52, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Trying to access resource Resource-0-at-0x1ef56e60 (defined @ keras-core/keras_core/backend/tensorflow/__init__.py:22) located in device /job:localhost/replica:0/task:0/device:CPU:0 from device /job:localhost/replica:0/task:0/device:GPU:0
 Cf. https://www.tensorflow.org/xla/known_issues#tfvariable_on_a_different_device [Op:__inference_train_step_651]

Believe the issue os accessing CPU resource from GPU. This error doesn't come up when running the JAX Workflow on GPU.

Same error when running demo_custom_layer_backend_agnostic.py with TensorFlow backend.

Not sure if this is specific to my local setting. Can someone confirm that TF workflow runs fine on GPU? On CPU / Mac, they run fine.

Numerical utils not available under `keras_core.utils` namespace

keras_core.utils.normalize and keras_core.utils.to_categorical not available, only keras_core.utils.numerical_utils. Solution is probably to add to keras_core/utils/__init__.py

from keras_core.utils.numerical_utils import normalize
from keras_core.utils.numerical_utils import to_categorical

But this gives issues with circular import.

Custom multi-input model with layer built in the constructor fails with ValueError

I tried building a custom model with two inputs, but it throws an exception when I call it:

import keras_core as keras
import numpy as np

class MultiInputModel(keras.Model):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.dense1 = keras.layers.Dense(10)
        
    def call(self, inputs):
        a, b = inputs
        r = self.dense1(a)
        return keras.layers.concatenate([r, b])

model = MultiInputModel()
X1 = np.random.rand(3, 3)
X2 = np.random.rand(3, 2)
model((X1, X2))  #!!! ValueError

The exception is:

ValueError: Cannot convert '((3, 3), (3, 2))' to a shape. Found invalid entry '(3, 3)'.
Dynamic shapes (shapes with `None` entries) are not allowed with the tensorflow,
except for the batch size (axis 0).

Full stack trace:

---

ValueError Traceback (most recent call last)
Cell In[1], line 17
15 X1 = np.random.rand(3, 3)
16 X2 = np.random.rand(3, 2)
---> 17 model((X1, X2)) #!!! ValueError

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/utils/traceback_utils.py:122, in filter_traceback..error_handler(*args, **kwargs)
119 filtered_tb = _process_traceback_frames(e.traceback)
120 # To get the full stack trace, call:
121 # keras_core.config.disable_traceback_filtering()
--> 122 raise e.with_traceback(filtered_tb) from None
123 finally:
124 del filtered_tb

File ~/opt/miniconda3/envs/kerascore/lib/python3.8/site-packages/keras_core/src/backend/common/variables.py:446, in standardize_shape(shape, allow_dynamic_batch_size, allow_all_dynamic)
439 else:
440 msg += (
441 "Dynamic shapes (shapes with None entries) "
442 f"are not allowed with the {config.backend()}. "
443 "All dimensions should be positive integers, "
444 "including the batch size (axis 0)."
445 )
--> 446 raise ValueError(msg)
447 if e < 0:
448 raise ValueError(
449 f"Cannot convert '{shape}' to a shape. "
450 "Negative dimensions are not allowed."
451 )

ValueError: Cannot convert '((3, 3), (3, 2))' to a shape. Found invalid entry '(3, 3)'. Dynamic shapes (shapes with None entries) are not allowed with the tensorflow, except for the batch size (axis 0).

Side note: the error message should say "the tensorflow backend" instead of just "the tensorflow".

I tested with the other backends, and I get the same error (except for PyTorch it says Cannot convert '(torch.Size([3, 3]), torch.Size([3, 2]))' to a shape. Found invalid entry 'torch.Size([3, 3])'.)

I also tested with tf.keras, and it works fine.

Lastly, I tried building the Dense layer inside the build() method instead of the constructor, and it actually fixes the issue:

import keras_core as keras
import numpy as np

class MultiInputModel(keras.Model):
    def build(self, batch_input_shape):
        self.dense1 = keras.layers.Dense(10)
        self.built = True

    def call(self, inputs):
        a, b = inputs
        r = self.dense1(a)
        return keras.layers.concatenate([r, b])

model = MultiInputModel()
X1 = np.random.rand(3, 3)
X2 = np.random.rand(3, 2)
model((X1, X2))  #works fine now

So it looks like the problem only occurs when the model has multiple inputs and it creates a layer in the constructor.

Splitting metrics & losses implementations

At this time it seems that @ariG23498, @grasskin, and @AakashKumarNain are working on losses or metrics, which can lead to some collisions. Let's split the work:

@ariG23498:

• Accuracy metrics

@grasskin:

• Regression metrics
• Regression losses

@AakashKumarNain:

• Hinge losses
• Hinge metrics

This includes both plain functions and their wrapper classes. All objects should have unit tests and should have a complete docstring (for the most part you can just copy the Keras docstring and remove the TF references, etc.).

For loss functions that are redundant with metrics, the plain function should live in losses/ and be imported in metrics/. The class-based wrapper should exist both as a Loss in losses/ and as a Metric in metrics/.

Programatically set the docstrings on the Trainer subclasses

          Something we should do soon is make sure the base `Trainer` class has docstrings everywhere, that the Trainer subclasses don't have docstrings, then we programatically set the docstrings on the subclasses (using the ones from the base class)

Originally posted by @fchollet in #328 (comment)

Constructing a torch optimizer from a Keras model fails

I'm trying to run the demo_custom_torch_workflow.py and gpu_demo_torch.py examples, and they both fail because there's no way to get an iterable containing the parameters to optimize from a Keras model.

Here is the relevant portion of code that fails becausemodel doesn't have a parameters() function.

model = keras_core.Sequential(
        [
            layers.Input(shape=(28, 28, 1)),
            layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
            layers.MaxPooling2D(pool_size=(2, 2)),
            layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
            layers.MaxPooling2D(pool_size=(2, 2)),
            layers.Flatten(),
            layers.Dropout(0.5),
            layers.Dense(num_classes),
        ]
)

optimizer = optim.Adam(model.parameters(), lr=learning_rate)

These examples also wrap the Keras model into an nn.Module:

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.model = keras_core.Sequential(
            [
                layers.Input(shape=(28, 28, 1)),
                layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
                layers.MaxPooling2D(pool_size=(2, 2)),
                layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
                layers.MaxPooling2D(pool_size=(2, 2)),
                layers.Flatten(),
                layers.Dropout(0.5),
                layers.Dense(num_classes),
            ]
        )

    def forward(self, x):
        return self.model(x)

torch_module = MyModel()
optimizer = optim.Adam(torch_module.parameters(), lr=learning_rate)

But this also fails because the list of parameters comes back empty.

How do I get past this?

Batch dot operation

I have implemented the batch_dot(x1, x2, axes) operation but I am not sure what's the right module for it. Should we include this in ops.numpy?

PS: This is required for implementing the Dot layer