Need to re-code all with determined trial class ?

[determined-ai]

Need to re-code all with determined trial class ?

[arita37]

Thanks for answer and releasing your tools. Think after some re-design and addon compatibility with pytorch and al. Tool would be very handy to use. Honestly, re-writing an existing codebase is quite some efforts, having a good interface and usage makes the big difference (!)

…

On May 30, 2020, at 4:50, Shiyuan ***@***.***> wrote:  @arita37 Right now, we don't have Native API with Pytorch support. For Pytorch, the answer to your question is yes we need to re-code the code with the trial class. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[shiyuann]

Hi,

Thanks for trying out this project!

There is an experimental API called Native API that has idiomatic design as the native frameworks, like Tensorflow Keras and Estimator. It requires much less effort to re-code your current model than Trial API. Please check out this tutorial for using it with Tensorflow Keras (https://docs.determined.ai/latest/tutorials/native-tf-keras/index.html). Full documentation is on its way in the next release!

Please join our slack community for any questions or support: https://join.slack.com/t/determined-community/shared_invite/zt-cnj7802v-KcVbaUrIzQOwmkmY7gP0Ew

[Mistobaan]

Exactly my same comment of @arita37 The moment I saw

from determined.pytorch import DataLoader, PyTorchTrial

or

from determined.keras import TFKerasTrial, TFKerasTrialContext

class FashionMNISTTrial(TFKerasTrial):

I closed the tutorial
the new approach is definitely more interesting

from determined.experimental.keras import init
# When running from this code from a notebook, add a `command` argument to
# init() specifying the notebook file name.
context = init(config, mode=experimental.Mode.CLUSTER, context_dir=".")
...
model = context.wrap_model(model)

which I would see it evolved into a single decorator line

import determined as dd

@dd.config
def build_model(): 
    return tf.keras.models.Sequential(
    [
        tf.keras.layers.Flatten(input_shape=(28, 28)),
        tf.keras.layers.Dense(128, activation="relu"),
        tf.keras.layers.Dropout(0.2),
        tf.keras.layers.Dense(10, activation="softmax"),
    ]

all the configs and other details can be configured at run time

dd.parse_args

which clearly puts almost zero efforts for the user to try out the library.

Worst case scenario the software does the same thing as before
best case scenario it does much more.
Just my 2cents :)

[arita37]

Thanks. But, what about pytorch support ?

…

On May 1, 2020, at 1:58, Shiyuan ***@***.***> wrote:  Hi, Thanks for trying out this project! There is an experimental API called Native API that has idiomatic design as the native frameworks, like Tensorflow Keras and Estimator. It requires much less effort to re-code your current model than Trial API. Please check out this tutorial for using it with Tensorflow Keras (https://docs.determined.ai/latest/tutorials/native-tf-keras/index.html). Full documentation is on its way in the next release! Please join our slack community for any questions or support: https://join.slack.com/t/determined-community/shared_invite/zt-cnj7802v-KcVbaUrIzQOwmkmY7gP0Ew — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[arita37]

Few more comments: Believe you guys should follow the steps of Horovod or others , which aims same thing. (Hyper params less). Horovod works on Pytorch CPU and GPU. It seems Determined miss this part too. Adding layer of new code or new classes (ie Trial...) is the opposite of what you are claiming : Simplify DL compute. “Native API” looks better, however, it seems still at early stage, a bit poorly designed... (ie similar to TF 1.* heavy overload design). However, this is still a good step.

…

On May 1, 2020, at 9:54, Fabrizio Milo ***@***.***> wrote:  Exactly my same comment of @arita37 The moment I saw from determined.pytorch import DataLoader, PyTorchTrial or from determined.keras import TFKerasTrial, TFKerasTrialContext class FashionMNISTTrial(TFKerasTrial): I closed the tutorial the new approach is definitely more interesting from determined.experimental.keras import init # When running from this code from a notebook, add a `command` argument to # init() specifying the notebook file name. context = init(config, mode=experimental.Mode.CLUSTER, context_dir=".") ... model = context.wrap_model(model) which I would see it evolved into a single decorator line import determined as dd @dd.config def build_model(): return tf.keras.models.Sequential( [ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(128, activation="relu"), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10, activation="softmax"), ] all the configs and other details can be configured at run time dd.parse_args which clearly puts almost zero efforts for the user to try out the library. Worst case scenario the software does the same thing as before best case scenario it does much more. Just my 2cents :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[Mistobaan]

You could simply check if the object is a subclass of one or the other project:

[shiyuann]

@arita37 Right now, we don't have Native API with Pytorch support. For Pytorch, the answer to your question is yes we need to re-code the code with the trial class.

[shiyuann]

@Mistobaan Thanks for the idea of the decorator and the feedback on the new API. The decorator approach definitely looks more flexible. The reason why we don't support decorator is that given how the current system loads user code we want to ensure the model is built after calling init. That's why we only support .wrap_model for the time being.