[RFC] Code structure refactor

[d2l-ai]

[RFC] Code structure refactor

[mli]

@astonzhang @smolix @AnirudhDagar @archersama

[astonzhang]

Thanks. The idea is very good and the workaround of breaking a class def into multiple cells may address our earlier concerns. One thing we need to think carefully is the design of the Trainer base class, such as its fit method: D2L has many different use cases such as sequence to sequence and object detection.

[AnirudhDagar]

As @astonzhang suggested, I have the same concerns with the design of the Trainer base class fit method which needs to handle various scenarios.

Other than that, type hints will surely be a great addition for type checking using mypy and it also adds to the readability of the code, making it more understandable. We should definitely add it to the key APIs.
Also fstrings in python3 again help the print statements be more readable and concise, so there is no reason not to go ahead with refactoring these.

The idea of keeping all the framework implementations in a single library looks good, but this will make things a bit more complex in terms of mantainence of the single framework with multiple dependencies. This is a design choice which we should discuss in detail.

Yes, the API redesign, will definitely look good for the mini framework-agnostic library that d2l can become. But, we do not want to over-design things and somehow end up with complex APIs. This may also hinder the understanding of DL concepts along with code for people who are just starting and are very new to the world of Deep Learning.

[mli]

we can have multiple version of Trainer, for example, the basic CPU trainer BasicTrainer, the multi-gpu trainer Trainer, and others. The idea is to reuse codes.

We don't need to have a single trainer to support every framework, but make them as similar as possible. I'm thinking about have all codes in the d2l module, then

from d2l import mxnet as d2l  # for mxnet implementations
from d2l import torch as d2l # for pytorch implementations
from d2l import tensorflow as d2l  # for tensorflow implementations