Tensorboard and CSV Logger

[biomedsciai]

Tensorboard and CSV Logger

[shatz01]

So combining curves in tensorboard is possible, albeit inelegant. I found some SO q's that referenced doing this like this one: https://stackoverflow.com/questions/48951136/plot-multiple-graphs-in-one-plot-using-tensorboard

So I was able to acheive this in fuse by making a function like this in fuse/dl/lightning/pl_funcs.py:

# Tensorboard ONLY
def tensorboard_epoch_end_compute_and_log_losses_combined(pl: pl.LightningModule, mode: str, batch_losses: Sequence[Dict]) -> None:
    """
    On epoch end average out the batch losses and log the averaged losses
    :param pl: LightiningModule. Used for logging.
    :param mode: prefix to add to each loss name (when logging), typically validation/train/test
    :param batch_losses: list of batch_dict["losses"] as added by 'epoch_losses'
    :return: None
    """
    keys = batch_losses[0].keys()
    for key in keys:
        losses = []
        for elem in batch_losses:
            if isinstance(elem[key], torch.Tensor):
                losses.extend(elem[key].detach().cpu().tolist())
            else:
                losses.append(elem[key])
        loss = mean(losses)
        pl.log(f"combined.losses.{key}", {mode: loss}, on_epoch=True)

And then you can use it in the model training_epoch_end the same way we do other logging:

# Log Combined (tensorboard ONLY)
fuse_pl.tensorboard_epoch_end_compute_and_log_losses_combined(self, "train", [e["losses"] for e in step_outputs])

Which results in the desired behavior, but also makes a seperate entry for every additional metric plotted by this method (resulting in multiple "runs" for something that is actually just 1 run):

[shatz01]

Also, it seems like more advanced logging frameworks such as wandb already support mixing plots elegantly, so i suggest we dont do this.

[shatz01]

Regarding CSV logging, pytorch lightning CSVLogger seems totally compatible with fuse's implementation. You can just import it, make its logging dir the same as that of the fuse_logger, and pass it to lightning trainer:

from pytorch_lightning.loggers import CSVLogger
pl_logger_csv = CSVLogger(paths["model_dir"], name="my_model4")

pl_trainer = pl.Trainer(
        default_root_dir=paths["model_dir"],
        max_epochs=train_params["trainer.num_epochs"],
        accelerator=train_params["trainer.accelerator"],
        strategy=train_params["trainer.strategy"],
        devices=train_params["trainer.num_devices"],
        auto_select_gpus=True,
        # logger=[pl_logger_tensorboard, pl_logger_csv],
    )

[shatz01]

Since there are no changes necessary for this issue ill close it :)

[shatz01]

Actually Ill reopen this again and add CSVLogger to a few examples before closing 😅

[shatz01]

added them in #204