rlworkgroup/dowel

A little logger for machine learning research

dowel's Issues

Logging Numpy arrays, Torch Tensors and Tensorflow Tensors

Hi,

Thank you for this nice a simple tool for logging machine learning research. I often encounter situations where I would like to save multi-dimensional Numpy arrays. For example, the observation at each time-step in a reinforcement learning experiment.

It would be nice to have an output logger that supports Numpy arrays, Pytorch Tensors and Tensorflow Tensors.

I have written a simple output logger, NpzOutput, that writes Numpy arrays to a .npz file using Numpys savez functions. It is not optimal (no incremental saving), but thought I share it in case somebody is interested.

Fix DeprecationWarnings in the Python 3.7 build

e.g. https://travis-ci.com/rlworkgroup/dowel/jobs/200192982#L460

Mention SSH setup in CONTRIBUTING.md

When I tried to run the following commands from the "Git recipes" in CONTRIBUTING.md, I got error messages:

git remote add rlworkgroup [email protected]:rlworkgroup/dowel.git

git reset --hard master rlworkgroup/master

However, the following would work:

git remote add rlworkgroup https://github.com/rlworkgroup/dowel.git

git checkout master
git fetch rlworkgroup
git reset --hard rlworkgroup/master

Should CONTRIBUTING.md be updated?

Make the logger multiprocessing-aware

It would be nice for the logger to show when a message comes from a worker process versus the main thread.

Fix module documentation generation

Link https://dowel.readthedocs.io/en/latest/py-modindex.html is broken.

Switch test runner to pytest

pytest tends to generate better error messages and has a more modern design.

Make matplotlib and optional extra

This is quite heavy and not everyone will want it

Fix coverage reporting

Coverage reporting to CodeCov is not actually working in the CI. This is probably a misconfiguration.

See https://travis-ci.com/rlworkgroup/dowel/builds/111406917#L744 for an example log of the error.

Replace logger.tabular with Logger.log_kv

Logger.TabularInput is needlessly cumbersome for the most common logging use case, and forces LogOutputs to replicate parts of the logger's dispatch and accounting logic internally. It would be simpler and more consistent just to provide a key-based API.

This would also open up the logger to features like filtering output handlers based on key regexs.

Use a package-global warn_once

...rather than implementing one for each object

Replace utils.mkdir_p with os.path or pathlib

Support for logging levels in logger

e.g. some log entries (including log_tabular) are level DEBUG, some are level INFO, etc.

GrafanaOutput for the logger

Besides visdom (#280), another monitoring solution that could be more useful than TensorBoard would be Grafana.
It's also used by OpenAI for their DOTA project:
https://blog.openai.com/openai-five/

Replace utils.colorize with colorama

Replace csvkit with tablib?

http://docs.python-tablib.org/en/master/

Simple LogOutput decorator

Provide a decorator for quickly adding a LogOutput by decorating a callable, e.g.

@logger.output(str, int)
def my_fun_handler(data, prefix=None):
    print('{} | datum: {}'.format(prefix, datum))

Robust handling of inconsistent TabularInput keys

Introduction

Dowel is a tool that the garage Team uses for logging results from our various Reinforcement learning experiments.

Dowel can be used to log different types of data such as floats or strings. The logs can be logged to stdout (the console), CSV files, and Tensorboard.

You can check out an example of how Dowel is used here. In fact, almost all parts of the Dowel API are used in this example.

The problem

After statistics such as loss have been logged, and a call to logger.dump_all() is made for the first time, new tabular data can’t be written to a CSV output. This is because currently data cannot be inconsistently logged to CSV, meaning that on every single call to dump_all, the same logger keys must appear. Data that is inconsistently logged will not appear in the CSV output. This is a design flaw that we have been able to work around but affects our workflows.

Your goal is to solve the problem as well as introduce tests into our testing framework in order to verify your solution.

Some General Instructions

1. Fork Dowel and install all necessary dependencies.
2. Take a look at this toy example which when run exposes the bug and the accompanying issue mentioned above.
3. When you have finished writing your solution and tests, upload a PR onto your fork, not onto the upstream repository.
4. When you are done email us back with the link to your pull request.
5. Follow the rules of the contributing.md.

If you have any questions, open an issue in your fork, and tag @avnishn and @haydenshively. Our preferred mode of communication on any questions that you have is through github issues and pull requests, as this is how the Garage team communicates generally. For this reason, we won’t respond to any direct emails with regards to help with your project. We will however respond to any other questions that you have via email (interview scheduling, etc).

Best of luck, and let us know if there are any issues as early on as possible

TensorBoardOutput's resource is not cleaned up nicely

When using the logger with tensorboard, it will hang if we don't call logger.remove_all(). Here is a simple snippet for reproduce it:

import dowel
from dowel import logger

logger.add_output(dowel.TensorBoardOutput(log_dir))

This is mostly due to tensorboardX.SummaryWriter failed to clean up its resource.

HDF5Output for the logger

Hi guys,
I am currently in need of recording trajectories of experiments, hence, I will add my own hdf5 logger. Hence the feature request to have the capability to log HDF5 files with data in the future.

Justification:
The current params.pkl is not a very convenient way to store trajectories (and other heavy statistics/data) for a few reasons:

• pkl files usually quite large, hdf5 files are more compact
• current params.pkl requires running a session and using joblib. I need more framework-independent storage (for example, you can imagine if I would like to use Matlab later to load trajectories and do some postprocessing for the paper).

Robust handling of inconsistent TabularInput keys

Currently, CsvOutput emits a warning if the keys of a TabularInput change after the first call to logger.log(TabularInput). A new key not seen before will be ignored and an old key not presented will be left blank. In other words, CsvOutput conservatively handles dynamic fieldnames.

This behaviour of CsvOutput makes it tricky to log performance of Multi- and Meta- ML algorithms, where there are usually per-task fields but not every task is presented in every iteration, resulting in missing of logs for some tasks.

The desired behaviour to handle inconsistent keys should be

• When a new key is encountered
  • Expand header with the new key.
  • Expand old rows with empty cells for the new key.
• If the value of any key is missing, leave the cell blank.

dowel causes the main process to hang forever, if it contains a TensorboardOutput when the process is closing

This is because it attempts to close the underlying TensorboardX writer in TensorboardOutput.__del__. However, global teardown of the python interpreter has already closed the thread used by TensorboardX.

VisdomOutput for the logger

This is worth a try

https://github.com/facebookresearch/visdom