rlworkgroup / dowel Goto Github PK
View Code? Open in Web Editor NEWA little logger for machine learning research
License: MIT License
A little logger for machine learning research
License: MIT License
This is because it attempts to close the underlying TensorboardX writer in TensorboardOutput.__del__
. However, global teardown of the python interpreter has already closed the thread used by TensorboardX.
This is worth a try
This is quite heavy and not everyone will want it
Currently, CsvOutput
emits a warning if the keys of a TabularInput
change after the first call to logger.log(TabularInput)
. A new key not seen before will be ignored and an old key not presented will be left blank. In other words, CsvOutput
conservatively handles dynamic fieldnames.
This behaviour of CsvOutput
makes it tricky to log performance of Multi- and Meta- ML algorithms, where there are usually per-task fields but not every task is presented in every iteration, resulting in missing of logs for some tasks.
The desired behaviour to handle inconsistent keys should be
Besides visdom (#280), another monitoring solution that could be more useful than TensorBoard would be Grafana.
It's also used by OpenAI for their DOTA project:
https://blog.openai.com/openai-five/
Hi guys,
I am currently in need of recording trajectories of experiments, hence, I will add my own hdf5 logger. Hence the feature request to have the capability to log HDF5 files with data in the future.
Justification:
The current params.pkl is not a very convenient way to store trajectories (and other heavy statistics/data) for a few reasons:
When I tried to run the following commands from the "Git recipes" in CONTRIBUTING.md, I got error messages:
git remote add rlworkgroup [email protected]:rlworkgroup/dowel.git
git reset --hard master rlworkgroup/master
However, the following would work:
git remote add rlworkgroup https://github.com/rlworkgroup/dowel.git
git checkout master
git fetch rlworkgroup
git reset --hard rlworkgroup/master
Should CONTRIBUTING.md be updated?
pytest tends to generate better error messages and has a more modern design.
Dowel is a tool that the garage Team uses for logging results from our various Reinforcement learning experiments.
Dowel can be used to log different types of data such as floats or strings. The logs can be logged to stdout (the console), CSV files, and Tensorboard.
You can check out an example of how Dowel is used here. In fact, almost all parts of the Dowel API are used in this example.
After statistics such as loss have been logged, and a call to logger.dump_all()
is made for the first time, new tabular data can’t be written to a CSV output. This is because currently data cannot be inconsistently logged to CSV, meaning that on every single call to dump_all, the same logger keys must appear. Data that is inconsistently logged will not appear in the CSV output. This is a design flaw that we have been able to work around but affects our workflows.
Your goal is to solve the problem as well as introduce tests into our testing framework in order to verify your solution.
If you have any questions, open an issue in your fork, and tag @avnishn and @haydenshively. Our preferred mode of communication on any questions that you have is through github issues and pull requests, as this is how the Garage team communicates generally. For this reason, we won’t respond to any direct emails with regards to help with your project. We will however respond to any other questions that you have via email (interview scheduling, etc).
Best of luck, and let us know if there are any issues as early on as possible
Provide a decorator for quickly adding a LogOutput by decorating a callable, e.g.
@logger.output(str, int)
def my_fun_handler(data, prefix=None):
print('{} | datum: {}'.format(prefix, datum))
When using the logger with tensorboard, it will hang if we don't call logger.remove_all()
. Here is a simple snippet for reproduce it:
import dowel
from dowel import logger
logger.add_output(dowel.TensorBoardOutput(log_dir))
This is mostly due to tensorboardX.SummaryWriter
failed to clean up its resource.
Link https://dowel.readthedocs.io/en/latest/py-modindex.html is broken.
Coverage reporting to CodeCov is not actually working in the CI. This is probably a misconfiguration.
See https://travis-ci.com/rlworkgroup/dowel/builds/111406917#L744 for an example log of the error.
e.g. some log entries (including log_tabular) are level DEBUG, some are level INFO, etc.
...rather than implementing one for each object
Hi,
Thank you for this nice a simple tool for logging machine learning research. I often encounter situations where I would like to save multi-dimensional Numpy arrays. For example, the observation at each time-step in a reinforcement learning experiment.
It would be nice to have an output logger that supports Numpy arrays, Pytorch Tensors and Tensorflow Tensors.
I have written a simple output logger, NpzOutput
, that writes Numpy arrays to a .npz
file using Numpys savez
functions. It is not optimal (no incremental saving), but thought I share it in case somebody is interested.
It would be nice for the logger to show when a message comes from a worker process versus the main thread.
Logger.TabularInput is needlessly cumbersome for the most common logging use case, and forces LogOutputs to replicate parts of the logger's dispatch and accounting logic internally. It would be simpler and more consistent just to provide a key-based API.
This would also open up the logger to features like filtering output handlers based on key regexs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.