rustingsword/tensorboard_logger

C++ API to log data in tensorboard format.

TensorBoard Logger

Standalone C++ API to log data in TensorBoard format, without any code dependency on TensorFlow or TensorBoard.

Only support scalar, histogram, image, audio text and projector at the moment.

Using the library

Protobuf is the only dependency and assumed to be available via cmake's find_package.

To build and install TensorBoard Logger with cmake:

> BUILD_DIR=build
> mkdir -p $BUILD_DIR && cmake -B $BUILD_DIR . && cmake --build $BUILD_DIR -j
> cmake --install $BUILD_DIR

Alternatively to build a shared library set BUILD_SHARED_LIBS=ON:

> BUILD_DIR=build
> mkdir -p $BUILD_DIR && cmake -B $BUILD_DIR -DBUILD_SHARED_LIBS=ON . && cmake --build $BUILD_DIR -j
> cmake --install $BUILD_DIR

Then use find_package(tensorboard_logger REQUIRED) in your project.

Alternatively use via cmake FetchContent:

FetchContent_Declare(
    tensorboard_logger
    GIT_REPOSITORY      https://github.com/RustingSword/tensorboard_logger.git
    GIT_TAG             master
)
FetchContent_MakeAvailable(tensorboard_logger)

Make sure to add tensorboard_logger to target_link_libraries of your projects target.

To run the test:

> BUILD_DIR=build
> mkdir -p $BUILD_DIR && cmake -DBUILD_TEST=ON -B $BUILD_DIR . && cmake --build $BUILD_DIR -j
> mkdir -p demo && ./$BUILD_DIR/tensorboard_logger_test
> tensorboard --logdir demo  # try adding --load_fast=false if you don't see projector tab

Screenshots

Acknowledgement

• Inspired by dmlc tensorboard project.

• CRC code from The SNIPPETS Portable C/C++ Source Code Collection, via http://stackoverflow.com/a/303020, with some modifications.

• Audio sample from File Examples.

tensorboard_logger's Issues

Multiple scalars in single plot

Thanks for the work. Its great and handy.

Is there a way to print multiple scalars in single plot ?

Example

add image with raw pixel data

Hi, I read all codes in the test file that I actually didn't find an example to save images with its raw pixel data.
Does it already supported or not?

Any documentation On how to use it ?

I would like to know how to use it , How should I call the Logger from my c++ file

Resuming from a previous run

I'm currently using this project as a way to track metrics during ML training. Since training can take long, I have a way which can save/store models/data as a checkpoint during an exit call, and will reload that to resume training. It would also be nice if metrics can continue to track rather than starting from scratch.

Would this be a useful feature to add? Currently, I've changed the constructor as the following (for my personal use)

// Header of TensorBoardLogger
// ofs_ = new std::ofstream(log_file, std::ios::out | std::ios::trunc | std::ios::binary);
ofs_ = new std::ofstream(log_file, std::ios::out | std::ios::app | std::ios::binary);

Adding this feature would be a quick change. Maybe have a resume flag as a default argument set false in the header for TensorBoardLogger? If this something that would be welcomed, I could make a PR.

[question] Running event file on tensorboard

Hi there, I was able to generate the output event file in demo folder named as tfevents.pb. My question is how do I visualise this on tensorboard as you have depicted in your README. I tried running tensorboard --logdir=demo which I verified using find demo | grep tfevents is able to find the events file but opening the link in the web browser doesn't show any output.
Thanks.

Also, it would be really helpful if you could mention the protobuf version as 3.11 somewhere this link helped install properly. And in Makefile had to use LDFLAGS = -pthread -I/usr/local/include -L/usr/local/lib -lprotobuf for compiling ( output of pkg-config --cflags --libs protobuf ). Thanks for helping. Brilliant work !!

undefined reference error

Hi I am struggling to get this running . i have followed this steps

1. clone repo
2. make
3. included directories in my cmake files
4. tried adding .a file to local and .a file to cmakelist file

/usr/bin/ld: /home/dewe/CLionProjects/FastDRL/utilities/logger.cpp:93: undefined reference to TensorBoardLogger::add_scalar(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, double)' /usr/bin/ld: /home/dewe/CLionProjects/FastDRL/utilities/logger.cpp:94: undefined reference to TensorBoardLogger::add_scalar(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, double)'
/usr/bin/ld: /home/dewe/CLionProjects/FastDRL/utilities/logger.cpp:95: undefined reference to TensorBoardLogger::add_scalar(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, double)' /usr/bin/ld: CMakeFiles/FastDRL.dir/utilities/logger.cpp.o: in function Logger::~Logger()':

Know why i'm just getting a core dump?

after make

:~/tensorboard_logger$ ./test
terminate called after throwing an instance of 'std::system_error'
  what():  Unknown error -1
Aborted (core dumped)

Relative path of proto files should be kept as-is

Currently those proto files are put in one directory, which requires manually modifying some import directives. It is better to keep them in the original hierarchy.

add includes to target

I use this repo as a dependency, however I had to alter the CMake file slightly.

From

include_directories(
    ${TENSORBOARD_INCLUDE_PATH}
    ${Protobuf_INCLUDE_DIRS}
)

add_library(tensorboard_logger STATIC
  ${sources}
  ${PROTO_SRCS}
)

To

add_library(tensorboard_logger STATIC
  ${sources}
  ${PROTO_SRCS}
)

target_include_directories(
  tensorboard_logger PRIVATE ${Protobuf_INCLUDE_DIRS}
)
target_include_directories(
  tensorboard_logger PUBLIC ${TENSORBOARD_INCLUDE_PATH}
)

This allows adding the header files by simply linking the library to whatever library I am working on, instead of manually linking the header directory. Could this be added to master?

Data flushed on every write

I noticed in src that data is flushed on every event. Could this be controlled (like in tensorboard clients in pytorch/tensorflow) to happen only every Nth write and/or timeout?

file INSTALL cannot find on Windows

Hi,

Thank you for this great package! I'm trying to follow the readme to compile and install it on a Windows machine, however, I'm getting errors during the install stage.

Namely, I followed:

> BUILD_DIR=build
> mkdir -p $BUILD_DIR && cmake -B $BUILD_DIR . && cmake --build $BUILD_DIR -j
> cmake --install $BUILD_DIR

Which runs file for --build (although I got a bunch of warnings such as warning C5045: Compiler will insert Spectre mitigation for memory load if /Qspectre switch specified from this package and warning C4514: 'google::protobuf::Descr iptor::reserved_range_count': unreferenced inline function has been removed from protobuf). However, when I try to run cmake --install $BUILD_DIR, I got:

-- Install configuration: "Release"
CMake Error at build/cmake_install.cmake:39 (file):
  file INSTALL cannot find
  "C:/nonsys/workspace/tensorboard_cpp/build/Release/tensorboard_logger.lib":
  No error.

It seems to be a badly phrased error message relating to the install command in cmakelist (https://gitlab.kitware.com/cmake/cmake/-/issues/19950) but I'm not sure how to resolve it.

Moreover, when trying to compile with the tests via:

> BUILD_DIR=build
> mkdir -p $BUILD_DIR && cmake -DBUILD_TEST=ON -B $BUILD_DIR . && cmake --build $BUILD_DIR -j
> mkdir -p demo && ./$BUILD_DIR/tensorboard_logger_test

I got many errors like this:

tensorboard_logger.lib(projector_config.pb.obj) : error LNK2001: unresolved external symbol "class google::protobuf::i
nternal::ExplicitlyConstructed<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >
 > google::protobuf::internal::fixed_address_empty_string" (?fixed_address_empty_string@internal@protobuf@google@@3V?$
ExplicitlyConstructed@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@123@A) [C:\nonsys\workspace\tenso
rboard_cpp\build\tensorboard_logger_test.vcxproj]
C:\nonsys\workspace\tensorboard_cpp\build\Debug\tensorboard_logger_test.exe : fatal error LNK1120: 1 unresolved extern
als [C:\nonsys\workspace\tensorboard_cpp\build\tensorboard_logger_test.vcxproj]

May I ask what version of protobuf are we supposed to use?

Many thanks!

Event files not detected by tensorboard

My event files are not automatically detected by tensorboard, I need to specifically set the file extension to something along the lines of events.out.tfevents for data to appear in tensorboard.

Outdated link to embedding projector

When starting embedding projector with
tensorboard --logdir path
the link which appears in the console localhost: (for me) always redicrect to http://localhost:/# which shows an error message. Only when manually changing the link to http://localhost:PORT/#projector it works.

This works:

In [1]: import numpy as np
In [2]: np.savetxt('demo/vec.tsv', np.random.random((1000, 16)), delimiter="\t")
In [3]: np.savetxt('demo/label.tsv', np.arange(1000))

> ls demo
label.tsv              projector_config.pbtxt vec.tsv
> cat demo/projector_config.pbtxt
embeddings {
    tensor_path: "vec.tsv"
    metadata_path: "label.tsv"
}
> tensorboard --logdir demo

Originally posted by @RustingSword in #2 (comment)