Collect information from the environment at runtime to print before backtraces

[rethinkdb]

Collect information from the environment at runtime to print before backtraces

[mlucy]

So, after thinking about it for a bit, I believe we should also print the full version number at startup. That way if we have a non-crashing bug, we can try to reproduce it with the same version of RethinkDB as the user. This is less important now, but will become more important as versions diverge more significantly.

[coffeemug]

We already have rethinkdb --version. We just need to print that by default.

[coffeemug]

Update: it looks like rethinkdb --version implemented in the PPA returns the string 'rethinkdb' . We should probably fix that :)

[al3xandru]

Ideally, I think rethinkdb --version should print not only the version number, but also some details about the platform RethinkDB is installed on (Kernel v, FS, etc.).

[jdoliner]

I agree that we should be able to print that information. I disagree about
putting it under --version. --version has a pretty well understood meaning
in the linux world and I don't think it includes this. It should be a
different command.

On Mon, Nov 12, 2012 at 9:34 AM, Alex Popescu [email protected]:

Ideally, I think rethinkdb --version should print not only the version
number, but also some details about the platform RethinkDB is installed on
(Kernel v, FS, etc.).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/39#issuecomment-10296559.

[coffeemug]

rethinkdb --arch? or rethinkdb --platform?

[jdoliner]

I like platform best.

[mlucy]

The first part of this is done (we now print out version strings before backtraces, along with the compiler used to compile it).

It sounds like we also want to make RethinkDB collect some data about its environment at startup. Let's try to make a list of what we want to see before a backtrace:

• The version of RethinkDB
• The output of uname -a
• The output of df -T

Is there anything else?

[mglukhovsky]

If journaling or other mount options matter, then df -T won't show those-- the output of mount will.

[coffeemug]

cat /etc/issue

However, before we do this we should be aware that this ought to be portable. For example, /etc/issue is meaningless in rpm (I think), so on that platform, we should call something else.

[jdoliner]

We had some code at one point that collected disk information that might be
a good place to start.

On Monday, November 12, 2012, Michael Lucy wrote:

The first part of this is done (we now print out version strings before
backtraces, along with the compiler used to compile it).

It sounds like we also want to make RethinkDB collect some data about its
environment at startup. Let's try to make a list of what we want to see
before a backtrace:

• The version of RethinkDB
• The output of uname -a
• The output of df -T

Is there anything else?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/39#issuecomment-10316645.

[mlucy]

Alright, I have a code review (#68) out for collecting runtime information. Here's what it looks like now when someone crashes:

Version: rethinkdb 1.2.6-25-ga5eba2-dirty (debug) (CLANG 3.0 (tags/RELEASE_30/final))
Runtime Data:
/etc/issue: Ubuntu 11.10 \n \l


/proc/version: Linux version 3.0.0-13-server (buildd@crested) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #22-Ubuntu SMP Wed Nov 2 15:09:08 UTC 2011

CWD: /home/ssd0/mlucy/rethinkdb/src
/proc/mounts: rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=32976644k,nr_inodes=8244161,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,relatime,size=13194172k,mode=755 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
/dev/disk/by-uuid/3945d61b-67ba-48e7-989c-c5049d4eeeea / ext4 rw,relatime,errors=remount-ro,user_xattr,acl,barrier=1,data=ordered 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,nosuid,nodev,relatime 0 0
/dev/sdb1 /home/ssd0 xfs rw,noatime,attr2,delaylog,allocsize=512k,noquota 0 0
/dev/sdc1 /home/ssd1 xfs rw,noatime,attr2,delaylog,allocsize=512k,noquota 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0
rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0

error: Error in extproc/pool.cc at line 215:
error: Error on worker process socket
error: Backtrace:
info: Server got SIGINT; shutting down...
info: Shutting down client connections...
error: Wed Nov 14 00:13:36 2012

       1: lazy_backtrace_t::lazy_backtrace_t() at backtrace.cc:251
       2: format_backtrace(bool) at backtrace.cc:198
       3: report_fatal_error(char const*, int, char const*, ...) at errors.cc:76
       4: extproc::pool_t::worker_t::on_error() at pool.cc:215
       5: extproc::pool_t::worker_t::on_event(int) at pool.cc:207
       6: non-virtual thunk to extproc::pool_t::worker_t::on_event(int) at pool.cc:208
       7: linux_event_fd_watcher_t::on_event(int) at socket_stream.cc:71
       8: non-virtual thunk to linux_event_fd_watcher_t::on_event(int) at socket_stream.cc:71
       9: linux_event_watcher_t::on_event(int) at event_watcher.cc:73
       10: non-virtual thunk to linux_event_watcher_t::on_event(int) at event_watcher.cc:85
       11: epoll_event_queue_t::run() at epoll.cc:114
       12: linux_thread_pool_t::start_thread(void*) at thread_pool.cc:136
       13: +0x7e9a at 0x7f067f0dbe9a (/lib/x86_64-linux-gnu/libpthread.so.0)
       14: clone+0x6d at 0x7f067ee0974d (/lib/x86_64-linux-gnu/libc.so.6)
error: Exiting.
trace/breakpoint trap (core dumped)

[coffeemug]

1. I think the value of printing mount info is very questionable. It turns the message into a wall of text, and I don't see what useful information we can extract out of it.
2. cat /etc/issue isn't portable -- for example, we'll be adding rpm support soon and beyond. This needs to be abstracted (if it isn't already) so it can work on all systems or at least degrade to "can't determine the distro version".
3. uname -a is better than /proc/version -- the former exists everywhere, whereas the latter does not.

[mlucy]

At first I was going to say that /proc/version is basically universal, but then I remembered that we're planning an OS X port, which means no /proc for us. (Actually, don't we use /proc in the stats collection, too?)

I'm hesitant to start forking processes while we're crashing. Why don't we just print the version number when we crash, like we do now, and fold this into the bug-report functionality?

Also, this sort of thing should really be encapsulated in a shell script rather than in C++ code.

Closing and merging into Issue #41, unless anybody objects.

[coffeemug]

Makes sense to me.