bosun-monitor / bosun Goto Github PK
View Code? Open in Web Editor NEWTime Series Alerting Framework
Home Page: http://bosun.org
License: MIT License
bosun's People
Contributors
Stargazers
Watchers
Forkers
houbl supportingit alihalabyah bs-github thejaspm skirmish wessie gaurav46 martinwalsh baijum yekeqiang rhoml virhappy xuzhaokui david-brabant artschwagerb lostz ajufrancis doellerbe vicever no2key four2s dieterbe vimeo vnaar shelocks inthecloud247 d10v aravindtj mingpengxiao oliveagle hucarrot g-var cw2018 dpvreony-forks wkshare df07 cmiller8 lwf alienth voilet zihuxinyu somesky rchunping justintung mvc4 xyhp915 zhuweijava k2v xuxueyun cluo kkossack mrxiaoz wuchuguang chenny vitorboschi onetwotrip canghai908 kadaan zofuthan seekingalpha zengnotes nashtsai edgarlanting blezek cyclefusion buglomi xudong-wen southwolf zdpwilliam xdl huayl dszeto gpduck delkyd cyngn nvhoang inkel mathpl couchand walkeran leekui wenxiaoyu gitter-badger mxk1235 yyljlyy slashmanx frank3nst3in tchen0123 ht808s brightbird mehulkar bytedance dieface azhurbilo briantist jmj bridgewell fisdap cazacugmihaibosun's Issues
Date, Time, Number input types not supported in FF
The date, time and number form input types are not supported in FireFox (and IE I think, but since the site doesn't even load in IE I'm guessing you don't care). They are rendered as plain text inputs boxes with no picker options or built in validation.
Alert Rule documentation link
There should be a link or quick start sidebar on the alert rule page explaining how alert rules work, and giving an overview of what is needed to write one.
Bad name
This doesn't have an inscrutable in joke for a name, fix it.
Ability to retain queries in application
It would be cool if one could save their large queries on the website as well as having the ability to reach it directly via a URL string.
Display issue with computations on the dashboard
Link to Expression syntax doc
There should be a link to the expression syntax docs with examples on the Expressions and Expressions Graph pages so that we can quickly look up how the expression syntax works (and refresh on little used syntax).
Better Error Message for Time Argument to q func?
Starting at line Line 61:
alert os.diskspace {
template = generic
$q = avg(q("avg:rate:os.disk.fs.percent_free{host=*}", 30m))
#crit is 5 here because once Linux reaches 5% free space than not-root users can no longer
# write to the disk
crit = $q <= 5
warn = $q < 15
critNotification = default
}
Returns the following:
go run main.go -c ~/src/tsaf-config/prod.conf -t
2014/03/06 10:00:47 conf: /Users/kbrandt/src/tsaf-config/prod.conf:66:1: at <crit = $q < 5>: expr: unexpected "m" in func
exit status 1
Error message would be more informative if:
- It was able to refer to the line $q is defined on instead of the evaluation line
- The error was something roughly like: "second argument 'duration' of func q must be string"
Alert is Triggered Even though it Evaluates to 0
Not sure if this is a dupe or not, but figured I would file it just in case or as another example.
Expression causes 2G memory
change("sum:rate{counter,,1}:web.question.closed{site=*}", "5d", "5m") > change("sum:rate{counter,,1}:web.answer.created{site=*}", "5d", "5m")
Group unknowns by tag/metric/notification
runtime: goroutine stack exceeds 1000000000-byte limit
2014/02/25 09:18:27 cpu at 12.633333333333333 on ny-devtsdb04.ds.stackexchange.com
2014/02/25 09:18:32 cpu at 12.633333333333333 on ny-devtsdb04.ds.stackexchange.com
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow
runtime stack:
runtime.throw(0x780b00)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:464 +0x69
runtime.newstack()
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/stack.c:295 +0x3bc
runtime.morestack()
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/asm_amd64.s:225 +0x61
goroutine 24 [stack split]:
reflect.flag.mustBe(0x156, 0x15)
/usr/local/Cellar/go/1.2/libexec/src/pkg/reflect/value.go:238 fp=0x3ea6f100
reflect.Value.MapKeys(0x2bee20, 0xc21000f460, 0x156, 0x0, 0xc21bde9000, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/reflect/value.go:1131 +0x33 fp=0x3ea6f180
encoding/json.(_mapEncoder).encode(0xc21036a690, 0xc210084160, 0x2bee20, 0xc21000f460, 0x156, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:617 +0xdb fp=0x3ea6f218
encoding/json._mapEncoder.(encoding/json.encode)·fm(0xc210084160, 0x2bee20, 0xc21000f460, 0x156, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:635 +0x58 fp=0x3ea6f250
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea6f370
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea6f3a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea6f3e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea6f418
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea6f450
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea6f570
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea6f5a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea6f5e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea6f618
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea6f650
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea6f770
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea6f7a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea6f7e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea6f818
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea6f850
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea6f970
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea6f9a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea6f9e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea6fa18
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea6fa50
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea6fb70
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea6fba8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea6fbe0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea6fc18
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea6fc50
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea6fd70
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea6fda8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea6fde0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea6fe18
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea6fe50
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea6ff70
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea6ffa8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea6ffe0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70018
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70050
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70170
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea701a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea701e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70218
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70250
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70370
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea703a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea703e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70418
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70450
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70570
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea705a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea705e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70618
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70650
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70770
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea707a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea707e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70818
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70850
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70970
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea709a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea709e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70a18
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70a50
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70b70
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea70ba8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea70be0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70c18
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70c50
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70d70
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea70da8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea70de0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea70e18
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea70e50
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea70f70
----- stack segment boundary -----
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea711a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea711e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea71218
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea71250
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea71370
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea713a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea713e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea71418
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea71450
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea71570
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea715a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea715e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea71618
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea71650
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea71770
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea717a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea717e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea71818
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea71850
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea71970
encoding/json._structEncoder.(encoding/json.encode)·fm(0xc210084160, 0x358ee0, 0xc21000f460, 0x196, 0xc21000f400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:604 +0x58 fp=0x3ea719a8
encoding/json.(_ptrEncoder).encode(0xc21036aea0, 0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:712 +0xd9 fp=0x3ea719e0
encoding/json._ptrEncoder.(encoding/json.encode)·fm(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:717 +0x58 fp=0x3ea71a18
encoding/json.func·003(0xc210084160, 0x26bbc0, 0xc21000f4a8, 0x166, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:342 +0x6d fp=0x3ea71a50
encoding/json.(_structEncoder).encode(0xc2103446f0, 0xc210084160, 0x358ee0, 0xc21000f460, 0x196, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/encoding/json/encode.go:590 +0x277 fp=0x3ea71b70
created by net/http.(_Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
goroutine 1 [select (no cases)]:
main.main()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:33 +0x17f
goroutine 3 [chan receive]:
github.com/StackExchange/tsaf/search.Process(0xc210038180)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/search/search.go:82 +0x3a
created by github.com/StackExchange/tsaf/search.init·1
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/search/search.go:64 +0x31
goroutine 10 [IO wait]:
net.runtime_pollWait(0x8cc978, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc210081610, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc210081610, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).accept(0xc2100815b0, 0x434188, 0x0, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:382 +0x2c2
net.(_TCPListener).AcceptTCP(0xc210000468, 0xeaaeb, 0xb8bd90, 0xeaaeb)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(_TCPListener).Accept(0xc210000468, 0x8ccd08, 0xc2100d8380, 0xc2101aff80, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(_Server).Serve(0xc2100a2820, 0x8cb9d8, 0xc210000468, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1622 +0x91
net/http.(*Server).ListenAndServe(0xc2100a2820, 0xc2100a2820, 0x8cb958)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1612 +0xa0
net/http.ListenAndServe(0x378e00, 0x5, 0x8cb958, 0xc2100a7120, 0xc210044f30, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1677 +0x6d
github.com/StackExchange/tsaf/relay.RelayHTTP(0x378e00, 0x5, 0xc2100a3720, 0x26, 0x6, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/relay/relay.go:20 +0x2c7
main.func·001()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:29 +0x5b
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:29 +0x12f
goroutine 1130 [IO wait]:
net.runtime_pollWait(0x8cc048, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc210186060, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc210186060, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc210186000, 0xc21032f000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d8b80, 0xc21032f000, 0x1000, 0x1000, 0x30, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
bufio.(_Reader).fill(0xc2101d68a0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).Peek(0xc2101d68a0, 0x1, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:119 +0xcb
net/http.(_persistConn).readLoop(0xc2101d3380)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:687 +0xb7
created by net/http.(_Transport).dialConn
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:528 +0x607
goroutine 11 [IO wait]:
net.runtime_pollWait(0x8cc828, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc2100c4840, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc2100c4840, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).accept(0xc2100c47e0, 0x434188, 0x0, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:382 +0x2c2
net.(_TCPListener).AcceptTCP(0xc210000910, 0xeaaeb, 0xb87d08, 0xeaaeb)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(_TCPListener).Accept(0xc210000910, 0x8ccd08, 0xc2100d8f00, 0xc2101afa00, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(_Server).Serve(0xc2100a2e60, 0x8cb9d8, 0xc210000910, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1622 +0x91
net/http.(*Server).ListenAndServe(0xc2100a2e60, 0xc2100a2e60, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1612 +0xa0
net/http.ListenAndServe(0x378e40, 0x5, 0x0, 0x0, 0xc2100b0a50, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1677 +0x6d
github.com/StackExchange/tsaf/web.Listen(0x378e40, 0x5, 0x393ce0, 0x3, 0xc2100a3720, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/web/web.go:50 +0x9df
main.func·002()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:30 +0x78
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:30 +0x15c
goroutine 12 [chan receive]:
github.com/StackExchange/tsaf/sched.(*Schedule).Run(0x77a660, 0x0, 0x0)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:74 +0x24b
github.com/StackExchange/tsaf/sched.Run(0x0, 0x0)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:54 +0x2a
main.func·003()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:31 +0x1e
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:31 +0x16a
goroutine 23 [IO wait]:
net.runtime_pollWait(0x8cc780, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc2100c4610, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc2100c4610, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc2100c45b0, 0xc21014d000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d86d0, 0xc21014d000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
net/http.(_liveSwitchReader).Read(0xc2101af328, 0xc21014d000, 0x1000, 0x1000, 0x3, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:204 +0xa5
io.(_LimitedReader).Read(0xc210149a80, 0xc21014d000, 0x1000, 0x1000, 0x8, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/io/io.go:398 +0xbb
bufio.(_Reader).fill(0xc2100af8a0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).ReadSlice(0xc2100af8a0, 0xc2101c3d0a, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:274 +0x204
bufio.(_Reader).ReadLine(0xc2100af8a0, 0x0, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:305 +0x63
net/textproto.(_Reader).readLineSlice(0xc2100a71b0, 0x8c2000, 0x278d60, 0xb83ce8, 0x213c2, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:55 +0x61
net/textproto.(_Reader).ReadLine(0xc2100a71b0, 0xc2102579c0, 0x0, 0xc21014e000, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:36 +0x27
net/http.ReadRequest(0xc2100af8a0, 0xc2102579c0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/request.go:526 +0x88
net/http.(_conn).readRequest(0xc2101af300, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:575 +0x1bb
net/http.(_conn).serve(0xc2101af300)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1123 +0x3b4
created by net/http.(_Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
goroutine 20 [syscall]:
syscall.Syscall6(0x16b, 0x9, 0x0, 0x0, 0xc2100bedc0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/syscall/asm_darwin_amd64.s:41 +0x5
syscall.kevent(0x9, 0x0, 0x0, 0xc2100bedc0, 0xa, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/syscall/zsyscall_darwin_amd64.go:199 +0x85
syscall.Kevent(0x9, 0x0, 0x0, 0x0, 0xc2100bedc0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/syscall/syscall_bsd.go:431 +0x9d
github.com/howeyc/fsnotify.(*Watcher).readEvents(0xc2100b1270)
/Users/kbrandt/src/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:334 +0x31b
created by github.com/howeyc/fsnotify.NewWatcher
/Users/kbrandt/src/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:108 +0x2e6
goroutine 17 [syscall]:
runtime.goexit()
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/proc.c:1394
goroutine 21 [chan receive]:
github.com/howeyc/fsnotify.(*Watcher).purgeEvents(0xc2100b1270)
/Users/kbrandt/src/go/src/github.com/howeyc/fsnotify/fsnotify.go:21 +0x41
created by github.com/howeyc/fsnotify.NewWatcher
/Users/kbrandt/src/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:109 +0x2fd
goroutine 22 [select]:
main.func·004()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:48 +0x233
created by main.watcher
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:56 +0x12c
goroutine 25 [IO wait]:
net.runtime_pollWait(0x8cc630, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc2100c4370, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc2100c4370, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc2100c4310, 0xc21010b000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d8750, 0xc21010b000, 0x1000, 0x1000, 0xc210046e30, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
net/http.(_liveSwitchReader).Read(0xc2101af428, 0xc21010b000, 0x1000, 0x1000, 0x8, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:204 +0xa5
io.(_LimitedReader).Read(0xc210149ac0, 0xc21010b000, 0x1000, 0x1000, 0x192, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/io/io.go:398 +0xbb
bufio.(_Reader).fill(0xc2100af960)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).ReadSlice(0xc2100af960, 0xc20000000a, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:274 +0x204
bufio.(_Reader).ReadLine(0xc2100af960, 0x0, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:305 +0x63
net/textproto.(_Reader).readLineSlice(0xc210292de0, 0x8c2000, 0x278d60, 0xcefce8, 0x213c2, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:55 +0x61
net/textproto.(_Reader).ReadLine(0xc210292de0, 0xc210257680, 0x0, 0xc21010c000, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:36 +0x27
net/http.ReadRequest(0xc2100af960, 0xc210257680, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/request.go:526 +0x88
net/http.(_conn).readRequest(0xc2101af400, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:575 +0x1bb
net/http.(_conn).serve(0xc2101af400)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1123 +0x3b4
created by net/http.(_Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
goroutine 26 [IO wait]:
net.runtime_pollWait(0x8cc588, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc2100c4060, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc2100c4060, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc2100c4000, 0xc21010d000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d8790, 0xc21010d000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
net/http.(_liveSwitchReader).Read(0xc2101af4a8, 0xc21010d000, 0x1000, 0x1000, 0x1, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:204 +0xa5
io.(_LimitedReader).Read(0xc210149ae0, 0xc21010d000, 0x1000, 0x1000, 0x8, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/io/io.go:398 +0xbb
bufio.(_Reader).fill(0xc2100afa20)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).ReadSlice(0xc2100afa20, 0xc2101a5a0a, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:274 +0x204
bufio.(_Reader).ReadLine(0xc2100afa20, 0x0, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:305 +0x63
net/textproto.(_Reader).readLineSlice(0xc2100a7420, 0x8c2000, 0x278d60, 0xcedce8, 0x213c2, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:55 +0x61
net/textproto.(_Reader).ReadLine(0xc2100a7420, 0xc210236340, 0x0, 0xc21010e000, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:36 +0x27
net/http.ReadRequest(0xc2100afa20, 0xc210236340, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/request.go:526 +0x88
net/http.(_conn).readRequest(0xc2101af480, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:575 +0x1bb
net/http.(_conn).serve(0xc2101af480)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1123 +0x3b4
created by net/http.(_Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
goroutine 27 [IO wait]:
net.runtime_pollWait(0x8cc4e0, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc2100c48b0, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc2100c48b0, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc2100c4850, 0xc210110000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d87d0, 0xc210110000, 0x1000, 0x1000, 0x300, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
net/http.(_liveSwitchReader).Read(0xc2101af528, 0xc210110000, 0x1000, 0x1000, 0x2, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:204 +0xa5
io.(_LimitedReader).Read(0xc210149b00, 0xc210110000, 0x1000, 0x1000, 0x8, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/io/io.go:398 +0xbb
bufio.(_Reader).fill(0xc2100af000)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).ReadSlice(0xc2100af000, 0xc21015110a, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:274 +0x204
bufio.(_Reader).ReadLine(0xc2100af000, 0x0, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:305 +0x63
net/textproto.(_Reader).readLineSlice(0xc2100a7600, 0x8c2000, 0x278d60, 0xcebce8, 0x213c2, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:55 +0x61
net/textproto.(_Reader).ReadLine(0xc2100a7600, 0xc210236a90, 0x0, 0xc2100ce000, 0x801)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:36 +0x27
net/http.ReadRequest(0xc2100af000, 0xc210236a90, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/request.go:526 +0x88
net/http.(_conn).readRequest(0xc2101af500, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:575 +0x1bb
net/http.(_conn).serve(0xc2101af500)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1123 +0x3b4
created by net/http.(_Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
goroutine 28 [IO wait]:
net.runtime_pollWait(0x8cc438, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc2100bc220, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc2100bc220, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc2100bc1c0, 0xc210132000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d8f00, 0xc210132000, 0x1000, 0x1000, 0xc210046e30, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
net/http.(_liveSwitchReader).Read(0xc2101afa28, 0xc210132000, 0x1000, 0x1000, 0x8, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:204 +0xa5
io.(_LimitedReader).Read(0xc2100d1fa0, 0xc210132000, 0x1000, 0x1000, 0x192, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/io/io.go:398 +0xbb
bufio.(_Reader).fill(0xc2100aff00)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).ReadSlice(0xc2100aff00, 0xc20000000a, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:274 +0x204
bufio.(_Reader).ReadLine(0xc2100aff00, 0x0, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:305 +0x63
net/textproto.(_Reader).readLineSlice(0xc210152990, 0x8c2000, 0x278d60, 0xce9ce8, 0x213c2, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:55 +0x61
net/textproto.(_Reader).ReadLine(0xc210152990, 0xc2102368f0, 0x0, 0xc21019b000, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:36 +0x27
net/http.ReadRequest(0xc2100aff00, 0xc2102368f0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/request.go:526 +0x88
net/http.(_conn).readRequest(0xc2101afa00, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:575 +0x1bb
net/http.(_conn).serve(0xc2101afa00)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1123 +0x3b4
created by net/http.(_Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
goroutine 29 [IO wait]:
net.runtime_pollWait(0x8cc390, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc210153530, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc210153530, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc2101534d0, 0xc2100e4000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d8380, 0xc2100e4000, 0x1000, 0x1000, 0x737142, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
net/http.(_liveSwitchReader).Read(0xc2101affa8, 0xc2100e4000, 0x1000, 0x1000, 0x737142, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:204 +0xa5
io.(_LimitedReader).Read(0xc2100bd840, 0xc2100e4000, 0x1000, 0x1000, 0x737142, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/io/io.go:398 +0xbb
bufio.(_Reader).fill(0xc2100af780)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).ReadSlice(0xc2100af780, 0x4ec0a, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:274 +0x204
bufio.(_Reader).ReadLine(0xc2100af780, 0x0, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:305 +0x63
net/textproto.(_Reader).readLineSlice(0xc2100a74b0, 0x8c2000, 0x278d60, 0xcafce8, 0x213c2, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:55 +0x61
net/textproto.(_Reader).ReadLine(0xc2100a74b0, 0xc2100b19c0, 0x0, 0xc2100e5000, 0xcafb6f)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/textproto/reader.go:36 +0x27
net/http.ReadRequest(0xc2100af780, 0xc2100b19c0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/request.go:526 +0x88
net/http.(_conn).readRequest(0xc2101aff80, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:575 +0x1bb
net/http.(_conn).serve(0xc2101aff80)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1123 +0x3b4
created by net/http.(_Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
goroutine 1126 [IO wait]:
net.runtime_pollWait(0x8cc8d0, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(_pollDesc).Wait(0xc21011fbc0, 0x72, 0x8cb128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(_pollDesc).WaitRead(0xc21011fbc0, 0x23, 0x8cb128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(_netFD).Read(0xc21011fb60, 0xc210239000, 0x1000, 0x1000, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:204 +0x2a0
net.(_conn).Read(0xc2100d8af0, 0xc210239000, 0x1000, 0x1000, 0x30, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/net.go:122 +0xc5
bufio.(_Reader).fill(0xc2101d66c0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:91 +0x110
bufio.(_Reader).Peek(0xc2101d66c0, 0x1, 0x0, 0x0, 0x0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/bufio/bufio.go:119 +0xcb
net/http.(_persistConn).readLoop(0xc2101d3280)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:687 +0xb7
created by net/http.(_Transport).dialConn
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:528 +0x607
goroutine 1127 [select]:
net/http.(_persistConn).writeLoop(0xc2101d3280)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:791 +0x271
created by net/http.(_Transport).dialConn
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:529 +0x61e
goroutine 1131 [select]:
net/http.(_persistConn).writeLoop(0xc2101d3380)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:791 +0x271
created by net/http.(_Transport).dialConn
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:529 +0x61e
goroutine 2954 [select]:
github.com/StackExchange/tsaf/sched.(_Schedule).Notify(0x77a660, 0xc21015b880, 0xc21004f1b0, 0xc21000f460, 0xc210292660)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:163 +0x21b
created by github.com/StackExchange/tsaf/sched.(_Schedule).Notify
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:167 +0x206
exit status 2
URL for complex queries are ridiculously large
Can we come up with a way to make the url more manageable or have a url shortening/mapping service built in?
Possibility of renaming the graph metrics to make it more user understandable?
Managers aren't going to want to decipher the json-like legend for graph colors. It would be nice if we could specify the name of the graph in the legend to "prettify" the graph.
Missing Value when calling .E, not sure why
With the following alert rule:
alert os.diskspace {
template = diskspace
$time = "30m"
$q = avg(q("avg:os.disk.fs.percent_free{host=*,disk=*}", $time))
$used = avg(q("avg:os.disk.fs.space_used{host=*,disk=*}", $time))
$total = avg(q("avg:os.disk.fs.space_total{host=*,disk=*}", "$time))
#crit is 5 here because once Linux reaches 5% free space than not-root users can no longer
#write to the disk
#Testing out some squelching
squelch = disk=/media.*|/home/gbeech.*
crit = $q <= 5
warn = $q < 15
critNotification = default
}
I have the following subject:
subject = {{.Last.Status}}: Diskpace: ({{.Alert.Vars.used | .E}}/{{.Alert.Vars.total | .E}}) {{.Alert.Vars.q | .E}}% Free on {{.Group.host}}
For some reason used works, but total does not. Not sure why:
TSAF Crash due to Memory Explosion
TSAF was killed by Linux since the memory went from 500MB RSS to 1.8GB:
Mar 12 13:08:43 localhost kernel: Out of memory: Kill process 7422 (tsaf) score 941 or sacrifice child
Mar 12 13:08:43 localhost kernel: Killed process 7422, UID 0, (tsaf) total-vm:16774416kB, anon-rss:1698616kB, file-rss:144kB
tsaf.log was overwritten on a restart, so no more information there. Will fix that so it appends.
Alerts Linger when the Defs no longer Exist
I assume due to state saving, but none of these linux rules exist anymore in the config, but they still show up in the interface:
Alerts that fail due to Query Errors are Silent
The query in this alert is causing a duplicate data exception in OpenTSDB:
alert os.diskspace.forecast {
template = generic
$filter = host=*,disk=*
#Downsampling avg on opentsdb side will save the linear regression a lot of work
$q = (forecastlr(q("avg:30m-avg:os.disk.fs.space_free{$filter}", "1w"), 0) / 60 / 60)
warn = $q > 0 && $q < 60
crit = $q > 0 && $q < 30
}
Interestingly enough, the tagset doesn't make any sense and the duplicate data only happens with the downsampling aspect. I filed a bug with opentsdb.
Anyways, when underlying queries are broken we need to make the visible. Probably using the future unknown status.
Squelch doesn't work on the Web Interface's Rule Page
No indicator of what is squelched
"Broken" Metrics incorrectly report 0 values to alerts
TSAF logged one of the duplicate data "Shouldn't be here". I think somehow expressions should return an error on N/A and not 0 when there is an error. Currently we get something like:
Panic caused by too many arguments in inner func
For example with the following:
avg(q("avg:rate:redis.used_memory{host=ny-redis01,port=6379}", "1h", 1))
q has to many arguments, this causes a panic when q is wrapped in a function like avg().
Here is the panic:
2014/03/11 18:03:32 http: panic serving 127.0.0.1:50817: reflect: Call with too many input arguments
goroutine 15 [running]:
net/http.func·009()
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1093 +0xae
runtime.panic(0x2ab100, 0xc2101b8660)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:248 +0x106
github.com/StackExchange/tsaf/expr.errRecover(0x98ebe8)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:69 +0x124
runtime.panic(0x2ab100, 0xc2101b8660)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:248 +0x106
reflect.Value.call(0x332480, 0x467888, 0x130, 0x3a90e0, 0x4, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/reflect/value.go:400 +0x114c
reflect.Value.Call(0x332480, 0x467888, 0x130, 0xc21007dc00, 0x5, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/reflect/value.go:345 +0x9d
github.com/StackExchange/tsaf/expr.(*state).walkFunc(0xc2101e7e80, 0xc2100b3c40, 0x93fa60, 0xc2101b7900, 0x101, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:342 +0x548
github.com/StackExchange/tsaf/expr.(*state).walkFunc(0xc2101e7e80, 0xc2100b3bd0, 0x93fa60, 0xc2101b7900, 0x93fa01, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:336 +0x195
github.com/StackExchange/tsaf/expr.(*state).walk(0xc2101e7e80, 0x935ce8, 0xc2100b3bd0, 0x93fa60, 0xc2101b7900, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:173 +0x16a
github.com/StackExchange/tsaf/expr.func·001(0x93fa60, 0xc2101b7900)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:53 +0x63
github.com/MiniProfiler/go/miniprofiler.(*Timing).Step(0xc2101b7840, 0x3d6670, 0xc, 0xc2101e7ea0)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/types.go:197 +0x105
github.com/MiniProfiler/go/miniprofiler.(*Profile).Step(0xc210050b40, 0x3d6670, 0xc, 0xc2101e7ea0)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/types.go:176 +0x55
github.com/StackExchange/tsaf/expr.(*Expr).Execute(0xc2100c8b08, 0x93fa38, 0xc2101b8630, 0x935c70, 0xc210050b40, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:54 +0x182
github.com/StackExchange/tsaf/web.Expr(0x935c70, 0xc210050b40, 0x935c38, 0xc2101bf280, 0xc21005d340, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/web/expr.go:16 +0x11a
github.com/StackExchange/tsaf/web.func·002(0x935c70, 0xc210050b40, 0x935c38, 0xc2101bf280, 0xc21005d340)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/web/web.go:77 +0x56
github.com/MiniProfiler/go/miniprofiler.Handler.ProfileRequest(0xc2100b01c0, 0xc210050b40, 0x935c38, 0xc2101bf280, 0xc21005d340)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/miniprofiler.go:292 +0xb7
github.com/MiniProfiler/go/miniprofiler.Handler.ServeHTTP(0xc2100b01c0, 0x0, 0x935c38, 0xc2101bf280, 0xc21005d340)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/miniprofiler.go:286 +0x8c
github.com/MiniProfiler/go/miniprofiler.(*Handler).ServeHTTP(0xc2100b01d0, 0x935c38, 0xc2101bf280, 0xc21005d340)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/doc.go:1 +0xae
github.com/gorilla/mux.(*Router).ServeHTTP(0xc2100a1460, 0x935c38, 0xc2101bf280, 0xc21005d340)
/Users/kbrandt/src/go/src/github.com/gorilla/mux/mux.go:98 +0x217
net/http.(*ServeMux).ServeHTTP(0xc21000a660, 0x935c38, 0xc2101bf280, 0xc21005d340)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1496 +0x163
net/http.serverHandler.ServeHTTP(0xc2100a1c80, 0x935c38, 0xc2101bf280, 0xc21005d340)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1597 +0x16e
net/http.(*conn).serve(0xc210059b80)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1167 +0x7b7
created by net/http.(*Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
2014/03/11 18:03:32 http: panic serving 127.0.0.1:50819: reflect: Call with too many input arguments
goroutine 1803 [running]:
net/http.func·009()
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1093 +0xae
runtime.panic(0x2ab100, 0xc2101b87a0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:248 +0x106
github.com/StackExchange/tsaf/expr.errRecover(0xd7bbe8)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:69 +0x124
runtime.panic(0x2ab100, 0xc2101b87a0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:248 +0x106
reflect.Value.call(0x332480, 0x467888, 0x130, 0x3a90e0, 0x4, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/reflect/value.go:400 +0x114c
reflect.Value.Call(0x332480, 0x467888, 0x130, 0xc21007de00, 0x5, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/reflect/value.go:345 +0x9d
github.com/StackExchange/tsaf/expr.(*state).walkFunc(0xc21013e080, 0xc2100ae070, 0x93fa60, 0xc2101b7b40, 0x101, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:342 +0x548
github.com/StackExchange/tsaf/expr.(*state).walkFunc(0xc21013e080, 0xc2100ae000, 0x93fa60, 0xc2101b7b40, 0x41501, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:336 +0x195
github.com/StackExchange/tsaf/expr.(*state).walk(0xc21013e080, 0x935ce8, 0xc2100ae000, 0x93fa60, 0xc2101b7b40, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:173 +0x16a
github.com/StackExchange/tsaf/expr.func·001(0x93fa60, 0xc2101b7b40)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:53 +0x63
github.com/MiniProfiler/go/miniprofiler.(*Timing).Step(0xc2101b79c0, 0x3d6670, 0xc, 0xc21013e0a0)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/types.go:197 +0x105
github.com/MiniProfiler/go/miniprofiler.(*Profile).Step(0xc210050bd0, 0x3d6670, 0xc, 0xc21013e0a0)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/types.go:176 +0x55
github.com/StackExchange/tsaf/expr.(*Expr).Execute(0xc2100c8be0, 0x93fa38, 0xc2101b8770, 0x935c70, 0xc210050bd0, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/expr.go:54 +0x182
github.com/StackExchange/tsaf/web.Expr(0x935c70, 0xc210050bd0, 0x935c38, 0xc2101bf320, 0xc2101b91a0, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/web/expr.go:16 +0x11a
github.com/StackExchange/tsaf/web.func·002(0x935c70, 0xc210050bd0, 0x935c38, 0xc2101bf320, 0xc2101b91a0)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/web/web.go:77 +0x56
github.com/MiniProfiler/go/miniprofiler.Handler.ProfileRequest(0xc2100b01c0, 0xc210050bd0, 0x935c38, 0xc2101bf320, 0xc2101b91a0)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/miniprofiler.go:292 +0xb7
github.com/MiniProfiler/go/miniprofiler.Handler.ServeHTTP(0xc2100b01c0, 0x0, 0x935c38, 0xc2101bf320, 0xc2101b91a0)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/miniprofiler.go:286 +0x8c
github.com/MiniProfiler/go/miniprofiler.(*Handler).ServeHTTP(0xc2100b01d0, 0x935c38, 0xc2101bf320, 0xc2101b91a0)
/Users/kbrandt/src/go/src/github.com/MiniProfiler/go/miniprofiler/doc.go:1 +0xae
github.com/gorilla/mux.(*Router).ServeHTTP(0xc2100a1460, 0x935c38, 0xc2101bf320, 0xc2101b91a0)
/Users/kbrandt/src/go/src/github.com/gorilla/mux/mux.go:98 +0x217
net/http.(*ServeMux).ServeHTTP(0xc21000a660, 0x935c38, 0xc2101bf320, 0xc2101b91a0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1496 +0x163
net/http.serverHandler.ServeHTTP(0xc2100a1c80, 0x935c38, 0xc2101bf320, 0xc2101b91a0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1597 +0x16e
net/http.(*conn).serve(0xc21007dc80)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1167 +0x7b7
created by net/http.(*Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
Support Alert Definitions on the web interface
It would be very useful to be able to put alerts rules that support local and global variable substitution in the expression page. In that way users can define alerts on the expression page, and then copy them directly into the config.
Another nice feature for this page would be able to set the time of "now". That way we can see how the alert would have behaved in previous situations.
No progress indication on long graph load
When you load a graph with a lot of data, there is no visual indicator that the page is working. Since it is loaded through JS you don't see the spinner on the tab. Some sort of "working" animation would be useful so you know the page has not crashed/timed out/just not working.
Able to set the color of the graph line from the graph interface?
Bart is colorblind. He probably would like to be able to configure what colors his graphs show in.
When auto-downsample is selected, window/downsample options should be greyed out
I found myself getting confused when I tried to set a window/downsample setting. I was changing the settings but the graph wasn't changing. Kyle alerted me that if "Auto Downsample" is checked, it overrides all settings in window/downsample. It seems like this is a usability problem because frankly I didn't even see that Auto Downsample checkbox let alone look at its tooltip. It should be more obvious that the window settings have no effect if that default checkbox is checked.
Alert Remains on Dashboard after Adding Squelch
alert os.high_cpu {
template = generic
$q = avg(q("avg:rate:os.cpu{host=*}", $default_time, ""))
warn = $q > 80
crit = $q >= 100
critNotification = default
#back01 incorrectly reports cpu in proc/uptime, maybe old centos bug...
squelch = host=ny-back01
}
Strange Subject Value
Not sure where that number is coming from....
Allow selection across x axis to "zoom in" on a time range
When you are looking at graphs that show a dip in the time series data, my tendency is to want to know exactly when that dip occurs. It is a trial and error process to select start/end dates to get that exact time. It would be considerably easier to be able to click-and-drag across the x axis to select a time frame.
Prior art is in status.se. You can select from the graph to tighten the date range in that interface. Nick Craver mentioned it is pretty easy to do in d3:
Nick Craver
1953860 9:45:39
if curious, grep opserver js for "brush"
Squelch doesn't seem to work as expected
I have the following:
alert os.diskspace {
template = generic
$q = avg(q("avg:os.disk.fs.percent_free{host=*,disk=*}", "30m"))
#crit is 5 here because once Linux reaches 5% free space than not-root users can no longer
# write to the disk
squelch = disk=/media.*
squelch = disk=/home/gbeech.*
crit = $q <= 5
warn = $q < 15
critNotification = default
}
Yet I still see:
I'm sure if multiple squelch lines are supported, but I think they need to be since that is the only way I can think of to express OR operations currently. Even if they are not supported, it should error as an invalid config.
Template Prototyping
Much like the "Rule" Page, it would be useful to have a way to rapidly prototype alert templates. Currently the workflow is:
- Hope that alert is actually triggered at the moment so you can see it
- Edit the conf
- Restart a local instance of tsaf, wait for it to trigger
- Look at the dashboard, look at your email inbox
I'm sure if this should be the rule page or a different page. But I would like to be able to define a template with an alert on a page, and get a preview of the email and subject. In order to see non-triggered alerts, it might be nice to falsely trigger an alert. But maybe just being able to edit the alert "criteria" would be sufficient.
Alert doesn't clear
I deleted ny-statuswolf01 a couple of days ago. So it seems like alerts only clear when the the evaluates to a non-critical state.
This behavior might be okay if we introduce the unknown state. I'm not sure what the best thing to do is right now, just wanted to capture this issue.
Also, I'm not sure where the 20.422... value is coming from, maybe it was the most recent value before the alert could not longer be evaluated?
Favicon
Sure would be nice if TSAF had a favicon so I could distinguish it better among my dozens of open tabs. Obviously low priority, but a nice to have.
One click for Counter Rates
Having to check Rate, Counter, and set Counter Reset to 1 is not a particularly intuitive way to select a rate counter. Having a single checkbox which automatically populates all three fields, similar to how Auto Downsample works, would be nice.
I've already talked to Jibson about it, but figured I would formalize it here so it doesn't get lost in time.
Do we need the extra ()s?
sum(q("sum:rate{counter,,1}:linux.proc.cpu{host=ny-git01,name=scollector}", "2d", "") > .2)
Needs to be:
sum((q("sum:rate{counter,,1}:linux.proc.cpu{host=ny-git01,name=scollector}", "2d", "") > .2))
Draft Proposal: Vertical Reducers
While this is "Draft" I'm just gather my thoughts, don't consider a complete idea.
Lets say I want to alert on the number of processes of a certain type on a host:
count(q("linux.proc.mem{host=,name=apache,id=}", "10m") > 0) > 100
Number of down servers in an haproxy backend (but in this case I really want to GROUP BY pxname):
count(q("haproxy.server.check_status{pxname=,svname=}" > 0)
Panic when $q is a series in something like `warn = $q > 60`
The following rule is invalid because I'm comparing to series:
alert haproxy.server.error_connections {
template = generic
$metric = "sum:rate:haproxy.server.econ"
$q = q($metric, $default_time)
warn = $q > 0
warnNotification = default
}
However, the rule causes TSAF to panic instead of an error:
panic: interface conversion: expr.Value is expr.Series, not expr.Number
goroutine 29 [running]:
runtime.panic(0x346620, 0xc21193f040)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:266 +0xb6
github.com/StackExchange/tsaf/sched.(*Schedule).CheckExpr(0x7df540, 0xc2100b9210, 0xc210000aa8, 0x0, 0x0, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:300 +0x536
github.com/StackExchange/tsaf/sched.(*Schedule).CheckAlert(0x7df540, 0xc2100b9210)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:267 +0x91
github.com/StackExchange/tsaf/sched.(*Schedule).Check(0x7df540)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:229 +0x185
github.com/StackExchange/tsaf/sched.(*Schedule).Run(0x7df540, 0x0, 0x0)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:166 +0x1be
github.com/StackExchange/tsaf/sched.Run(0x0, 0x0)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:62 +0x2a
main.func·003()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:36 +0x1e
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:36 +0x21e
goroutine 1 [select (no cases)]:
main.main()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:38 +0x233
goroutine 3 [chan receive]:
github.com/StackExchange/tsaf/search.Process(0xc210037180)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/search/search.go:82 +0x3a
created by github.com/StackExchange/tsaf/search.init·1
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/search/search.go:64 +0x31
goroutine 37 [runnable]:
net/http.(*persistConn).readLoop(0xc2100b2c80)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:778 +0x68f
created by net/http.(*Transport).dialConn
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:528 +0x607
goroutine 27 [IO wait]:
net.runtime_pollWait(0x939be8, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(*pollDesc).Wait(0xc2100c4e60, 0x72, 0x938128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(*pollDesc).WaitRead(0xc2100c4e60, 0x23, 0x938128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(*netFD).accept(0xc2100c4e00, 0x46d488, 0x0, 0x938128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:382 +0x2c2
net.(*TCPListener).AcceptTCP(0xc210000cc0, 0x18, 0xc2100c2810, 0x1179c3)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(*TCPListener).Accept(0xc210000cc0, 0x938bf0, 0xc2100b6870, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(*Server).Serve(0xc2100b3aa0, 0x938c48, 0xc210000cc0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1622 +0x91
net/http.(*Server).ListenAndServe(0xc2100b3aa0, 0xc2100b3aa0, 0x938bc8)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1612 +0xa0
net/http.ListenAndServe(0x3a8fa0, 0x5, 0x938bc8, 0xc2100b6750, 0xc2100bf490, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1677 +0x6d
github.com/StackExchange/tsaf/relay.RelayHTTP(0x3a8fa0, 0x5, 0xc2100ac180, 0x26, 0x1, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/relay/relay.go:20 +0x2c7
main.func·001()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:34 +0x5b
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:34 +0x1e3
goroutine 28 [IO wait]:
net.runtime_pollWait(0x939b40, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(*pollDesc).Wait(0xc2100d00d0, 0x72, 0x938128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(*pollDesc).WaitRead(0xc2100d00d0, 0x23, 0x938128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(*netFD).accept(0xc2100d0070, 0x46d488, 0x0, 0x938128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:382 +0x2c2
net.(*TCPListener).AcceptTCP(0xc210000610, 0x18, 0xc21006d010, 0x1179c3)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(*TCPListener).Accept(0xc210000610, 0x938bf0, 0xc21006fab0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(*Server).Serve(0xc2100b35a0, 0x938c48, 0xc210000610, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1622 +0x91
net/http.(*Server).ListenAndServe(0xc2100b35a0, 0xc2100b35a0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1612 +0xa0
net/http.ListenAndServe(0xc2100004e8, 0x5, 0x0, 0x0, 0xc2100b0740, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1677 +0x6d
github.com/StackExchange/tsaf/web.Listen(0xc2100004e8, 0x5, 0x3c4480, 0x3, 0xc2100ac180, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/web/web.go:48 +0x8ca
main.func·002()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:35 +0x78
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:35 +0x210
goroutine 38 [select]:
net/http.(*persistConn).writeLoop(0xc2100b2c80)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:791 +0x271
created by net/http.(*Transport).dialConn
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/transport.go:529 +0x61e
goroutine 32 [sleep]:
time.Sleep(0xdf8475800)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/time.goc:31 +0x31
github.com/StackExchange/tsaf/sched.func·001()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:151 +0x35
created by github.com/StackExchange/tsaf/sched.(*Schedule).Run
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:154 +0x6e
goroutine 33 [runnable]:
sync.runtime_Semacquire(0x7df544)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/sema.goc:199 +0x30
sync.(*Mutex).Lock(0x7df540)
/usr/local/Cellar/go/1.2/libexec/src/pkg/sync/mutex.go:66 +0xd6
github.com/StackExchange/tsaf/sched.(*Schedule).CheckNotifications(0x7df540, 0x0)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:187 +0x4c
github.com/StackExchange/tsaf/sched.(*Schedule).Poll(0x7df540)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:181 +0x80
created by github.com/StackExchange/tsaf/sched.(*Schedule).Run
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:156 +0xaa
goroutine 36 [syscall]:
runtime.goexit()
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/proc.c:1394
exit status 2
Dashboard shows rule that doesn't exist in the config
Is no where in my current prod.conf
Support Boolean Operations for Non-Union Queries
It seems right now that we still can't do one of our canonical alert example: Network bandwidth is high, and redis is not replicating:
I imagine something like:
$redis.replication = avg(q("max:redis.is_replicating{host=ny-redis*}", "1m")) < 1
$high.router.bandwidth = avg(q("rate:os.net.bytes{host=ny-edge*,iface=GigabitEthernet0/0}", "1m") * 8 > 160,000,000
crit = $high.router.bandwidth && ! $redis.replication
However I think we currently can't do this since there is no union between the tag sets. Maybe something like brackets could mean "evaluate this expression independently":
$redis.replication = [avg(q("max:redis.is_replicating{host=ny-redis*}", "1m")) < 1]
$high.router.bandwidth = [avg(q("rate:os.net.bytes{host=ny-edge*,iface=GigabitEthernet0/0}", "1m") * 8 > 160,000,000]
crit = $high.router.bandwidth && ! $redis.replication
No visual indication that alerts on main page are clickable
There is not visual clue that the alerts on the main page are clickable. There should be some sort of visual clue - changed curser (not an I bar or normal pointer), show/hide arrows or just text along the lines of "click for more detail"
overrides don't do anything
Care about counter reset and max
Our WMI counters are uint64s, but opentsdb only accepts int64s. Thus, our big counters could roll over. We should convert these data to int64s so opentsdb doesn't start throwing errors at us, and, since the conversion could render them negative, should start to use the counter max values.
Panic when Calling .E in the template Body
When I try to call .E in the template body with something like the following, I get a panic:
template diskspace {
body = `<p><a href="{{.Ack}}">Acknowledge alert</a>
<p>Host: {{.Group.host}}
<p>Disk: {{.Group.disk}}
<p>Percent Free: {{.Alert.Vars.q | .E}}
Panic:
panic: interface conversion: parse.Node is *parse.FieldNode, not *parse.IdentifierNode
goroutine 411 [running]:
runtime.panic(0x343680, 0xc210dbc280)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:266 +0xb6
html/template.ensurePipelineContains(0xc21009dd70, 0xc210dcce40, 0x1, 0x3)
/usr/local/Cellar/go/1.2/libexec/src/pkg/html/template/escape.go:233 +0x1ae
html/template.(*escaper).commit(0xc210dbc180)
/usr/local/Cellar/go/1.2/libexec/src/pkg/html/template/escape.go:737 +0x23e
html/template.escapeTemplates(0xc2100ae620, 0x9d0da8, 0x1, 0x1, 0x20, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/html/template/escape.go:46 +0x3c3
html/template.(*Template).escape(0xc2100ae620, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/html/template/template.go:55 +0x100
html/template.(*Template).Execute(0xc2100ae620, 0x931e60, 0xc210bd1af0, 0x379100, 0xc21079c0c0, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/html/template/template.go:66 +0x2c
github.com/StackExchange/tsaf/sched.(*Schedule).ExecuteBody(0x7d8540, 0x931e60, 0xc210bd1af0, 0xc2100bc210, 0xc210511c40, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/template.go:116 +0xc5
github.com/StackExchange/tsaf/sched.(*Schedule).Email(0x7d8540, 0xc2100bc210, 0xc2100b71e0, 0xc210511c40)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/mail.go:18 +0xa0
created by github.com/StackExchange/tsaf/sched.(*Schedule).Notify
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/sched/sched.go:313 +0x62
goroutine 1 [select (no cases)]:
main.main()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:38 +0x233
goroutine 3 [chan receive]:
github.com/StackExchange/tsaf/search.Process(0xc210038180)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/search/search.go:82 +0x3a
created by github.com/StackExchange/tsaf/search.init·1
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/search/search.go:64 +0x31
goroutine 25 [IO wait]:
net.runtime_pollWait(0x931b40, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(*pollDesc).Wait(0xc2100d81b0, 0x72, 0x930128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(*pollDesc).WaitRead(0xc2100d81b0, 0x23, 0x930128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(*netFD).accept(0xc2100d8150, 0x469ba8, 0x0, 0x930128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:382 +0x2c2
net.(*TCPListener).AcceptTCP(0xc2100002d0, 0x18, 0xc210070010, 0x115433)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(*TCPListener).Accept(0xc2100002d0, 0x930bf0, 0xc2100bbb70, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(*Server).Serve(0xc2100b6f00, 0x930c48, 0xc2100002d0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1622 +0x91
net/http.(*Server).ListenAndServe(0xc2100b6f00, 0xc2100b6f00, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1612 +0xa0
net/http.ListenAndServe(0xc210000468, 0x5, 0x0, 0x0, 0xc2100c2120, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1677 +0x6d
github.com/StackExchange/tsaf/web.Listen(0xc210000468, 0x5, 0x3c11e0, 0x3, 0xc2100ad210, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/web/web.go:47 +0x883
main.func·002()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:35 +0x78
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:35 +0x210
goroutine 24 [IO wait]:
net.runtime_pollWait(0x931be8, 0x72, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(*pollDesc).Wait(0xc2100c6760, 0x72, 0x930128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(*pollDesc).WaitRead(0xc2100c6760, 0x23, 0x930128)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(*netFD).accept(0xc2100c6700, 0x469ba8, 0x0, 0x930128, 0x23)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/fd_unix.go:382 +0x2c2
net.(*TCPListener).AcceptTCP(0xc210000af0, 0x11565b, 0x988d90, 0x11565b)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(*TCPListener).Accept(0xc210000af0, 0x931f28, 0xc210b7f738, 0xc210b99180, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(*Server).Serve(0xc2100b67d0, 0x930c48, 0xc210000af0, 0x0, 0x0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1622 +0x91
net/http.(*Server).ListenAndServe(0xc2100b67d0, 0xc2100b67d0, 0x930bc8)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1612 +0xa0
net/http.ListenAndServe(0x3a5e00, 0x5, 0x930bc8, 0xc2100c9120, 0xc2100c2310, ...)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1677 +0x6d
github.com/StackExchange/tsaf/relay.RelayHTTP(0x3a5e00, 0x5, 0xc2100ad210, 0x26, 0x1, ...)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/relay/relay.go:20 +0x2c7
main.func·001()
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:34 +0x5b
created by main.main
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/main.go:34 +0x1e3
goroutine 26 [chan receive]:
github.com/StackExchange/tsaf/expr/parse.(*Tree).peek(0xc210bd1a80, 0x0, 0x7fc160, 0x0, 0x1)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/parse/parse.go:85 +0xab
github.com/StackExchange/tsaf/expr/parse.(*Tree).F(0xc210bd1a80, 0x1, 0x1000d29e8)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/parse/parse.go:258 +0x32
github.com/StackExchange/tsaf/expr/parse.(*Tree).M(0xc210bd1a80, 0x35, 0x65e2d)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/parse/parse.go:246 +0x32
github.com/StackExchange/tsaf/expr/parse.(*Tree).P(0xc210bd1a80, 0x4, 0x1)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/parse/parse.go:234 +0x32
github.com/StackExchange/tsaf/expr/parse.(*Tree).C(0xc210bd1a80, 0x985ce8, 0xc210d9dc30)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/parse/parse.go:222 +0x32
github.com/StackExchange/tsaf/expr/parse.(*Tree).A(0xc210bd1a80, 0xcb27, 0xc210010000)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/parse/parse.go:210 +0x32
github.com/StackExchange/tsaf/expr/parse.(*Tree).O(0xc210bd1a80, 0x8, 0x465a58)
/Users/kbrandt/src/go/src/github.com/StackExchange/tsaf/expr/parse/parse.go:198 +0x32
github.com/StackExchange/tsaf/expr/parse.(*T
Rule crashes web browsers
The rule text is:
$t = "5m"
$cluster_tag=pxname
$node_tag=svname
$metric=haproxy.server.stot
$cluster_metric = "sum:rate:$metric{$cluster_tag=*}"
$node_metric = "sum:rate:$metric{$cluster_tag=*,$node_tag=*}"
$cluster_total = avg(q($cluster_metric, $t, ""))
$node_total = avg(q($node_metric, $t, ""))
$node_perc = 1/len(t(avg(q($node_metric, $t, "")), "cluster"))
$q = abs($node_perc - ($node_total / $cluster_total)) > $node_perc * .5
crit = $q
This isn't entirely surprising because there are a lot of pxname svname combinations:
Link to TSAF graphs such that they produce static images of the chart in question
Internally at SE I can see many opportunities where we'd like to link the graph from tsaf into chat quickly. It would be awesome if we could have a url that triggered oneboxing of an image of the graph.
Incorrect parsing order with -
-8 > 0 incorrectly applies - to 8 > 0 instead of 8.
Font on scale sides does not scale font-size based on size of number
For my project, I am dealing with numbers that are between 0-5, but the majority of the numbers lay between 4 and 4.5. When the scale shows up, it shows up in like ten-millionths of a unit and the scale number is masked.
Add Lookup Functions so the same alert can have different values
We wanted to wait until there was a use case, so here is one:
q("avg:os.disk.fs.percent_free{disk=/,host=nyhq-netflow01}", "30m")
nyhq-netflow01 is below our warning threshold as a percentage. But since it seems to be stable, it would make sense to adjust the threshold to something lower.
Can't use Durations That are Greater than signed 32bit Int in MS
It would seem that opentsdb doesn't like large ms values:
q("avg:os.disk.fs.space_used{disk=/,host=ny-puppetdash01.ds.stackexchange.com}","2147483647ms") works
ungroup and t don't error on panic
ungroup(5) should display an error on the website.
Support "Unknown" for alert definitions
Having to create rules that utilize the since function adds a lot of configuration, since you likely need at least one for each other rule (and possibly more).
I propose we consider doing the following instead:
alert os.high_cpu {
template = generic
$q = avg(q("avg:rate:os.cpu{host=*}", "5m"))
warn = $q > 80
crit = $q >= 100
unknown = "1m"
critNotification = default
}
What unknown would do in effect is automatically call since on every q function defined in the alert, and set the state of the alert to unknown if since's return value is greater than the value of unknown ("1m") in this example.
A Crit Warning is being reported as Normal in the Web Summary for some reason
The rule is:
$default_time = "2m"
alert os.high_cpu {
template = generic
$time = "1m"
$q = avg(q("avg:rate:os.cpu{host=*}", $default_time))
warn = $q > 80
crit = $q >= 100
critNotification = default
}
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.