Would gogstash silent fail in elastic output?

[tsaikd]

Would gogstash silent fail in elastic output?

[tengattack]

Oh I noticed that action changed in master branch.
It start using elastic's bulk API to make output.

However, would it silent fail in latest code, because I haven't find out the error handling for the elastic's BulkProcessor
https://github.com/olivere/elastic/blob/a487785d708c8e6a99b8d498947c220511b6ec7f/bulk_processor.go#L524

[tsaikd]

Thanks for the feedback.

olivere/elastic#666 add an interface Backoff to provide customized error handling. I will push a new patch to fix the retry issue in a few days later.

[tengattack]

Great!
BTW, could we add a backoff maximum time config for elastic output? So that I can customized the backoff action. Currently, It can only retry about 5 times until it reach the maximum time (200ms to 10000ms).

[tsaikd]

	// RetryTimes when write to elastic failed, how many times to retry.
	// Defaults to 3 and can be set to -1 to be disabled.
	RetryTimes int `json:"retry_times,omitempty"`

	// RetryInterval retry interval
	// Defaults to 10 seconds
	RetryInterval time.Duration `json:"retry_interval_ms,omitempty"`

the config above is enough?

[tengattack]

Does it set the elastic's backoff type? Maybe we can find a way to specify the backoff type:
https://github.com/olivere/elastic/blob/a7f6620ddddaaccb6fa041eaa281182fd4b4fcc4/backoff.go

I prefer to use the default exponential backoff :)

[tsaikd]

There are no stop condition in ConstantBackoff and ExponentialBackoff. I mean it will retry forever, until memory full or app crash. But maybe it make sense for logging scenario. What about this?

	// ExponentialBackoffInitialTimeout used to set the first/minimal interval in elastic.ExponentialBackoff
	// Defaults to 10 seconds
	ExponentialBackoffInitialTimeout time.Duration `json:"exponential_backoff_initial_timeout,omitempty"`

	// ExponentialBackoffMaxTimeout used to set the maximum wait interval in elastic.ExponentialBackoff
	// Defaults to 300 seconds
	ExponentialBackoffMaxTimeout time.Duration `json:"exponential_backoff_initial_timeout,omitempty"`

[tengattack]

Ummm, I think ExponentialBackoff will stop when it reach the maximum timeout:
https://github.com/olivere/elastic/blob/a7f6620ddddaaccb6fa041eaa281182fd4b4fcc4/backoff.go#L91

But that's work! Thank you!

[tengattack]

BTW, the key of maximum timeout wrong?

[tengattack]

And we can add SetErrorLog for the elastic client or After callback for the BulkRequest so that it will not silent fail?

[tengattack]

BTW, does the type time.Duration support some kind of writing like "5s" or "1h"

[tsaikd]

done

[tengattack]

If we just connect the SetErrorLog to config.Logger 24a6a2d does the error logs will send to stdout instead of stderr? Maybe we need to make a Printf to Errorf redirect?

[tsaikd]

I create a new error logger for it.

[tengattack]

oh! Beautiful!
Super thanks!

[tengattack]

BTW, Logger's Out changed to Stdout, does it affect the config.Logger.Errorf output?

[tsaikd]

No, Logger always output to stdout, ErrorLogger always output to stderr, no matter what's the function call.

[tengattack]

Oh I see

[tengattack]

Last question, as I trying to find out github.com/sirupsen/logrus's code, I found the owner name (sirupsen) is lowercase initials, shall we correct it?
a8ad508#diff-beaa11a399561a57ec0d693757f6afb1R6

[tsaikd]

Thanks.
Replaced with lowercase and remember to update dependent lib before rebuilding.

[tengattack]

Cheers!