ttl for elasticsearch indexes

[uken]

ttl for elasticsearch indexes

[grusy]

You have to manage the ttl by adjusting the mapping of the index.
Either set a default ttl or send your custom ttl with each record and map the field against it.

[svartalf]

Yeah, I got it, elastichsearch doesn't have a ttl for indexes, just for a documents.

I'm talking about a some mechanism like a logrotate, it can be useful here.
That plugin with a logstash-like format creates indexes for each day, like a logstash-2013.12.14 … logstash-2013.12.16, so in a few weeks there will be a lot of indexes.

As I understand, fluentd plugins system doesn't have any hooks for a periodic called routines, so it will be problematic to remove too old indexes from a plugin.

[grusy]

Ok.
Have a look at this: https://github.com/imperialwicket/elasticsearch-logstash-index-mgmt
You can also use the logstash_dateformat option to create weekly or monthly indexes instead of the daily ones.

[pitr]

That looks good, @grusy, thanks! @svartalf usage of ttl on individual records is discouraged, see sections "number of indices" and "rotate and optimize " at http://www.elasticsearch.org/tutorials/using-elasticsearch-for-logs/