Decouple open-nti.py from the database

[juniper]

Decouple open-nti.py from the database

[mmellin]

As an fyi - We noticed influx gets pretty slow when sending lots of syslog
to it via Unix logger so we built out logstash with Kafka instead of using
fluentd.

Matt

On Aug 25, 2016 10:42 PM, "Damien" [email protected] wrote:

Hi
We've been talking about it for a while,
I think it will be great to decouple the netconf collector (open-nti.py)
from the Influxdb.
The idea would be to send data to Fluentd or Telegraf (format to be
defined) to leverage their plugin architecture. Both have the ability to
execute external script periodically to accept plugin in any languages.

Anil has started to look into this and we realized that with the current
implementation it's gonna be challenging because of how the delta
computation is done today.
Is this part still mandatory ?
Do we know if Influxdb has improved on this point ? it would be great to
get rid of this part if possible

Would love to get everyone's feedback on that.
Damien

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Juniper/open-nti/issues/81, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABW46dz1s6yHp0tNcQDvXW-Ivpm7kmoSks5qjny_gaJpZM4Jtwfm
.

[dgarros]

@mmellin interesting, would love to see what you guys have built.

[cookiesowns]

Any luck on this? We've recently built a stack using InfluxDB, Grafana/Chronograf & Telegraf/Graphite for monitoring our servers / networking devices.

The last piece of our puzzle is integrating our JunOS devices into the network for monitoring without resorting to hacky solutions with SNMP OID's and SNMP input plugins on Telegraf..

Even for now it would be great if additional documentation on open-nti.py could be provided on how one could leverage this within an existing InfluxDB deployment

[dgarros]

Hi @cookiesowns

I pushed recently an alpha version of the NETCONF collector that can work in standalone mode.
https://github.com/Juniper/py-netconf-collector

It's still early but you are welcome to give it a try.
it's based on the current open-nti.py script

To summarize the changes I made:

• Converted the code into an object oriented model … now we have 2 objects “parser_manager” and “netconf_collector”
• Add “type” in parser and cleanup the regex-command
• Remove all dependancies to influxdb, currently the collector is optimized to run with Telegraf but it would be easy to add support for other format/tool

the migration to an object oriented solution has a lot of benefits, it become much easier to maintain, test and add features.

I proposed a slight change of format of the parsers and we are still haven't agreed on it with the rest of the team so for now, the number of parsers is very limited.
We should probably move the discussion to github to collect everyone's feedback.

Least but not least, in // I'm working on integrating this new collector within OpenNTI. the changes are part of the branch consul

Let me know if you need help to get started

Damien

cc/ @3fr61n @psagrera