Unable to bring up nginx after adding the kafka module

[brg-liuwei]

Unable to bring up nginx after adding the kafka module

[divgwd]

Finally got it up and running :)
Thank you :)

[brg-liuwei]

Seems like you hadn't compiled ngx_kafka_module into nginx.
Try the following command:
ldd /your/path/to/nginx | grep librdkafka
Is there any output?

[divgwd]

Below is the output
[root@smsc ~]# ldd /usr/local/nginx/sbin/nginx |grep librdkafka
librdkafka.so.1 => not found
[root@smsc ~]# locate librdkafka.so.1
/opt/librdkafka/src/librdkafka.so.1
/usr/local/lib/librdkafka.so.1
so I copied the binary to /lib64/
[root@smsc lib64]# cp /usr/local/lib/librdkafka.so.1 .
[root@smsc lib64]# ldd /usr/local/nginx/sbin/nginx |grep librdkafka
librdkafka.so.1 => /lib64/librdkafka.so.1 (0x00007f64bbb46000)
but even after that when I restart i get the below error
[root@smsc lib64]# service nginx restart
nginx: [emerg] unknown directive "kafka" in /etc/nginx/nginx.conf:37
nginx: configuration file /etc/nginx/nginx.conf test failed
[root@smsc lib64]#
[root@smsc lib64]#
[root@smsc lib64]# ldd /usr/local/nginx/sbin/nginx |grep librdkafka
librdkafka.so.1 => /lib64/librdkafka.so.1 (0x00007f3a77c4b000)
[root@smsc lib64]#

[brg-liuwei]

seems like you have more than one nginx installed in your system:
/usr/local/nginx/sbin/nginx and /usr/sbin/nginx
make sure which nginx is used when executing service nginx restart.
Maybe you should try /usr/local/nginx/sbin/nginx -t

[divgwd]

Thanks you were right my system has two nginx installed.
I am getting the below error after restarting with the right instance if nginx.
[root@smsc lib64]# service nginx restart
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Stopping nginx: [FAILED]
grep: unrecognized option '--add-module=/opt/ngx_kafka_module'
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
useradd: unrecognized option '--add-module=/opt/ngx_kafka_module'

[brg-liuwei]

Oh, it seems like there is something wrong with your service scripts (/etc/init.d/nginx)
Please do those operations as follows to check if your nginx is worked:

sudo killall nginx
/usr/local/nginx/sbin/nginx -c /etc/nginx/nginx.conf -t
If test ok:
/usr/local/nginx/sbin/nginx -c /etc/nginx/nginx.conf

If it works, It is means that you should fix some errors in /etc/init.d/nginx.

[divgwd]

Hi,
I am unable to send msgs to kafka via the curl command
curl http://172.16.23.17:8000/alarmdata/kafkatopic/ -d "message send to kafka topic"
[root@smsc ~]# netstat -anlp|grep nginx
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 24346/nginx
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 24346/nginx
tcp 0 0 172.16.23.17:38532 172.16.23.15:9092 ESTABLISHED 24347/nginx
unix 3 [ ] STREAM CONNECTED 196746843 24346/nginx
unix 3 [ ] STREAM CONNECTED 196746842 24346/nginx

Do we have to specify a different port for nginx's kafka module ??

[brg-liuwei]

Oh, It looks like a FAQ, check config/server.properties and nginx.conf again.

Make sure the value of host.name in config/server.properties equals to the hostname of kafka_broker_list in nginx.conf.

Many users make wrong configuration as follows:

file config/server.properties:
host.name=localhost

file nginx.conf:
kafka_broker_list 127.0.0.1:9092;

With this context, kafka could not recv msgs from nginx, because the resolver of kafka cannot resolve localhost to 127.0.0.1 successfully.

I read nginx.txt you attached upon, you set kafka_broker_list 172.16.23.15:9092;. So, plz check file /path/to/kafka/config/server.properties in 172.16.23.15, have you set host.name=172.16.23.15 ?

[divgwd]

Hi,
can we send the data to a particular Id of the kafka topic??

[brg-liuwei]

For now, we send data to kafka topic for automatic partitioning using the topic's partitioner function.
However, I think this requirement is rather reasonable. So, I am adding a directive named kafka_partition as soon as possiable.
Maybe you will see this feature tomorrow.

[brg-liuwei]

can we send the data to a particular Id of the kafka topic?

@divgwd A new directive kafka_partition is added. Does it satisfies your requirement?

[divgwd]

Sorry was caught up with other issues...I wanted know if we could insert the data to kafka after routing it thru haproxy??

[brg-liuwei]

Of course you can

[divgwd]

so I set
include /etc/nginx/conf.d/*.conf;
kafka;

kafka_broker_list <I.P>:8086 ; # host:port ...

8086 is the port my haproxy is binding to

[brg-liuwei]

No, that's wrong, absolutely. The right architecture should be:

HAProxy -> Nginx -> Kafka

I'm really confused why you set HAProxy on the front of Kafka.

[divgwd]

we were pushing netdata's data from the site servers to the local kafka installation we had nginx set up on one of the site servers which has a public IP which intern routes it to haproxy installation on one of our local servers which has a public IP .

[brg-liuwei]

8086 is the port my haproxy is binding to

The kafka_broker_list <ip>:<port> set in nginx.conf must be the port kafka is binding to

[divgwd]

Thank you!!
I changed the proxy server from HAproxy to Nginx now.
the set up is now
kafka<Nginx_with_kafka<Nginx<list_of_netdata_servers
from
kafka<HAproxy<Nginx_with_kafka<list_of_netdata_servers