erlio/vmq_webhooks

[OBSOLETE since VerneMQ 1.0.0rc1 - now part of the main repo] VerneMQ Webhooks - Implement VerneMQ plugins as http endpoints

VerneMQ Webhooks

This repo is obsolete as the vmq_webhooks is now part of the main VerneMQ repo: https://github.com/erlio/vernemq.

vmq_webhooks's Issues

auth_on_register username/password are "undefined"

Username/password parameter in auth_on_register hook are strings with value "undefined" if connecting client doesn't send username/password.

How can I distinguish between client send nothing and client really send "undefined" as username/password? Maybe a empty username/password should be represented by empty string or false.

(see vmq_diversity#5)

auth_on_subscribe, auth_on_publish not being called.

Hi,
After a quick test following the README, it seems like neither auth_on_subscribe nor auth_on_publish are being called. I am using Paho.MQTT.Client.

client = new Paho.MQTT.Client(location.hostname, 8083, "web_" + parseInt(Math.random() * 100, 10));
client.subscribe('test_topic');

When VerneMQ triggers event "auth_on_subscribe" and "auth_on_publish"

I tested vmq_webhooks with "auth_on_register", "on_publish". It works well. Thank you very much for the great product.

However, I need to verify the topics before allowing clients subscribe/publish them. I tried:

vmq-admin webhooks register hook=auth_on_subscribe endpoint="http://localhost:3001/api/v1/mqtt/auth_on_subscribe" 

This command returned successfully

After that, I wrote a MQTT client to connect and subscribe/publish a topic on MQTT server. The client works, but we did not received event "auth_on_subscribe" in webhooks.

Did I make something wrong? please help.

Webhooks Compilation Failure

Hi,

We are trying to compile the webhooks to include in vmq. We've been getting the following error:

===> Verifying dependencies...
===> Compiling vmq_webhooks
===> Compiling src/vmq_webhooks_cli.erl failed
src/vmq_webhooks_cli.erl:38: syntax error before: '{'
src/vmq_webhooks_cli.erl:70: syntax error before: '{'

src/vmq_webhooks_cli.erl:31: function b64opt/1 undefined
src/vmq_webhooks_cli.erl:53: function get_opts/1 undefined

Can you please advise us on how to proceed ?

Thanks,
Karthik

Payload format

Hi;
The messages we use are all binary ( protobuf v3 ).
the on_publish and on_deliver hooks presents the payload in the Post request as a json field "payload".
How this will work with binary payloads ?
( I sent a similar questionn to the main repository )

Thanks

Persisting web hooks across broker restarts

Hi,

we have some problems in persisting webhooks across broker restarts. In the documentation there are two files mentioned which can be used to configure persistent webhooks: vernemq.conf and vmq_webhooks.conf. Which one is the right one or are they both needed?
And another question: how can we then disable acl and pwd persistently?

Thanks in advance for your help

Regards
Matthias

"Auth on Publish" fails repeatly

I am using vmq_webhooks plugin to call external authentication with three hooks as below:

• "auth on register" -> it work properly
• "auth on subscriber" -> it work properly
• "auth on publish" -> It work, but the connection is not disconnected. After receiving "result": { "error": "some error message" } from external authentication service, vmq_webhooks continuously call the authentication. So I got the logs as below

2016-09-30 15:29:40.059 [error] <0.3385.2>@vmq_mqtt_fsm:auth_on_publish:631 can't auth publish [undefined,{[],<<"mqttjs_e73dea9a">>},0,[<<"presence">>],<<"Hello mqtt">>,false] due to [error]
2016-09-30 15:29:40.059 [warning] <0.3385.2>@vmq_ranch:teardown:115 [<0.3385.2>] session stopped abnormally due to 'publish_not_authorized_3_1_1'
2016-09-30 15:29:41.083 [error] <0.3389.2>@vmq_mqtt_fsm:auth_on_publish:631 can't auth publish [undefined,{[],<<"mqttjs_e73dea9a">>},0,[<<"presence">>],<<"Hello mqtt">>,false] due to [error]
2016-09-30 15:29:41.083 [warning] <0.3389.2>@vmq_ranch:teardown:115 [<0.3389.2>] session stopped abnormally due to 'publish_not_authorized_3_1_1'
2016-09-30 15:29:42.108 [error] <0.3393.2>@vmq_mqtt_fsm:auth_on_publish:631 can't auth publish [undefined,{[],<<"mqttjs_e73dea9a">>},0,[<<"presence">>],<<"Hello mqtt">>,false] due to [error]

Received payload is not JSON error on "on_publish" hook

Here in docs https://github.com/erlio/vmq_webhooks#on_publish is stated, that response should be empty.

When we return empty response, plugin throws an error
[error] <0.388.0>@vmq_webhooks_plugin:call_endpoint:471 calling endpoint failed due to: received_payload_not_json.

When we return { "result": "ok" }, then it is proceeded without any error. So docs doesnot follow the implementation.

Validation for vmq_webhooks.conf missing

Hey there,

The following incorrect vmq_webhooks.conf will be processed by vmq_webhook plugin but not by vernemq it self.
{hooks,1, [ {"http://localhost/api/auth/register", auth_on_register, [{base64_payload, true}]}, {"http://localhost/api/auth/subscribe", auth_on_subscribe, [{base64_payload, true}]}, {"http://localhost/api/auth/publish", publish, [{base64_payload, true}]} ]}.

The webhook "publish" doesn't exist. However the vmq-admin webhooks status shows
`+-----------------+-----------------------------------+-------------+

| hook | endpoint |base64payload|

+-----------------+-----------------------------------+-------------+

|auth_on_subscribe|http://localhost/api/auth/subscribe| true |

|auth_on_register |http://localhost/api/auth/register | true |

| auth_on_publish | http://localhost/api/auth/publish | true |

+-----------------+-----------------------------------+-------------+`

But publish webhook won't get into the vmq_plugin.conf.
{plugins,[{application,vmq_passwd,[]}, {application,vmq_acl,[]}, {module,vmq_config,[{hooks,[{change_config,change_config,1}]}]}, {module,vmq_lvldb_store, [{hooks,[{msg_store_write,msg_store_write,2}]}]}, {module,vmq_lvldb_store, [{hooks,[{msg_store_delete,msg_store_delete,2}]}]}, {module,vmq_lvldb_store, [{hooks,[{msg_store_find,msg_store_find,1}]}]}, {module,vmq_lvldb_store, [{hooks,[{msg_store_read,msg_store_read,2}]}]}, {application,vmq_webhooks, [{paths,["/app/broker/plugins/vmq_webhooks/_build/default/"]}]}, {module,vmq_webhooks_plugin, [{hooks,[{auth_on_register,auth_on_register,5}]}]}, {module,vmq_webhooks_plugin, [{hooks,[{auth_on_subscribe,auth_on_subscribe,3}]}]}]}.

I think there's some additional validation inside of the vmq_webhooks plugin necessary.

Best,
Michael

Saving webhooks on stop, loading webhooks on start

Right now, after I register some webhooks then stop then start vernemq the registered hooks dissapear.
Is there any place I can set the webhooks to automatically re-attach after start?

Plugin enable failing with an error

I have cloned the project and built the application as specified, now when I try to enable it, it doesn't seem to work. This is the command that doesn't work

vmq-admin plugin enable --name=vmq_webhooks --path=/root/verne/default/plugins/vmq_webhooks/_build/default/

The error message
RPC to '[email protected]' failed: {'EXIT', {{function_clause, [{vmq_plugin_mgr,init_plugins_cli, [[[]], ["/root/verne/default/rel/vernemq/bin/../lib/vmq_acl-0.15.2+build.1252.refcab8d43/priv", "/root/verne/default/rel/vernemq/bin/../lib/vmq_passwd-0.15.2+build.1252.refcab8d43/priv", "/root/verne/default/rel/vernemq/lib/vmq_server-0.15.2+build.1252.refcab8d43/priv"]], [{file, "/root/vernemq/_build/default/lib/vmq_plugin/src/vmq_plugin_mgr.erl"}, {line,508}]}, {vmq_plugin_mgr,load_plugins,2, [{file, "/root/vernemq/_build/default/lib/vmq_plugin/src/vmq_plugin_mgr.erl"}, {line,411}]}, {vmq_plugin_mgr,update_plugins,2, [{file, "/root/vernemq/_build/default/lib/vmq_plugin/src/vmq_plugin_mgr.erl"}, {line,356}]}, {vmq_plugin_mgr,handle_plugin_call,2, [{file, "/root/vernemq/_build/default/lib/vmq_plugin/src/vmq_plugin_mgr.erl"}, {line,184}]}, {gen_server,try_handle_call,4, [{file,"gen_server.erl"},{line,629}]}, {gen_server,handle_msg,5, [{file,"gen_server.erl"},{line,661}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,240}]}]}, {gen_server,call, [vmq_plugin_mgr, {enable_plugin,vmq_webhooks, [{paths, ["/root/verne/default/plugins/vmq_webhooks/_build/default/"]}]}, infinity]}}}

I'm running on Ubuntu 16.04.1

Vernemq instance seems to work just fine on 0.0.0.0:1883 and I can send and receive messages.

Documentation gap regarding webhook conflicts

Hey guys,
I think I found a gap in your documentation.

To use the auth_on_register it's necessary to disable the vmq_passwd plugin.
And to use the auth_on_publish and auth_on_subscribe I had to disable the vmq_acl plugin.
Otherwise Vernemq ignores my webhooks.

Best,
Michael

Subscriber_id modifier with auth_on_register hook is not applied

We have business rule to enforce same username and clientId when MQTT client connects to prevent client takeover. VerneMQ does not have this feature in global config, so we decided to implement it using webhooks.

So for auth_on_register hook we do "ok" response message with the subscriber_id modifier as following:

{"result":"ok","modifiers":{"subscriber_id":"xxx"}}

where "xxx" equeals to the username.

Issues is that subscriber_id is not modified and flow continues with the original subscriber_id.

Add optional caching for `auth_on_subscribe` and `auth_on_publish` hooks