noxdafox / rabbitmq-cloudwatch-exporter Goto Github PK
View Code? Open in Web Editor NEWRabbitMQ Plugin for publishing cluster metrics to AWS CloudWatch
License: Mozilla Public License 2.0
rabbitmq-cloudwatch-exporter's Issues
Choosing only a subset of metrics to publish
I'm currently using this library and would be really nice to choose between a subset of metrics to publish on every collection.
Why? Well, my broker has something like 300 queues/exchanges. Even when i choose only a specific collection (e.g "queue"), i'm publishing about 3k+ metrics, which most of them i don't need at the moment, to AWS each time.
So, maybe a solution to this kind of problem could be publishing only a subset of metrics (e.g "MessagesReady").
In the config file could be something like this (with default to all supported):
cloudwatch_exporter.metrics.queue = ["messages_ready","messages_unacknowledged"]
Although i never worked with elixir, i'll try to submit a PR soon.
Issue when using exchange metrics
When I enable exporting exchange metrics I get the following
2019-12-16 14:08:37.293 [error] <0.335.0> Supervisor 'Elixir.Singleton.Supervisor' had child 'Elixir.Singleton.Manager' started with 'Elixir.Singleton.Manager':start_link('Elixir.RabbitMQCloudWatchExporter.Exporter', [], rabbitmq_cloudwatch_exporter, 'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ') at <0.6228.0> exit with reason #{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.12.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.13.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>ef2bff11-f087-433...">>} in 'Elixir.ExAws':'request!'/2 line 66 in context child_terminated
problem exporting metrics
Hi,
Getting a crash while exporting the metrics.
My rabbitmq.conf':
cloudwatch_exporter.aws.region = "eu-west-1"
cloudwatch_exporter.metrics.1 = overview
cloudwatch_exporter.metrics.2 = vhost
cloudwatch_exporter.metrics.3 = node
log.file.level = debug
** Generic server rabbitmq_cloudwatch_exporter terminating
** Last message in was export_metrics
** When Server state == [[{period,60},{collectors,[overview,vhost,node]},{namespace,<<"RabbitMQ">>}],[{region,<<"eu-west-1">>}]]
** Reason for termination ==
** {#{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:option, :server_only, :honor_cipher_order}}\n">>},[{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,66}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter',handle_info,2,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,60}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
2019-09-30 08:22:11 =CRASH REPORT====
crasher:
initial call: Elixir.RabbitMQCloudWatchExporter.Exporter:init/1
pid: <0.697.0>
registered_name: []
exception error: {#{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:option, :server_only, :honor_cipher_order}}\n">>},[{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,66}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter',handle_info,2,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,60}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
ancestors: ['singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ','Elixir.Singleton.Supervisor',<0.271.0>]
message_queue_len: 0
messages: []
links: [<0.696.0>]
dictionary: [{rand_seed,{#{bits => 58,jump => #Fun<rand.13.8986388>,next => #Fun<rand.10.8986388>,type => exsss,uniform => #Fun<rand.11.8986388>,uniform_n => #Fun<rand.12.8986388>},[98387376866142632|287016919119728529]}}]
trap_exit: false
status: running
heap_size: 10958
stack_size: 27
reductions: 577379
neighbours:
neighbour: [{pid,<0.696.0>},{registered_name,'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ'},{initial_call,{'Elixir.Singleton.Manager',init,['Argument__1']}},{current_function,{gen_server,loop,7}},{ancestors,['Elixir.Singleton.Supervisor',<0.271.0>]},{message_queue_len,0},{links,[<0.272.0>,<0.697.0>]},{trap_exit,false},{status,waiting},{heap_size,233},{stack_size,11},{reductions,94},{current_stacktrace,[{gen_server,loop,7,[{file,"gen_server.erl"},{line,394}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}]
2019-09-30 08:22:11 =SUPERVISOR REPORT====
Supervisor: {local,'Elixir.Singleton.Supervisor'}
Context: child_terminated
Reason: {#{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:option, :server_only, :honor_cipher_order}}\n">>},[{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,66}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter',handle_info,2,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,60}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
Offender: [{pid,<0.696.0>},{id,'Elixir.Singleton.Manager'},{mfargs,{'Elixir.Singleton.Manager',start_link,['Elixir.RabbitMQCloudWatchExporter.Exporter',[],rabbitmq_cloudwatch_exporter,'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ']}},{restart_type,transient},{shutdown,5000},{child_type,worker}]
[ ] rabbitmq_auth_backend_cache 3.7.18
[ ] rabbitmq_auth_backend_http 3.7.18
[ ] rabbitmq_auth_backend_ldap 3.7.18
[ ] rabbitmq_auth_mechanism_ssl 3.7.18
[E ] rabbitmq_cloudwatch_exporter 0.2.1
[ ] rabbitmq_consistent_hash_exchange 3.7.18
[ ] rabbitmq_event_exchange 3.7.18
[ ] rabbitmq_federation 3.7.18
[ ] rabbitmq_federation_management 3.7.18
[ ] rabbitmq_jms_topic_exchange 3.7.18
[E*] rabbitmq_management 3.7.18
[e*] rabbitmq_management_agent 3.7.18
[ ] rabbitmq_mqtt 3.7.18
[ ] rabbitmq_peer_discovery_aws 3.7.18
[ ] rabbitmq_peer_discovery_common 3.7.18
[ ] rabbitmq_peer_discovery_consul 3.7.18
[ ] rabbitmq_peer_discovery_etcd 3.7.18
[ ] rabbitmq_peer_discovery_k8s 3.7.18
[ ] rabbitmq_random_exchange 3.7.18
[ ] rabbitmq_recent_history_exchange 3.7.18
[ ] rabbitmq_sharding 3.7.18
[ ] rabbitmq_shovel 3.7.18
[ ] rabbitmq_shovel_management 3.7.18
[ ] rabbitmq_stomp 3.7.18
[ ] rabbitmq_top 3.7.18
[ ] rabbitmq_tracing 3.7.18
[ ] rabbitmq_trust_store 3.7.18
[e*] rabbitmq_web_dispatch 3.7.18
[ ] rabbitmq_web_mqtt 3.7.18
[ ] rabbitmq_web_mqtt_examples 3.7.18
[ ] rabbitmq_web_stomp 3.7.18
[ ] rabbitmq_web_stomp_examples 3.7.18```
```{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","3.7.18"},
{amqp_client,"RabbitMQ AMQP Client","3.7.18"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.7.18"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.7.18"},
{rabbit,"RabbitMQ","3.7.18"},
{rabbit_common,
"Modules shared by rabbitmq-server and rabbitmq-erlang-client",
"3.7.18"},
{credentials_obfuscation,
"Helper library that obfuscates sensitive values in process state",
"1.1.0"},
{hackney,"simple HTTP client","1.15.1"},
{cowboy,"Small, fast, modern HTTP server.","2.6.1"},
{ranch,"Socket acceptor pool for TCP protocols.","1.7.1"},
{ssl,"Erlang/OTP SSL application","9.4"},
{public_key,"Public key infrastructure","1.7"},
{asn1,"The Erlang ASN1 compiler version 5.0.9","5.0.9"},
{ssl_verify_fun,"SSL verification functions for Erlang\n","1.1.4"},
{singleton,"Global, supervised singleton processes for Elixir","1.2.0"},
{ex_aws_cloudwatch,
"Cloudwatch module for https://github.com/ex-aws/ex_aws","2.0.4"},
{ex_aws,"Generic AWS client","2.1.0"},
{logger,"logger","1.8.2"},
{poison,"An incredibly fast, pure Elixir JSON library","3.1.0"},
{elixir,"elixir","1.8.2"},
{inets,"INETS CXC 138 49","7.1"},
{mnesia,"MNESIA CXC 138 12","4.16.1"},
{certifi,"CA bundle adapted from Mozilla by https://certifi.io","2.5.1"},
{stdout_formatter,
"Tools to format paragraphs, lists and tables as plain text",
"0.2.2"},
{observer_cli,"Visualize Erlang Nodes On The Command Line","1.5.2"},
{os_mon,"CPO CXC 138 46","2.5.1"},
{recon,"Diagnostic tools for production use","2.5.0"},
{tools,"DEVTOOLS CXC 138 16","3.2.1"},
{sysmon_handler,"Rate-limiting system_monitor event handler","1.1.0"},
{metrics,"A generic interface to different metrics systems in Erlang.",
"1.0.1"},
{idna,"A pure Erlang IDNA implementation","6.0.0"},
{unicode_util_compat,
"unicode_util compatibility library for Erlang < 20","0.4.1"},
{jsx,"a streaming, evented json parsing toolkit","2.9.0"},
{cowlib,"Support library for manipulating Web protocols.","2.7.0"},
{mimerl,"Library to handle mimetypes","1.2.0"},
{xmerl,"XML parser","1.3.22"},
{crypto,"CRYPTO","4.6"},
{lager,"Erlang logging framework","3.8.0"},
{goldrush,"Erlang event stream processor","0.1.9"},
{compiler,"ERTS CXC 138 10","7.4.5"},
{syntax_tools,"Syntax tools","2.2.1"},
{sasl,"SASL CXC 138 11","3.4.1"},
{stdlib,"ERTS CXC 138 10","3.10"},
{kernel,"ERTS CXC 138 10","6.5"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang/OTP 22 [erts-10.5] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:64] [hipe]\n"},
Call to undefined function crypto:hmac
Hello everyone!
I'm trying this awesome RabbitMQ Plugin, but I have been having a problem with a call to undefined function crypto:hmac.
RabbitMQ version: 3.8.18
Erlang/OTP version: 24
rabbitmq-cloudwatch-exporter version: 1.0.0
Plugin section in rabbitmq.conf:
cloudwatch_exporter.aws.access_key_id = "ACCESS_KEY_XXXXX"
cloudwatch_exporter.aws.secret_access_key = "SECRET_KEY_XXXXX"
cloudwatch_exporter.aws.region="eu-west-1"
cloudwatch_exporter.metrics.overview.enable = true
cloudwatch_exporter.metrics.vhost.enable = true
cloudwatch_exporter.metrics.node.enable = false
The error in rabbitmq log:
2021-06-30 [error] <0.734.0> ** Generic server rabbitmq_cloudwatch_exporter terminating
** Last message in was export_metrics
** When Server state == [{aws,[{secret_access_key,<<"SECRET_KEY_XXXXXX">>},{access_key_id,<<"ACCESS_KEY_XXXXXX">>},{region,<<"eu-west-1">>}]},{period,60},{resolution,60},{namespace,<<"RabbitMQ">>},{collectors,[{overview,[]},{vhost,[]}]}]
** Reason for termination ==
** {'function not exported',[{crypto,hmac,[sha256,[<<"AWS4">>,<<"SECRET_KEY_XXXXXX">>],[<<"2021">>,<<"06">>,<<"30">>]],[]},{'Elixir.ExAws.Auth.Signatures',signing_key,3,[{file,"lib/ex_aws/auth/signatures.ex"},{line,14}]},{'Elixir.ExAws.Auth.Signatures',generate_signature_v4,4,[{file,"lib/ex_aws/auth/signatures.ex"},{line,7}]},{'Elixir.ExAws.Auth',auth_header,7,[{file,"lib/ex_aws/auth.ex"},{line,129}]},{'Elixir.ExAws.Auth',headers,6,[{file,"lib/ex_aws/auth.ex"},{line,45}]},{'Elixir.ExAws.Request',request_and_retry,7,[{file,"lib/ex_aws/request.ex"},{line,29}]},{'Elixir.ExAws.Operation.ExAws.Operation.Query',perform,2,[{file,"lib/ex_aws/operation/query.ex"},{line,33}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter','-export_metrics/3-fun-0-',3,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,146}]}]}
2021-06-30 [error] <0.734.0> CRASH REPORT Process <0.734.0> with 1 neighbours crashed with reason: call to undefined function crypto:hmac(sha256, [<<"AWS4">>,<<"SECRET_KEY_XXXXXX">>], [<<"2021">>,<<"06">>,<<"30">>])
2021-06-30 [error] <0.728.0> Supervisor 'Elixir.Singleton.Supervisor' had child 'Elixir.Singleton.Manager' started with 'Elixir.Singleton.Manager':start_link('Elixir.RabbitMQCloudWatchExporter.Exporter', [], rabbitmq_cloudwatch_exporter, 'singleton_0bpF3SXqPATXc25tAf+lu5E9sgN') at <0.733.0> exit with reason call to undefined function crypto:hmac(sha256, [<<"AWS4">>,<<"SECRET_KEY_XXXXXX">>], [<<"2021">>,<<"06">>,<<"30">>]) in context child_terminated
2021-06-30 [error] <0.731.0> CRASH REPORT Process <0.731.0> with 0 neighbours exited with reason: call to undefined function crypto:hmac(sha256, [<<"AWS4">>,<<"SECRET_KEY_XXXXXX">>], [<<"2021">>,<<"06">>,<<"30">>]) in application_master:terminate/2 line 236
2021-06-30 [info] <0.44.0> Application rabbitmq_cloudwatch_exporter exited with reason: call to undefined function crypto:hmac(sha256, [<<"AWS4">>,<<"SECRET_KEY_XXXXXX">>], [<<"2021">>,<<"06">>,<<"30">>])
Thanks for the help.
Error during export
Rabbitmq 3.8.0-rc.1
Erlang 22.0.7
Configuration in rabbitmq.conf :
cloudwatch_exporter.metrics.1 = overview
cloudwatch_exporter.metrics.2 = queue
cloudwatch_exporter.metrics.3 = node
ECS Task IAM used for credentials
AWS_DEFAULT_REGION and AWS_REGION environment variables defined
Stacktrace
** {{badfun,nil},[{'Elixir.RabbitMQCloudWatchExporter.Exporter','-handle_info/2-fun-3-',1,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,55}]},{'Elixir.Enum',flat_map_list,2,[{file,"lib/enum.ex"},{line,2986}]},{'Elixir.Enum',flat_map_list,2,[{file,"lib/enum.ex"},{line,2987}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter',handle_info,2,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,55}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
Support for IMDSV2
We have been using this plugin to capture the RMQ metrics @ cloudwatch for quite some time and it has been working great. Thanks a lot.
About the Enhancement:
The IMDS of AWS has a new version IMDSV2.
AWS recommends that everyone migrate to the V2 version, and has settings to turn off V1 at the machine level.
This is recommended for security reasons.
The peer_discovery_aws module has moved over:
rabbitmq/rabbitmq-server#2952
Also when I checked on the Dependency of this project ex_aws, there is a PR that is open for this:
ex-aws/ex_aws#739
It would be really helpful if we could support this enhancement for this library.
Missing dependencies on enabling plugin
I'm getting
rabbitmq_cloudwatch_exporter
Error:
{:missing_dependencies, [:jsx], [:erlcloud]}
when I try to enable the plugin, this was working before. Let me know if something is changed
Thanks
CaseClauseError when enabling plugin
Hi,
I am getting this exception when trying to enable the plugin:
** (CaseClauseError) no case clause matching: {:plugin_built_with_incompatible_erlang, 'hackney'}
(rabbitmqctl) lib/rabbitmq/cli/plugins/plugins_helpers.ex:176: RabbitMQ.CLI.Plugins.Helpers.update_enabled_plugins/2
(rabbitmqctl) lib/rabbitmq/cli/plugins/plugins_helpers.ex:98: RabbitMQ.CLI.Plugins.Helpers.update_enabled_plugins/4
(rabbitmqctl) lib/rabbitmq/cli/plugins/commands/enable_command.ex:98: anonymous fn/6 in RabbitMQ.CLI.Plugins.Commands.EnableCommand.do_run/2
(elixir) lib/stream.ex:1179: anonymous fn/2 in Stream.iterate/2
(elixir) lib/stream.ex:1354: Stream.do_unfold/4
(elixir) lib/stream.ex:1405: Enumerable.Stream.do_each/4
(elixir) lib/stream.ex:849: Stream.do_enum_transform/10
(elixir) lib/stream.ex:1405: Enumerable.Stream.do_each/4
Logs are not being sent to Cloudwatch
Hi,
Firstly, let me say that this plugin is easy to use and takes all of 5 minutes to get setup and start sending RabbitMQ metrics to Cloudwatch. The only issue I have is that I'm unable to send log data to Cloudwatch. I'm running the below config on a test instance.
I have noticed that the lager config is not appearing in the output of "rabbitmqctl environment" am I missing a plugin?
Versions:
Erlang: 24.3.2
RabbitMQ: 3.9.14
My rabbitmq.config:
[
% lager backend config
{lager,
[{handlers, [{lager_cloudwatch_backend, [info, "RabbitMQ"]}]}]},
% RabbitMQ Cloudwatch Exporter config
{rabbitmq_cloudwatch_exporter,
[{aws,
[{secret_access_key,"SOMEKEY"},
{access_key_id,"ANOTHERKEY"},
{region,"eu-west-2"}]},
{metrics,
[{queue,[{enable,true}]},
{exchange,[{enable,true}]},
{node,[{enable,true}]},
{vhost,[{enable,true}]},
{overview,[{enable,true}]}]},
{namespace,"rabbitmq"}]}
].
released .ez files can't be installed on 3.7
I'm encountering an error while installing. We're running rabbitmq from docker so I updated our Dockerfile to pull down this plugin and install it, but it fails with {invalid_ez,einval}.
Using this dockerfile:
FROM rabbitmq:3.7-management
ENV CWPLUGIN https://api.github.com/repos/noxdafox/rabbitmq-cloudwatch-exporter/releases/18074331
WORKDIR /opt/rabbitmq/plugins
RUN apt-get update && \
apt-get install -y jq curl && \
export DLS="$(curl $CWPLUGIN | jq -r '.assets[].browser_download_url')" && \
(for u in $DLS; do curl -O $u; done;) && \
apt-get purge -y jq curl
RUN rabbitmq-plugins enable rabbitmq_cloudwatch_exporter
the final command gives this error message:
2019-09-16T15:31:09.7540788Z Step 5/5 : RUN rabbitmq-plugins enable rabbitmq_cloudwatch_exporter
2019-09-16T15:31:10.3590127Z ---> Running in 4477ce81c079
2019-09-16T15:31:12.4438676Z Enabling plugins on node rabbit@4477ce81c079:
2019-09-16T15:31:12.4441404Z rabbitmq_cloudwatch_exporter
2019-09-16T15:31:12.6652172Z �[91mProblem reading some plugins: [{"/opt/rabbitmq/plugins/unicode_util_compat-0.4.1.ez",
2019-09-16T15:31:12.6652416Z {invalid_ez,einval}},
2019-09-16T15:31:12.6652792Z {"/opt/rabbitmq/plugins/ssl_verify_fun-1.1.4.ez",
2019-09-16T15:31:12.6652912Z {invalid_ez,einval}},
2019-09-16T15:31:12.6653233Z {"/opt/rabbitmq/plugins/singleton-1.2.0.ez",
2019-09-16T15:31:12.6653347Z {invalid_ez,einval}},
2019-09-16T15:31:12.6654225Z {"/opt/rabbitmq/plugins/rabbitmq_cloudwatch_exporter-0.2.1.ez",
2019-09-16T15:31:12.6654347Z {invalid_ez,einval}},
2019-09-16T15:31:12.6654681Z {"/opt/rabbitmq/plugins/poison-3.1.0.ez",
2019-09-16T15:31:12.6654777Z {invalid_ez,einval}},
2019-09-16T15:31:12.6655121Z {"/opt/rabbitmq/plugins/mimerl-1.2.0.ez",
2019-09-16T15:31:12.6655236Z {invalid_ez,einval}},
2019-09-16T15:31:12.6655566Z {"/opt/rabbitmq/plugins/metrics-1.0.1.ez",
2019-09-16T15:31:12.6655662Z {invalid_ez,einval}},
2019-09-16T15:31:12.6655992Z {"/opt/rabbitmq/plugins/logger-1.8.2.ez",
2019-09-16T15:31:12.6656103Z {invalid_ez,einval}},
2019-09-16T15:31:12.6656430Z {"/opt/rabbitmq/plugins/idna-6.0.0.ez",
2019-09-16T15:31:12.6656550Z {invalid_ez,einval}},
2019-09-16T15:31:12.6656862Z {"/opt/rabbitmq/plugins/hackney-1.15.1.ez",
2019-09-16T15:31:12.6656989Z {invalid_ez,einval}},
2019-09-16T15:31:12.6657330Z {"/opt/rabbitmq/plugins/ex_aws_cloudwatch-2.0.4.ez",
2019-09-16T15:31:12.6657628Z {invalid_ez,einval}},
2019-09-16T15:31:12.6658004Z {"/opt/rabbitmq/plugins/ex_aws-2.1.0.ez",
2019-09-16T15:31:12.6658097Z {invalid_ez,einval}},
2019-09-16T15:31:12.6658423Z {"/opt/rabbitmq/plugins/elixir-1.8.2.ez",
2019-09-16T15:31:12.6658533Z {invalid_ez,einval}},
2019-09-16T15:31:12.6659142Z {"/opt/rabbitmq/plugins/certifi-2.5.1.ez",
2019-09-16T15:31:12.6659262Z {invalid_ez,einval}}]
2019-09-16T15:31:12.7699242Z �[0m�[91mProblem reading some plugins: [{"/opt/rabbitmq/plugins/unicode_util_compat-0.4.1.ez",
2019-09-16T15:31:12.7699440Z {invalid_ez,einval}},
2019-09-16T15:31:12.7699845Z {"/opt/rabbitmq/plugins/ssl_verify_fun-1.1.4.ez",
2019-09-16T15:31:12.7699967Z {invalid_ez,einval}},
2019-09-16T15:31:12.7700333Z {"/opt/rabbitmq/plugins/singleton-1.2.0.ez",
2019-09-16T15:31:12.7700450Z {invalid_ez,einval}},
2019-09-16T15:31:12.7700785Z {"/opt/rabbitmq/plugins/rabbitmq_cloudwatch_exporter-0.2.1.ez",
2019-09-16T15:31:12.7700905Z {invalid_ez,einval}},
2019-09-16T15:31:12.7701230Z {"/opt/rabbitmq/plugins/poison-3.1.0.ez",
2019-09-16T15:31:12.7701342Z {invalid_ez,einval}},
2019-09-16T15:31:12.7701681Z {"/opt/rabbitmq/plugins/mimerl-1.2.0.ez",
2019-09-16T15:31:12.7701775Z {invalid_ez,einval}},
2019-09-16T15:31:12.7702103Z {"/opt/rabbitmq/plugins/metrics-1.0.1.ez",
2019-09-16T15:31:12.7702215Z {invalid_ez,einval}},
2019-09-16T15:31:12.7702539Z {"/opt/rabbitmq/plugins/logger-1.8.2.ez",
2019-09-16T15:31:12.7702643Z {invalid_ez,einval}},
2019-09-16T15:31:12.7702969Z {"/opt/rabbitmq/plugins/idna-6.0.0.ez",
2019-09-16T15:31:12.7703082Z {invalid_ez,einval}},
2019-09-16T15:31:12.7703411Z {"/opt/rabbitmq/plugins/hackney-1.15.1.ez",
2019-09-16T15:31:12.7703523Z {invalid_ez,einval}},
2019-09-16T15:31:12.7703841Z {"/opt/rabbitmq/plugins/ex_aws_cloudwatch-2.0.4.ez",
2019-09-16T15:31:12.7704326Z {invalid_ez,einval}},
2019-09-16T15:31:12.7704686Z {"/opt/rabbitmq/plugins/ex_aws-2.1.0.ez",
2019-09-16T15:31:12.7704796Z {invalid_ez,einval}},
2019-09-16T15:31:12.7705118Z {"/opt/rabbitmq/plugins/elixir-1.8.2.ez",
2019-09-16T15:31:12.7705211Z {invalid_ez,einval}},
2019-09-16T15:31:12.7705545Z {"/opt/rabbitmq/plugins/certifi-2.5.1.ez",
2019-09-16T15:31:12.7705656Z {invalid_ez,einval}}]
2019-09-16T15:31:12.8733431Z �[0m�[91mProblem reading some plugins: [{"/opt/rabbitmq/plugins/unicode_util_compat-0.4.1.ez",
2019-09-16T15:31:12.8734479Z {invalid_ez,einval}},
2019-09-16T15:31:12.8735490Z {"/opt/rabbitmq/plugins/ssl_verify_fun-1.1.4.ez",
2019-09-16T15:31:12.8735659Z {invalid_ez,einval}},
2019-09-16T15:31:12.8736062Z {"/opt/rabbitmq/plugins/singleton-1.2.0.ez",
2019-09-16T15:31:12.8736248Z {invalid_ez,einval}},
2019-09-16T15:31:12.8736613Z {"/opt/rabbitmq/plugins/rabbitmq_cloudwatch_exporter-0.2.1.ez",
2019-09-16T15:31:12.8736734Z {invalid_ez,einval}},
2019-09-16T15:31:12.8737493Z {"/opt/rabbitmq/plugins/poison-3.1.0.ez",
2019-09-16T15:31:12.8738294Z {invalid_ez,einval}},
2019-09-16T15:31:12.8739067Z {"/opt/rabbitmq/plugins/mimerl-1.2.0.ez",
2019-09-16T15:31:12.8739189Z {invalid_ez,einval}},
2019-09-16T15:31:12.8739566Z {"/opt/rabbitmq/plugins/metrics-1.0.1.ez",
2019-09-16T15:31:12.8739679Z {invalid_ez,einval}},
2019-09-16T15:31:12.8740003Z {"/opt/rabbitmq/plugins/logger-1.8.2.ez",
2019-09-16T15:31:12.8740130Z {invalid_ez,einval}},
2019-09-16T15:31:12.8740437Z {"/opt/rabbitmq/plugins/idna-6.0.0.ez",
2019-09-16T15:31:12.8740553Z {invalid_ez,einval}},
2019-09-16T15:31:12.8740883Z {"/opt/rabbitmq/plugins/hackney-1.15.1.ez",
2019-09-16T15:31:12.8741006Z {invalid_ez,einval}},
2019-09-16T15:31:12.8741345Z {"/opt/rabbitmq/plugins/ex_aws_cloudwatch-2.0.4.ez",
2019-09-16T15:31:12.8741444Z {invalid_ez,einval}},
2019-09-16T15:31:12.8741767Z {"/opt/rabbitmq/plugins/ex_aws-2.1.0.ez",
2019-09-16T15:31:12.8741878Z {invalid_ez,einval}},
2019-09-16T15:31:12.8742203Z {"/opt/rabbitmq/plugins/elixir-1.8.2.ez",
2019-09-16T15:31:12.8742313Z {invalid_ez,einval}},
2019-09-16T15:31:12.8742632Z {"/opt/rabbitmq/plugins/certifi-2.5.1.ez",
2019-09-16T15:31:12.8742744Z {invalid_ez,einval}}]
2019-09-16T15:31:12.8991396Z �[0m�[91mError:
2019-09-16T15:31:12.8991693Z {:plugins_not_found, [:rabbitmq_cloudwatch_exporter]}
2019-09-16T15:31:13.2883439Z The command '/bin/sh -c rabbitmq-plugins enable rabbitmq_cloudwatch_exporter' returned a non-zero code: 70
Support high resolution metrics
We need to get metrics in fine granularity. for .e.g. 20s intervals.
cloudwatch_exporter.export_period = 20
Tried with setting this but still, I can see 60s intervals in cloud watch
Error during enable plugin for erlang 21
Hi, could you tell me how to build for erlang 21 version?
2020-12-15 08:58:41 =ERROR REPORT====
Loading of /var/lib/rabbitmq/mnesia/rabbit@ip-10-0-7-83-plugins-expand/rabbitmq_cloudwatch_exporter-1.0.0/ebin/Elixir.RabbitMQCloudWatchExporter.ChannelMetrics.beam failed: badfile
2020-12-15 08:58:41 =ERROR REPORT====
beam/beam_load.c(1878): Error loading module 'Elixir.RabbitMQCloudWatchExporter.ChannelMetrics':
This BEAM file was compiled for a later version of the run-time system than 21.
To fix this, please recompile this module with an 21 compiler.
(Use of opcode 164; this emulator supports only up to 163.)
Getting an error after enabling rabbitmq_cloudwatch_exporter but able to get metrics.
Hi ,
Do we miss something?. Can we analyse below logs.
\n Sender\n MissingParameter\n The parameter MetricData.member.16.Dimensions.member.3.Value is required.\nThe parameter MetricData.member.17.Dimensions.member.3.Value is required.
Exporting metrics doesn't work anymore after reinstall
Hi
I've reinstalled my rabbitmq server, hence reinstalled the plugin, but the metrics aren't exporting anymore to cloudwatch.
Erlang version : 24.0.4
plugin files : up-to-date from new files on 1.0.0 release as I post
Getting the following error :
2021-07-29 09:05:11 =ERROR REPORT====
** Generic server 'Elixir.ExAws.Config.AuthCache' terminating
** Last message in was {refresh_auth,#{access_key_id => [{system,<<"AWS_ACCESS_KEY_ID">>},instance_role],host => <<"monitoring.eu-west-1.amazonaws.com">>,http_client => 'Elixir.ExAws.Request.Hackney',json_codec => 'Elixir.Jason',normalize_path => true,port => 443,region => <<"eu-west-1">>,retries => [{max_attempts,10},{base_backoff_in_ms,10},{max_backoff_in_ms,10000}],scheme => <<"https://">>,secret_access_key => [{system,<<"AWS_SECRET_ACCESS_KEY">>},instance_role]}}
** When Server state == 'Elixir.ExAws.Config.AuthCache'
** Reason for termination ==
** {'module could not be loaded',[{'Elixir.Jason','decode!',[<<"{\n \"Code\" : \"Success\",\n \"LastUpdated\" : \"2021-07-29T08:24:39Z\",\n \"Type\" : \"AWS-HMAC\",\n \"AccessKeyId\" : \"(...)\",\n \"SecretAccessKey\" : \"(...)/02\",\n \"Token\" : \"(...)==\",\n \"Expiration\" : \"2021-07-29T14:33:13Z\"\n}">>],[]},{'Elixir.ExAws.InstanceMeta',security_credentials,1,[{file,"lib/ex_aws/instance_meta.ex"},{line,72}]},{'Elixir.ExAws.Config.AuthCache',refresh_auth_now,2,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,116}]},{'Elixir.ExAws.Config.AuthCache',handle_call,3,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,45}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
** Client <0.20426.3> stacktrace
** [{gen,do_call,4,[{file,"gen.erl"},{line,233}]},{'Elixir.GenServer',call,3,[{file,"lib/gen_server.ex"},{line,1020}]},{'Elixir.ExAws.Config',retrieve_runtime_value,2,[{file,"lib/ex_aws/config.ex"},{line,92}]},{'Elixir.Stream','-map/2-fun-0-',4,[{file,"lib/stream.ex"},{line,572}]},{'Elixir.Enumerable.List',reduce,3,[{file,"lib/enum.ex"},{line,3686}]},{'Elixir.Enumerable.Stream',do_each,4,[{file,"lib/stream.ex"},{line,1609}]},{'Elixir.Enum',find,3,[{file,"lib/enum.ex"},{line,959}]},{'Elixir.ExAws.Config','-retrieve_runtime_config/1-fun-0-',2,[{file,"lib/ex_aws/config.ex"},{line,79}]}]
2021-07-29 09:05:11 =CRASH REPORT====
crasher:
initial call: Elixir.ExAws.Config.AuthCache:init/1
pid: <0.20427.3>
registered_name: 'Elixir.ExAws.Config.AuthCache'
exception error: {undef,[{'Elixir.Jason','decode!',[<<"{\n \"Code\" : \"Success\",\n \"LastUpdated\" : \"2021-07-29T08:24:39Z\",\n \"Type\" : \"AWS-HMAC\",\n \"AccessKeyId\" : \"(...)\",\n \"SecretAccessKey\" : \"(...)/02\",\n \"Token\" : \"I(...)Q==\",\n \"Expiration\" : \"2021-07-29T14:33:13Z\"\n}">>],[]},{'Elixir.ExAws.InstanceMeta',security_credentials,1,[{file,"lib/ex_aws/instance_meta.ex"},{line,72}]},{'Elixir.ExAws.Config.AuthCache',refresh_auth_now,2,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,116}]},{'Elixir.ExAws.Config.AuthCache',handle_call,3,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,45}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
ancestors: ['Elixir.ExAws.Supervisor',<0.1025.0>]
message_queue_len: 0
messages: []
links: [<0.1026.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 10958
stack_size: 29
reductions: 221235
neighbours:
2021-07-29 09:05:11 =SUPERVISOR REPORT====
Supervisor: {local,'Elixir.ExAws.Supervisor'}
Context: child_terminated
Reason: {undef,[{'Elixir.Jason','decode!',[<<"{\n \"Code\" : \"Success\",\n \"LastUpdated\" : \"2021-07-29T08:24:39Z\",\n \"Type\" : \"AWS-HMAC\",\n \"AccessKeyId\" : \"(...)\",\n \"SecretAccessKey\" : \"(...)/02\",\n \"Token\" : \"IQo(...)Q==\",\n \"Expiration\" : \"2021-07-29T14:33:13Z\"\n}">>],[]},{'Elixir.ExAws.InstanceMeta',security_credentials,1,[{file,"lib/ex_aws/instance_meta.ex"},{line,72}]},{'Elixir.ExAws.Config.AuthCache',refresh_auth_now,2,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,116}]},{'Elixir.ExAws.Config.AuthCache',handle_call,3,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,45}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
Offender: [{pid,<0.20427.3>},{id,'Elixir.ExAws.Config.AuthCache'},{mfargs,{'Elixir.ExAws.Config.AuthCache',start_link,[[]]}},{restart_type,permanent},{significant,false},{shutdown,5000},{child_type,worker}]
2021-07-29 09:05:11 =ERROR REPORT====
** Generic server rabbitmq_cloudwatch_exporter terminating
** Last message in was export_metrics
** When Server state == [{aws,[{region,<<"eu-west-1">>}]},{period,60},{resolution,60},{namespace,<<"RabbitMQ">>},{collectors,[{queue,[{export_regex,#{'__struct__' => 'Elixir.Regex',opts => <<>>,re_pattern => {re_pattern,0,0,0,<<69,82,67,80,87,0,0,0,0,0,0,0,81,0,0,0,255,255,255,255,255,255,255,255,97,0,95,0,0,0,0,0,0,0,64,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,131,0,19,29,97,29,115,29,103,29,95,29,109,29,115,29,95,94,12,120,0,19,0>>},re_version => {<<"8.44 2020-02-12">>,little},source => <<"asg_ms_.*">>}},{export_metrics,[<<"Messages">>]}]}]}]
** Reason for termination ==
** {{{undef,[{'Elixir.Jason','decode!',[<<"{\n \"Code\" : \"Success\",\n \"LastUpdated\" : \"2021-07-29T08:24:39Z\",\n \"Type\" : \"AWS-HMAC\",\n \"AccessKeyId\" : \"(...)\",\n \"SecretAccessKey\" : \"(...)/02\",\n \"Token\" : \"IQoJb3JpZ2luX2VjEIH//////////(...)FLJ/5FQ==\",\n \"Expiration\" : \"2021-07-29T14:33:13Z\"\n}">>],[]},{'Elixir.ExAws.InstanceMeta',security_credentials,1,[{file,"lib/ex_aws/instance_meta.ex"},{line,72}]},{'Elixir.ExAws.Config.AuthCache',refresh_auth_now,2,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,116}]},{'Elixir.ExAws.Config.AuthCache',handle_call,3,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,45}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]},{'Elixir.GenServer',call,['Elixir.ExAws.Config.AuthCache',{refresh_auth,#{access_key_id => [{system,<<"AWS_ACCESS_KEY_ID">>},instance_role],host => <<"monitoring.eu-west-1.amazonaws.com">>,http_client => 'Elixir.ExAws.Request.Hackney',json_codec => 'Elixir.Jason',normalize_path => true,port => 443,region => <<"eu-west-1">>,retries => [{max_attempts,10},{base_backoff_in_ms,10},{max_backoff_in_ms,10000}],scheme => <<"https://">>,secret_access_key => [{system,<<"AWS_SECRET_ACCESS_KEY">>},instance_role]}},30000]}},[{'Elixir.GenServer',call,3,[{file,"lib/gen_server.ex"},{line,1023}]},{'Elixir.ExAws.Config',retrieve_runtime_value,2,[{file,"lib/ex_aws/config.ex"},{line,92}]},{'Elixir.Stream','-map/2-fun-0-',4,[{file,"lib/stream.ex"},{line,572}]},{'Elixir.Enumerable.List',reduce,3,[{file,"lib/enum.ex"},{line,3686}]},{'Elixir.Enumerable.Stream',do_each,4,[{file,"lib/stream.ex"},{line,1609}]},{'Elixir.Enum',find,3,[{file,"lib/enum.ex"},{line,959}]},{'Elixir.ExAws.Config','-retrieve_runtime_config/1-fun-0-',2,[{file,"lib/ex_aws/config.ex"},{line,79}]},{maps,fold_1,3,[{file,"maps.erl"},{line,410}]}]}
2021-07-29 09:05:11 =CRASH REPORT====
crasher:
initial call: Elixir.RabbitMQCloudWatchExporter.Exporter:init/1
pid: <0.20426.3>
registered_name: []
exception exit: {{{undef,[{'Elixir.Jason','decode!',[<<"{\n \"Code\" : \"Success\",\n \"LastUpdated\" : \"2021-07-29T08:24:39Z\",\n \"Type\" : \"AWS-HMAC\",\n \"AccessKeyId\" : \"(...)\",\n \"SecretAccessKey\" : \"(...)\",\n \"Token\" : \(...)\",\n \"Expiration\" : \"2021-07-29T14:33:13Z\"\n}">>],[]},{'Elixir.ExAws.InstanceMeta',security_credentials,1,[{file,"lib/ex_aws/instance_meta.ex"},{line,72}]},{'Elixir.ExAws.Config.AuthCache',refresh_auth_now,2,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,116}]},{'Elixir.ExAws.Config.AuthCache',handle_call,3,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,45}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]},{'Elixir.GenServer',call,['Elixir.ExAws.Config.AuthCache',{refresh_auth,#{access_key_id => [{system,<<"AWS_ACCESS_KEY_ID">>},instance_role],host => <<"monitoring.eu-west-1.amazonaws.com">>,http_client => 'Elixir.ExAws.Request.Hackney',json_codec => 'Elixir.Jason',normalize_path => true,port => 443,region => <<"eu-west-1">>,retries => [{max_attempts,10},{base_backoff_in_ms,10},{max_backoff_in_ms,10000}],scheme => <<"https://">>,secret_access_key => [{system,<<"AWS_SECRET_ACCESS_KEY">>},instance_role]}},30000]}},[{'Elixir.GenServer',call,3,[{file,"lib/gen_server.ex"},{line,1023}]},{'Elixir.ExAws.Config',retrieve_runtime_value,2,[{file,"lib/ex_aws/config.ex"},{line,92}]},{'Elixir.Stream','-map/2-fun-0-',4,[{file,"lib/stream.ex"},{line,572}]},{'Elixir.Enumerable.List',reduce,3,[{file,"lib/enum.ex"},{line,3686}]},{'Elixir.Enumerable.Stream',do_each,4,[{file,"lib/stream.ex"},{line,1609}]},{'Elixir.Enum',find,3,[{file,"lib/enum.ex"},{line,959}]},{'Elixir.ExAws.Config','-retrieve_runtime_config/1-fun-0-',2,[{file,"lib/ex_aws/config.ex"},{line,79}]},{maps,fold_1,3,[{file,"maps.erl"},{line,410}]}]}
ancestors: ['singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ','Elixir.Singleton.Supervisor',<0.1156.0>]
message_queue_len: 0
messages: []
links: [<0.20425.3>]
dictionary: []
trap_exit: false
status: running
heap_size: 10958
stack_size: 29
reductions: 230419
neighbours:
neighbour: [{pid,<0.20425.3>},{registered_name,'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ'},{initial_call,{'Elixir.Singleton.Manager',init,['Argument__1']}},{current_function,{gen_server,loop,7}},{ancestors,['Elixir.Singleton.Supervisor',<0.1156.0>]},{message_queue_len,0},{links,[<0.1157.0>,<0.20426.3>]},{trap_exit,false},{status,waiting},{heap_size,233},{stack_size,12},{reductions,85},{current_stacktrace,[{gen_server,loop,7,[{file,"gen_server.erl"},{line,443}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}]
2021-07-29 09:05:11 =SUPERVISOR REPORT====
Supervisor: {local,'Elixir.Singleton.Supervisor'}
Context: child_terminated
Reason: {{undef,[{'Elixir.Jason','decode!',[<<"{\n \"Code\" : \"Success\",\n \"LastUpdated\" : \"2021-07-29T08:24:39Z\",\n \"Type\" : \"AWS-HMAC\",\n \"AccessKeyId\" : \"(...)\",\n \"SecretAccessKey\" : \(...)",\n \"Token\" : \(...),\n \"Expiration\" : \"2021-07-29T14:33:13Z\"\n}">>],[]},{'Elixir.ExAws.InstanceMeta',security_credentials,1,[{file,"lib/ex_aws/instance_meta.ex"},{line,72}]},{'Elixir.ExAws.Config.AuthCache',refresh_auth_now,2,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,116}]},{'Elixir.ExAws.Config.AuthCache',handle_call,3,[{file,"lib/ex_aws/config/auth_cache.ex"},{line,45}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]},{'Elixir.GenServer',call,['Elixir.ExAws.Config.AuthCache',{refresh_auth,#{access_key_id => [{system,<<"AWS_ACCESS_KEY_ID">>},instance_role],host => <<"monitoring.eu-west-1.amazonaws.com">>,http_client => 'Elixir.ExAws.Request.Hackney',json_codec => 'Elixir.Jason',normalize_path => true,port => 443,region => <<"eu-west-1">>,retries => [{max_attempts,10},{base_backoff_in_ms,10},{max_backoff_in_ms,10000}],scheme => <<"https://">>,secret_access_key => [{system,<<"AWS_SECRET_ACCESS_KEY">>},instance_role]}},30000]}}
Offender: [{pid,<0.20425.3>},{id,'Elixir.Singleton.Manager'},{mfargs,{'Elixir.Singleton.Manager',start_link,['Elixir.RabbitMQCloudWatchExporter.Exporter',[],rabbitmq_cloudwatch_exporter,'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ']}},{restart_type,transient},{significant,false},{shutdown,5000},{child_type,worker}]
The plugin seems to be well installed :
root@ip-10-1-102-210:/home/ubuntu# rabbitmq-plugins is_enabled rabbitmq_cloudwatch_exporter
Asking node rabbit@ip-10-1-102-210 if plugin rabbitmq_cloudwatch_exporter is enabled...
Plugin rabbitmq_cloudwatch_exporter is enabled on node rabbit@ip-10-1-102-210
The conf is simple and used to work with previous versions :
cloudwatch_exporter.aws.region = eu-west-1
cloudwatch_exporter.metrics.queue.enable = true
cloudwatch_exporter.metrics.queue.export_regex = "asg_ms_.*"
cloudwatch_exporter.metrics.queue.export_metrics.1 = "Messages"
Thx again for your work ! :)
HTTP error on RabbitMQ v3.8.4
Hi!
We tried this out in a test environment with RabbitMQ v3.8.4 (Erlang 23..) and get the following error in our logs:
12:33:28.193 [warn] ExAws: HTTP ERROR: {:options, {:sslv3, {:versions, [:"tlsv1.2", :"tlsv1.1", :tlsv1, :sslv3]}}}
Error: {options,{sslv3,{versions,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}}}
My guess is that this is due to Erlang 23 dropping SSLv3 and adding TLSv1.3: https://elixirforum.com/t/erlang-v-23-0-and-httpoison-error/31615
I started looking into it a bit, but I'm unsure whether upgrading hackney to 1.16 is enough however, as ex_aws also has a dependency to (an older version of afaik) hackney. Maybe it's possible to pass different options to hackney regardless, excluding sslv3, but I'm unsure where / how to do that.
bug with cluster name / node IP metric aggregation
If in use by an ASG or likely nodes are to be replaced..
The cluster name is likely to change along with the node IP, these really shouldn't be aggregated as when creating an alarm these values are likely to change if a node is replaced and there's a new leader.
It makes it impossible to create an alarm based off these metrics.
A fix for this is to remove metric aggregation and to have the cluster name parametrised
CloudWatch routingKey Metrics
Hi ! Thanks for that great plugin.
I am using RabbitMQ with the Stomp plugin. I don't know if I configured everything correctly, but I can see an "amq.topic" exchange containing my main topics as routing keys, and then I can see the consumers on the queues tab. So, everything is working fine but what I'd like to do is be able to get metrics for the routingKeys. These routingKeys are my main topics and I'd like to be able to track nb of messages and nb of consumers per topic / routingKey.
Is that possible ?
Singleton metrics publisher process
Currently, the plugin runs a Task process which collects the metrics and publishes them to AWS CW.
This process is unique per node which means either we loose metrics publishing if the node running the process goes down or we run one process per node and we publish metrics multiple times.
We need a singleton mechanism ensuring the Task process is always run once across multi-node clusters.
Issue when publishing node metrics
When I publish my metrics using the following configuration :
[{rabbitmq_cloudwatch_exporter,
[{aws, [{access_key_id, "***"},
{secret_access_key, "***"},
{region, "eu-west-3"}]},
{metrics, [overview, vhost, node, exchange, queue, connection, channel]}]}].
I get an error from ExAws :
** Started from <0.613.0>
** When function == fun Elixir.RabbitMQ.CloudWatchExporter.Exporter:run/2
** arguments == [[{period,60},{collectors,[overview,vhost,node,exchange,queue,connection,channel]},{namespace,<<"RabbitMQ">>}],[{access_key_id,<<"***">>},{secret_access_key,<<"***">>},{region,<<"eu-west-3">>}]]
** Reason for termination ==
** {#{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.7.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.8.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>c50b609e-76eb-11e9-a3e2-d57f24e0c0f8</RequestId>\\n</ErrorResponse>\\n\", headers: [{\"x-amzn-RequestId\", \"c50b609e-76eb-11e9-a3e2-d57f24e0c0f8\"}, {\"Content-Type\", \"text/xml\"}, {\"Content-Length\", \"399\"}, {\"Date\", \"Wed, 15 May 2019 08:31:00 GMT\"}, {\"Connection\", \"close\"}], status_code: 400}}}\n">>},[{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,66}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.RabbitMQ.CloudWatchExporter.Exporter',run,2,[{file,"lib/exporter.ex"},{line,60}]},{'Elixir.Task.Supervised',invoke_mfa,2,[{file,"lib/task/supervised.ex"},{line,90}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
2019-05-15 08:31:00.691 [error] <0.676.0> CRASH REPORT Process <0.676.0> with 0 neighbours crashed with reason: #{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.7.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.8.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>c50b609e-76eb-11e9-...">>} in 'Elixir.ExAws':'request!'/2 line 66
2019-05-15 08:31:00.692 [error] <0.613.0> Supervisor {<0.613.0>,'Elixir.Supervisor.Default'} had child 'Elixir.RabbitMQ.CloudWatchExporter.Exporter' started with 'Elixir.RabbitMQ.CloudWatchExporter.Exporter':start_link([]) at <0.676.0> exit with reason #{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.7.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.8.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>c50b609e-76eb-11e9-...">>} in 'Elixir.ExAws':'request!'/2 line 66 in context child_terminated
2019-05-15 08:32:00.814 [error] <0.679.0> ** Task <0.679.0> terminating
For the record, I've only one rabbitmq server, I've removed the node, connection, channel metrics and it worked
a
Question: I see that used old style config for node exporter plugin
In docs which available on page i see that used erlang style of rabbitmq config. I used 3.7.X version where available key=value style. does it support by this plugin?
Duplicate and dependency errors
Hi
I've been using your plugin for some time now, but I've had to reinstall my cluster. I encountered a problem as the file download "ex_aws-2.1.0.ez" was giving me a 404. I've realized that on the 1.0.0 release, you changed it for the 2.2.3 version, so I've updated that.
Now i get the following error, and I'm stuck with it :
Problem reading some plugins: [{"/usr/lib/rabbitmq/plugins/jsx-2.9.0.ez",
duplicate_plugin}]
Error:
Error:
{:missing_dependencies, [:telemetry], [:ex_aws]}
I've checked if the installation procedure had changed but it hasn't so I'm not sure what's what.
getting 'module could not be loaded' string_compat,to_lower in the latest rabbitmq 3.8
Seems like string_compat no longer has the to_lower functionality and its causing it to crash.
│ 2021-04-05 17:38:53.411 [error] <0.993.0> Supervisor 'Elixir.ExAws.Supervisor' had child 'Elixir.ExAws.Config.AuthCache' started with 'Elixir.ExAws.Config.AuthCache':start_link([{name,'Elixir.ExAws.Config.AuthCache'}]) at <0.13951.54> exit with reason call to undefined function string_compat:to_lower("16 │ │ 17:38:53.415 [error] GenServer :rabbitmq_cloudwatch_exporter terminating │ │ ** (stop) exited in: GenServer.call(ExAws.Config.AuthCache, {:refresh_config, %{access_key_id: [{:system, "AWS_ACCESS_KEY_ID"}, :instance_role], host: "monitoring.us-east-1.amazonaws.com", http_client: ExAws.Request.Hackney, json_codec: Poison, port: 443, region: "us-east-1", retries: [max_attempts: 10, │ │ ** (EXIT) an exception was raised: │ │ ** (UndefinedFunctionError) function :string_compat.to_lower/1 is undefined (module :string_compat is not available)
cloudwatch exporter - exporting logs issue
Hi,
I have an issue when using cloudwatch exporter plugin. My Rabbitmq cluster consist of 3 nodes running on AWS EC2 instances.
I'm using IAM role with attached policy to grant access to logs and cloudwatch.
I want to use cloudwatch exporter plugin to export logs and metrics. Unfortunately it seems that only metrics are expoerted but not logs. I'm getting the following errors in the logs:
error] <0.354.0> Lager failed to install handler lager_cloudwatch_backend into rabbit_log_upgrade_lager_event, retrying later : {'EXIT',
{undef,
[{crypto,
hmac,
[sha256,
"AWS4NEh8lTR6nL32s1qzS++EpOZ6A/veQlpVIKjXgrYk",
"20210927"],
[]},
{erlcloud_aws,
signing_key,
4,
[{file,
"src/erlcloud_aws.erl"},
{line,
1128}]},
{erlcloud_aws,
sign_v4,
8,
[{file,
"src/erlcloud_aws.erl"},
{line,
1048}]},
{erlcloud_cloudwatch_logs,
make_request_headers,
3,
[{file,
"src/erlcloud_cloudwatch_logs.erl"},
{line,
703}]},
{erlcloud_cloudwatch_logs,
maybe_cw_request,
3,
[{file,
"src/erlcloud_cloudwatch_logs.erl"},
{line,
687}]},
{erlcloud_cloudwatch_logs,
describe_log_groups,
4,
[{file,
"src/erlcloud_cloudwatch_logs.erl"},
{line,
334}]},
{lager_cloudwatch_backend,
maybe_create_log_group,
1,
[{file,
"src/lager_cloudwatch_backend.erl"},
{line,
129}]},
{lager_cloudwatch_backend,
init,
1,
[{file,
"src/lager_cloudwatch_backend.erl"},
{line,
78}]}]}}
My rabbitmq.conf setting related to plugin:
cloudwatch_exporter.aws.region = eu-west-1
cloudwatch_exporter.metrics.overview.enable = true
cloudwatch_exporter.metrics.vhost.enable = true
cloudwatch_exporter.metrics.node.enable = true
cloudwatch_exporter.metrics.exchange.enable = true
cloudwatch_exporter.metrics.queue.enable = true
log.file.level = debug
and in advanced.conf
[{lager,
[{handlers, [{lager_cloudwatch_backend, [debug, "RabbitMQ", "rabbitmq-sandbox"]}]}]}].
Any advice with this issue ?
Error: dependent plugins [goldrush] not found; used by [lager].
Hello,
I am trying to install the cloudwatch exporter plugin as mentioned in the documentation. Rest is working, I even get success plugins are applied, except this.
Error :
Applying plugin configuration to rabbit@ip-13-0-4-182... failed.
Error: dependent plugins [goldrush] not found; used by [lager].
Code I used
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/base16-1.0.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/certifi-2.5.1.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/eini-git.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/elixir-1.8.2.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/erlcloud-3.3.2.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/ex_aws-2.1.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/ex_aws_cloudwatch-2.0.4.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/hackney-1.15.2.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/idna-6.0.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/jsx-2.9.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/lager-3.8.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/lager_cloudwatch-0.1.1.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/lhttpc-git.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/logger-1.8.2.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/metrics-1.0.1.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/mimerl-1.2.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/parse_trans-3.3.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/poison-3.1.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/singleton-1.2.0.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/ssl_verify_fun-1.1.5.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/v1.0.0/unicode_util_compat-0.4.1.ez
sudo wget https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/releases/download/0.3.1/rabbitmq_cloudwatch_exporter-0.3.1.ez
sudo rabbitmq-plugins enable rabbitmq_cloudwatch_exporter
I even installed erlang-goldrush package via apt-get.... No use. Any ideas.
The above error I had was on rabbit-3.6.2... When I change rabbitmq-3.8.2... I get a different error when enabling plugin :
sudo rabbitmq-plugins enable rabbitmq_cloudwatch_exporter
Enabling plugins on node rabbit@ip-13-0-4-159:
rabbitmq_cloudwatch_exporter
The following plugins have been configured:
rabbitmq_cloudwatch_exporter
rabbitmq_management
rabbitmq_management_agent
rabbitmq_web_dispatch
Applying plugin configuration to rabbit@ip-13-0-4-159...
Stack trace:
** (FunctionClauseError) no function clause matching in :rabbit_plugins.is_strictly_plugin/1
(rabbit) :rabbit_plugins.is_strictly_plugin(false)
(stdlib) lists.erl:1286: :lists."-filter/2-lc$^0/1-0-"/2
(rabbitmqctl) lib/rabbitmq/cli/plugins/commands/enable_command.ex:161: RabbitMQ.CLI.Plugins.Commands.EnableCommand.filter_strictly_plugins/3
(elixir) lib/stream.ex:1325: anonymous fn/2 in Stream.iterate/2
(elixir) lib/stream.ex:1512: Stream.do_unfold/4
(elixir) lib/stream.ex:1583: Enumerable.Stream.do_each/4
(elixir) lib/stream.ex:952: Stream.do_enum_transform/7
(elixir) lib/stream.ex:1583: Enumerable.Stream.do_each/4
:function_clause
And when I check rabbitmq enabled plugins, I get
sudo rabbitmq-plugins list
Listing plugins with pattern ".*" ...
Configured: E = explicitly enabled; e = implicitly enabled
| Status: * = running on rabbit@ip-13-0-4-159
|/
[ ] rabbitmq_amqp1_0 3.8.2
[ ] rabbitmq_auth_backend_cache 3.8.2
[ ] rabbitmq_auth_backend_http 3.8.2
[ ] rabbitmq_auth_backend_ldap 3.8.2
[ ] rabbitmq_auth_backend_oauth2 3.8.2
[ ] rabbitmq_auth_mechanism_ssl 3.8.2
[E*] rabbitmq_cloudwatch_exporter 0.3.1
need to avoid using access key and secret key
For configuration permissions usually assign role with required permissions. Need to use for example ec2 metadata endpoint for getting creds.
Error: plugin_built_with_incompatible_erlang
Hi,
I am trying to use the exporter on RabbitMQ installed on Amazon Linux 2018.03.
$ cat /etc/os-release
NAME="Amazon Linux AMI"
VERSION="2018.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2018.03"
PRETTY_NAME="Amazon Linux AMI 2018.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
erlang and rabbitmq versions.
# rpm -qa | grep rabbit
rabbitmq-server-3.6.11-1.el6.noarch
# rpm -qa | grep erlang
erlang-19.0.4-1.el6.x86_64
When enabling the plugin I receive the following:
# rabbitmq-plugins enable rabbitmq_cloudwatch_exporter
The following plugins have been enabled:
poison
unicode_util_compat
idna
mimerl
certifi
ssl_verify_fun
metrics
hackney
elixir
logger
ex_aws
ex_aws_cloudwatch
rabbitmq_cloudwatch_exporter
Applying plugin configuration to rabbit@fulger... failed.
Error: {plugin_built_with_incompatible_erlang,"certifi"}
I have also tried to install from source using elixir 1.7.4. I receive the following:
# make
erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[1]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/rabbit'
erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[2]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/rabbitmq_cli'
erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[3]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/observer_cli'
/root/rabbitmq-cloudwatch-exporter/deps/rabbitmq_cli/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[4]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/recon'
/root/rabbitmq-cloudwatch-exporter/deps/rabbitmq_cli/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
/root/rabbitmq-cloudwatch-exporter/deps/rabbitmq_cli/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[4]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/recon'
/root/rabbitmq-cloudwatch-exporter/deps/rabbitmq_cli/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[3]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/observer_cli'
make[2]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/rabbitmq_cli'
make[2]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/syslog'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[2]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/syslog'
make[2]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/ranch'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[2]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/ranch'
make[2]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/lager'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[3]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/goldrush'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[3]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/goldrush'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[2]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/lager'
make[2]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/ra'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[3]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/elvis_mk'
make[3]: Nothing to be done for `noop'.
make[3]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/elvis_mk'
make[3]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/aten'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[3]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/aten'
make[3]: Entering directory `/root/rabbitmq-cloudwatch-exporter/deps/gen_batch_server'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
make[3]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/gen_batch_server'
/root/rabbitmq-cloudwatch-exporter/deps/rabbit/erlang.mk:30: Please upgrade to GNU Make 4 or later: https://erlang.mk/guide/installation.html
ERLC noop.erl ra.erl ra_app.erl ra_directory.erl ra_env.erl ra_file_handle.erl ra_flru.erl ra_lib.erl ra_log.erl ra_log_ets.erl ra_log_meta.erl ra_log_segment.erl ra_log_segment_writer.erl ra_log_snapshot.erl ra_log_sup.erl ra_log_wal.erl ra_log_wal_sup.erl ra_machine.erl ra_machine_ets.erl ra_machine_simple.erl ra_metrics.erl ra_metrics_ets.erl ra_server.erl ra_server_proc.erl ra_server_sup.erl ra_server_sup_sup.erl ra_snapshot.erl ra_sup.erl ra_system_sup.erl
Inlining: inline_size=24 inline_effort=150
src/ra_server_proc.erl:953: illegal pattern
src/ra_server_proc.erl:956: variable 'S' is unbound
src/ra_server_proc.erl:957: illegal pattern
src/ra_server_proc.erl:962: variable 'E' is unbound
src/ra_server_proc.erl:962: variable 'S' is unbound
make[3]: *** [ebin/ra.app] Error 1
make[2]: *** [app] Error 2
make[2]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/ra'
make[1]: *** [deps] Error 2
make[1]: Leaving directory `/root/rabbitmq-cloudwatch-exporter/deps/rabbit'
make: *** [deps] Error 2
Please tell me if the plugin could functions with my installed versions, or should I update erlang and rabbitmq. Thank you :).
Metrics export not working in ME-south-1
We have been using RMQ along with this library in many AWS regions and it has been working great.
We found that the metrics are not exported in ME-south-1 due to an issue with the dependent library ex_aws.
I have raised a PR and the same has been merged in ex_aws.
ex-aws/ex_aws#845
Can we please update dependencies and release to fix this issue?
Thanks.
Errors in logs when publishing metrics
Hi:
I had install and also enable this plugin, and setup credential access of AWS. However, CloudWatch console did not received any metrics from RabbitMQ, and also there are no info or error log output from RabbitMQ logger, do you have any idea about how to process this?
Running two nodes of cluster
{rabbitmq_cloudwatch_exporter,"rabbitmq_cloudwatch_exporter","0.2.1"}
{rabbit,"RabbitMQ","3.7.16"},
{erlang_version,
"Erlang/OTP 21 [erts-10.3.5.1] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:64] [hipe]\n"}
Thanks
Unable to install version v1.0.0 on docker
- I'm unable to install your plugin version
v1.0.0. Below is theDockerfile
FROM rabbitmq:3.7.17
# v1.0.0
ENV CWPLUGIN https://api.github.com/repos/noxdafox/rabbitmq-cloudwatch-exporter/releases/28477282
RUN apt-get update -y \
&& apt-get install -y jq curl wget \
&& export DLS="$(curl $CWPLUGIN | jq -r '.assets[].browser_download_url')" && (for u in $DLS; do wget -P /plugins $u; done;) \
&& rabbitmq-plugins --offline enable rabbitmq_management rabbitmq_cloudwatch_exporter
- docker build crashed with below error
Problem reading some plugins: [{"/opt/rabbitmq/plugins/lager-3.6.10.ez",
duplicate_plugin}]
Problem reading some plugins: [{"/opt/rabbitmq/plugins/lager-3.6.10.ez",
duplicate_plugin}]
Problem reading some plugins: [{"/opt/rabbitmq/plugins/lager-3.6.10.ez",
duplicate_plugin}]
Error:
{:plugins_not_found, [:rabbitmq_cloudwatch_exporter]}
-
I also tried with
rabbitmq:3.8.5but it's still crashed with same ERROR -
However, if I change plugin to
v0.3.1, then it works!
FROM rabbitmq:3.7.17
# v0.3.1
ENV CWPLUGIN https://api.github.com/repos/noxdafox/rabbitmq-cloudwatch-exporter/releases/22416288
RUN apt-get update -y \
&& apt-get install -y jq curl wget \
&& export DLS="$(curl $CWPLUGIN | jq -r '.assets[].browser_download_url')" && (for u in $DLS; do wget -P /plugins $u; done;) \
&& rabbitmq-plugins --offline enable rabbitmq_management rabbitmq_cloudwatch_exporter
Need your help on this @noxdafox thanks
Metrics not reset (accumulating)
Apologies in advance if this is a usage error.
I'm using this great plugin for two weeks now and recently enabled the export of "Publish", "Deliver" and "Ack". My expectation was that the plugin would export the number of such events within the last minute, but it seems to me it is exporting the total since the server start instead:
This diagram should contain spikes, but it contains steps instead, i.e. it is accumulating the metrics, which is incorrect. I double checked against the graphs in the RabbitMQ Management plugin to validate. I also double checked the configuration in CloudWatch, where everything seems correct. I did not override the storage_resolution parameter in my plugin configuration.
Is this the expected behavior?
Support more fine-grained metrics publishing filter
Currently, several users are struggling to configure the plugin to contain CloudWatch costs.
A more fine-grained mechanism to configure which metrics to be published is required.
The following is a proposal for a new configuration syntax which is more namespace-based.
# A metric group needs to be enabled for export
# If no other option is specified, everything from the group is exported
cloudwatch_exporter.metrics.overview.enable = true
cloudwatch_exporter.metrics.overview.enable = true
# If types of a specific group are provided, only metrics from those types are exported
cloudwatch_exporter.metrics.overview.export_metrics.1 = "Messages"
cloudwatch_exporter.metrics.overview.export_metrics.2 = "MessagesReady"
cloudwatch_exporter.metrics.queue.enable = true
# For exchange, queue, connection and channel groups, it is possible to control the names of the entities to be published via regular expressions.
cloudwatch_exporter.metrics.queue.export_regex = "^(?!amq.gen-.*).*$"
# It is possible to combine both filters
cloudwatch_exporter.metrics.exchange.enable = true
cloudwatch_exporter.metrics.exchange.export_regex = "foo.*|.*bar"
cloudwatch_exporter.metrics.exchange.export_metrics.1 = "PublishIn"
TODO: provide equivalent in rabbitmq.config old configuration format.
Lager failed to install handler lager_cloudwatch_backend into lager_event
trying to set this up with rabbitmq but errors.
here's my advanced.config
[{lager,
[{handlers, [{lager_cloudwatch_backend, [debug, "RabbitMQ", "stream1"]}]}]}].
rabbitmq.confg
cloudwatch_exporter.aws.access_key_id="asdsadsa"
cloudwatch_exporter.aws.secret_access_key="asdsadsaada"
cloudwatch_exporter.aws.region="us-east-2"
cloudwatch_exporter.metrics.overview.enable = true
cloudwatch_exporter.metrics.vhost.enable = true
cloudwatch_exporter.metrics.node.enable = true
cloudwatch_exporter.metrics.exchange.enable = true
cloudwatch_exporter.metrics.queue.enable = true
log.file.level = debug
log.console = true
log.console.level = debug
log.connection.level = debug
log.channel.level = debug
loopback_users = none
and the error:
2021-03-07 15:56:48.294 [error] <0.289.0> Lager failed to install handler lager_cloudwatch_backend into lager_event, retrying later : {'EXIT',
{{case_clause,
{error,
{socket_error,
{econnrefused,
[{lhttpc_client,
send_request,
1,
[{file,
"src/lhttpc_client.erl"},
{line,
252}]},
{lhttpc_client,
execute,
9,
[{file,
"src/lhttpc_client.erl"},
{line,
201}]},
{lhttpc_client,
request,
9,
[{file,
"src/lhttpc_client.erl"},
{line,
101}]}]}}}},
[{lager_cloudwatch_backend,
maybe_create_log_group,
1,
[{file,
"src/lager_cloudwatch_backend.erl"},
{line,
129}]},
{lager_cloudwatch_backend,
init,
1,
[{file,
"src/lager_cloudwatch_backend.erl"},
{line,
78}]},
{gen_event,
server_add_handler,
4,
[{file,
"gen_event.erl"},
{line,
516}]},
{gen_event,
handle_msg,
6,
[{file,
"gen_event.erl"},
{line,
361}]},
{proc_lib,
init_p_do_apply,
3,
[{file,
"proc_lib.erl"},
{line,
226}]}]}}
i hope someone can help. thanks
fatal: repository 'https://github.com/rabbitmq/rabbitmq-stream-management.git/' not found
I was following the instructions of building from source. After I gave my github credentials, I got an error message fatal: repository 'https://github.com/rabbitmq/rabbitmq-stream-management.git/' not found. The link needs to be updated?
ubuntu@ip-172-31-63-96:~/rabbitmq-cloudwatch-exporter$ make dist
git clone https://github.com/ninenines/erlang.mk .erlang.mk.build
Cloning into '.erlang.mk.build'...
remote: Enumerating objects: 8844, done.
remote: Total 8844 (delta 0), reused 0 (delta 0), pack-reused 8844
Receiving objects: 100% (8844/8844), 5.35 MiB | 9.27 MiB/s, done.
Resolving deltas: 100% (5724/5724), done.
Checking connectivity... done.
if [ -f build.config ]; then cp build.config .erlang.mk.build; fi
cd .erlang.mk.build && make
make[1]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/.erlang.mk.build'
2018/12/03: Relx has been updated to v3.27.0.
2018/12/03: Dependencies are no longer always rebuilt by
default. Please check the documentation for
information. A quick fix if this causes an
issue is to add `FULL = 1` at the top of your
Makefile.
2018/12/06: Change the default COVER_DATA_DIR to be the
same as COVER_REPORT_DIR.
2019/06/20: `list-deps` now includes local applications
found in the `APPS_DIR` directory.
2020/03/25: The `-lerl_interface` argument has been removed
from the default `LDLIBS`. This is an old library
that is getting removed in OTP-23 and should no
longer be used. Note that the default still includes
`-lei` which is part of the Erlang/OTP Erl_Interface
*application* but is built as a separate C library.
The removal only applies to `-lerl_interface` itself.
2020/06/18: Concuerror integration has been added. It is
currently minimal but usable. Experimentation
and feedback is welcome.
2020/11/30: Support for publishing Hex releases and docs
has been added. It is currently experimental.
Feedback is more than welcome.
make[1]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/.erlang.mk.build'
cp .erlang.mk.build/erlang.mk ./erlang.mk
rm -rf .erlang.mk.build
git clone https://github.com/rabbitmq/rabbitmq-common .rabbitmq-components.mk.build
Cloning into '.rabbitmq-components.mk.build'...
remote: Enumerating objects: 9616, done.
remote: Total 9616 (delta 0), reused 0 (delta 0), pack-reused 9616
Receiving objects: 100% (9616/9616), 3.52 MiB | 0 bytes/s, done.
Resolving deltas: 100% (6708/6708), done.
Checking connectivity... done.
cp .rabbitmq-components.mk.build/mk/rabbitmq-components.mk ./rabbitmq-components.mk
rm -rf .rabbitmq-components.mk.build
DEP rabbit (master)
DEP rabbitmq_management (master) [85/1612]
DEP rabbitmq_management_agent (master)
DEP lager_cloudwatch (0.1.1)
No beam files found.
Recompile: src/rebar
Recompile: src/rebar_abnfc_compiler
Recompile: src/rebar_app_utils
Recompile: src/rebar_appups
Recompile: src/rebar_asn1_compiler
Recompile: src/rebar_base_compiler
Recompile: src/rebar_cleaner
Recompile: src/rebar_config
Recompile: src/rebar_core
Recompile: src/rebar_cover_utils
Recompile: src/rebar_ct
src/rebar_ct.erl:291: Warning: crypto:rand_uniform/2 is deprecated; use rand:rand_uniform/1 instead
Recompile: src/rebar_deps
Recompile: src/rebar_dia_compiler
Recompile: src/rebar_dialyzer
Recompile: src/rebar_edoc
Recompile: src/rebar_erlc_compiler
Recompile: src/rebar_erlydtl_compiler
Recompile: src/rebar_escripter
Recompile: src/rebar_eunit
src/rebar_eunit.erl:282: Warning: crypto:rand_uniform/2 is deprecated; use rand:rand_uniform/1 instead
Recompile: src/rebar_file_utils
Recompile: src/rebar_getopt
Recompile: src/rebar_lfe_compiler
Recompile: src/rebar_log
Recompile: src/rebar_metacmds
Recompile: src/rebar_mustache
Recompile: src/rebar_neotoma_compiler
Recompile: src/rebar_otp_app
Recompile: src/rebar_otp_appup
Recompile: src/rebar_port_compiler
Recompile: src/rebar_proto_compiler
Recompile: src/rebar_proto_gpb_compiler
Recompile: src/rebar_protobuffs_compiler
Recompile: src/rebar_qc
Recompile: src/rebar_rand_compat
Recompile: src/rebar_rel_utils
Recompile: src/rebar_reltool
Recompile: src/rebar_require_vsn
Recompile: src/rebar_shell
Recompile: src/rebar_subdirs
Recompile: src/rebar_templater
Recompile: src/rebar_upgrade
Recompile: src/rebar_utils
Recompile: src/rebar_xref
Recompile: src/rmemo
==> rebar (compile)
==> rebar (escriptize)
Congratulations! You now have a self-contained script called "rebar" in
your current working directory. Place this script anywhere in your path
and you can use rebar to build OTP-compliant apps.
/home/ubuntu/rabbitmq-cloudwatch-exporter
make[1]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit_common'
DEP rabbitmq_codegen (master)
DEP lager (3.8.0)
DEP jsx (2.11.0)
DEP ranch (1.7.1)
DEP recon (2.5.1)
DEP credentials_obfuscation (2.2.0)
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/lager'
DEP goldrush (0.1.9)
make[3]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/goldrush'
make[3]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/goldrush'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/lager'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/jsx'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/jsx'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/ranch'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/ranch'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/recon'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/recon'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/credentials_obfuscation'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/credentials_obfuscation'
make[1]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit_common'
make[1]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit'
DEP rabbitmq_amqp1_0 (master)
DEP rabbitmq_auth_backend_cache (master)
DEP rabbitmq_auth_backend_http (master)
DEP rabbitmq_auth_backend_ldap (master)
DEP rabbitmq_auth_backend_oauth2 (master)
DEP rabbitmq_auth_mechanism_ssl (master)
DEP rabbitmq_consistent_hash_exchange (master)
DEP rabbitmq_event_exchange (master)
DEP rabbitmq_federation (master)
DEP rabbitmq_federation_management (master)
DEP rabbitmq_jms_topic_exchange (master)
DEP rabbitmq_mqtt (master)
DEP rabbitmq_peer_discovery_aws (master)
DEP rabbitmq_peer_discovery_common (master)
DEP rabbitmq_peer_discovery_consul (master)
DEP rabbitmq_peer_discovery_etcd (master)
DEP rabbitmq_peer_discovery_k8s (master)
DEP rabbitmq_prometheus (master)
DEP rabbitmq_random_exchange (master)
DEP rabbitmq_recent_history_exchange (master)
DEP rabbitmq_sharding (master)
DEP rabbitmq_shovel (master)
DEP rabbitmq_shovel_management (master)
DEP rabbitmq_stomp (master)
DEP rabbitmq_stream (master)
DEP rabbitmq_stream_management (master)
Username:
Password:
remote: Repository not found.
fatal: repository 'https://github.com/rabbitmq/rabbitmq-stream-management.git/' not found
/bin/sh: 1: cd: can't cd to /home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbitmq_stream_management
erlang.mk:5087: recipe for target '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbitmq_stream_management' failed
make[1]: *** [/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbitmq_stream_management] Error 2
make[1]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit'
erlang.mk:8069: recipe for target '/home/ubuntu/rabbitmq-cloudwatch-exporter/.erlang.mk/recursive-deps-list.log' failed
make: *** [/home/ubuntu/rabbitmq-cloudwatch-exporter/.erlang.mk/recursive-deps-list.log] Error 2
ubuntu@ip-172-31-63-96:~/rabbitmq-cloudwatch-exporter$ sudo make dist
make[1]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit_common'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/lager'
make[3]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/goldrush'
make[3]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/goldrush'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/lager'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/jsx'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/jsx'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/ranch'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/ranch'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/recon'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/recon'
make[2]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/credentials_obfuscation'
make[2]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/credentials_obfuscation'
make[1]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit_common'
make[1]: Entering directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit'
DEP rabbitmq_stream_management (master)
Username:
Password:
remote: Repository not found.
fatal: repository 'https://github.com/rabbitmq/rabbitmq-stream-management.git/' not found
/bin/sh: 1: cd: can't cd to /home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbitmq_stream_management
erlang.mk:5087: recipe for target '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbitmq_stream_management' failed
make[1]: *** [/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbitmq_stream_management] Error 2
make[1]: Leaving directory '/home/ubuntu/rabbitmq-cloudwatch-exporter/deps/rabbit'
erlang.mk:8069: recipe for target '/home/ubuntu/rabbitmq-cloudwatch-exporter/.erlang.mk/recursive-deps-list.log' failed
make: *** [/home/ubuntu/rabbitmq-cloudwatch-exporter/.erlang.mk/recursive-deps-list.log] Error 2
No logs published to Cloudwatch
Hi,
it seems there is an issue with publishing logs to cloudwatch.
My cofiguration it as follows:
- RabbitMQ 3.9.8
- Erlang 24.1
- cloudwatch_exporter 1.0.1
my rabbitmq.conf
cloudwatch_exporter.aws.region = eu-west-1
cloudwatch_exporter.metrics.overview.enable = true
cloudwatch_exporter.metrics.vhost.enable = true
cloudwatch_exporter.metrics.node.enable = true
cloudwatch_exporter.metrics.exchange.enable = true
cloudwatch_exporter.metrics.queue.enable = true
advanced.config
[{lager,
[{handlers, [{lager_cloudwatch_backend, [info, "RabbitMQ"]}]}]}].
env variable set :
AWS_DEFAULT_REGION=eu-west-1
Metrics are published correctly but there there are no logs in cloudwatch.
I attached the following policy to my role to publish metrics and logs:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"cloudwatch:",
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:DescribeLogStreams"
],
"Resource": [
""
]
}
]
}
There are no errors in rabbitmq related to cloudwatch exporter plugin (even in debug mode).
Any suggestions what might prevent logs publishing to cloudwatch ?
Thanks,
Greg
Error: no function clause matching Exporter:handle_info
Hello, good job on the plugin, its really nice.
Although, i'm having some erros on rabbit's log that i would like to know more about them.
Info about mt environment:
- plugin version = 0.2.0
- erlang version = 21.3.2
- rabbit version = 3.7.14
My conf file:
loopback_users = none
cloudwatch_exporter.aws.access_key_id = "<MY_AWS_KEY>"
cloudwatch_exporter.aws.secret_access_key = "<MY_SECRET_KEY>"
cloudwatch_exporter.aws.region = "<MY_REGION>"
cloudwatch_exporter.metrics.1 = queue
cloudwatch_exporter.export_period = 1800
Error that is showing on logs:
2019-06-12 11:25:03.968 [error] <0.13189.130> Supervisor 'Elixir.Singleton.Supervisor' had child 'Elixir.Singleton.Manager' started with 'Elixir.Singleton.Manager':start_link('Elixir.RabbitMQCloudWatchExporter.Exporter', [], rabbitmq_cloudwatch_exporter, 'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ') at <0.2488.139> exit with reason no function clause matching 'Elixir.RabbitMQCloudWatchExporter.Exporter':handle_info({ssl_closed,{sslsocket,{gen_tcp,#Port<0.969884>,tls_connection,undefined},[<0.11783.137>,<0.11791.137>]}}, [[{period,1800},{collectors,[queue]},{namespace,<<"RabbitMQ">>}],[{secret_access_key,<<"MY_SECRET_KEY">>},...]]) line 52 in context child_terminated
In my understanding, it seems that somehow the plugin is trying to invoke a function called handle_info. Is that correct to say? I've looked in the code and i didn't find this function.
Additional info, i've installed this plugin in my test environment, and it seems to be working without showing this error. The difference between these servers (prod and test) is that the test environment is using rabbit 3.7.12 and erlang 21.1.
This error is NOT preventing the plugin from publishing metrics to cloudwatch, i am just curious about it, maybe it can be a problem in the future.
Export Logs into CloudWatch
For the v1 roadmap, the plan is to add logs exporting into CloudWatch as well.
To do so, we need to first develop a Lager backend for CloudWatch, then we can add it as a dependency to the plugin.
The plugin itself would simply take care of correctly configure the CW lager backend for the User.
- Add log group and stream creation to
erlcloud(PR #643) - Develop Lager backend for CloudWatch (https://github.com/noxdafox/lager_cloudwatch)
- Integrate Lager backend into the plugin
Update plugin configuration in runtime (without a restart)
I wanted to know if there is an option to update the plugin configuration without having a restart to the node.
For example: to update this parameter - cloudwatch_exporter.metrics.queue.export_regex
Issue on region regular expression matching
When I add the following region to my configuration :
{ex_aws, [{access_key_id, "***"},
{secret_access_key, "***"},
{region, "eu-west-3"}]},
{rabbitmq_cloudwatch_exporter, [{metrics, [overview, vhost, node, exchange, queue, connection, channel]}]}
I get the following errors in logs :
** Started from <0.613.0>
** When function == fun Elixir.RabbitMQ.CloudWatchExporter.Exporter:run/2
** arguments == [[{period,60},{collectors,[overview,vhost,node,exchange,queue,connection,channel]},{namespace,<<"RabbitMQ">>}],[]]
** Reason for termination ==
** {function_clause,[{'Elixir.Regex',run,[#{'__struct__' => 'Elixir.Regex',opts => <<>>,re_pattern => {re_pattern,1,0,0,<<69,82,67,80,121,0,0,0,16,0,0,0,65,0,0,0,255,255,255,255,255,255,255,255,0,0,45,0,0,0,1,0,0,0,64,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,131,0,53,27,133,0,9,0,1,29,117,29,115,119,0,7,29,101,29,117,119,0,7,29,97,29,112,119,0,7,29,115,29,97,119,0,7,29,99,29,97,120,0,37,29,45,95,11,29,45,95,7,25,120,0,53,0>>},re_version => {<<"8.41 2017-07-05">>,little},source => <<"^(us|eu|ap|sa|ca)\\-\\w+\\-\\d+$">>},"eu-west-3",[]],[{file,"lib/regex.ex"},{line,279}]},{'Elixir.Enum',find_list,3,[{file,"lib/enum.ex"},{line,2948}]},{'Elixir.ExAws.Config.Defaults',host,2,[{file,"lib/ex_aws/config/defaults.ex"},{line,53}]},{'Elixir.ExAws.Config.Defaults',get,2,[{file,"lib/ex_aws/config/defaults.ex"},{line,42}]},{'Elixir.ExAws.Config',build_base,2,[{file,"lib/ex_aws/config.ex"},{line,47}]},{'Elixir.ExAws.Config',new,2,[{file,"lib/ex_aws/config.ex"},{line,34}]},{'Elixir.ExAws',request,2,[{file,"lib/ex_aws.ex"},{line,49}]},{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,61}]}]}
2019-05-14 09:06:16.970 [error] <0.659.0> CRASH REPORT Process <0.659.0> with 0 neighbours crashed with reason: no function clause matching 'Elixir.Regex':run(#{'__struct__' => 'Elixir.Regex',opts => <<>>,re_pattern => {re_pattern,1,0,0,<<69,82,67,80,121,...>>},...}, "eu-west-3", []) line 279
2019-05-14 09:06:16.971 [error] <0.613.0> Supervisor {<0.613.0>,'Elixir.Supervisor.Default'} had child 'Elixir.RabbitMQ.CloudWatchExporter.Exporter' started with 'Elixir.RabbitMQ.CloudWatchExporter.Exporter':start_link([]) at <0.659.0> exit with reason no function clause matching 'Elixir.Regex':run(#{'__struct__' => 'Elixir.Regex',opts => <<>>,re_pattern => {re_pattern,1,0,0,<<69,82,67,80,121,...>>},...}, "eu-west-3", []) line 279 in context child_terminated
RabbitMQ errors in log file
Hi,
I just managed to configure the rabbitmq-cloudwatch-exporter plugin in my RabbitMQ cluster and it is sending metrics to CloudWatch but I am getting the following error messages into the log file at a regular interval (~2mins).
This is my configuration:
listeners.tcp.default = 5672
management.tcp.port = 15672
proxy_protocol = false
loopback_users.guest = false
vm_memory_high_watermark.relative = 0.850
default_vhost = dev
default_user = rabbitmq-user
default_pass = my_secure_password
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_aws
cluster_formation.aws.region = us-east-1
cluster_formation.aws.use_autoscaling_group = true
log.console = true
log.console.level = info
log.default.level = info
cloudwatch_exporter.aws.region = "us-east-1"
cloudwatch_exporter.metrics.1 = overview
cloudwatch_exporter.metrics.2 = vhost
cloudwatch_exporter.metrics.3 = node
cloudwatch_exporter.metrics.4 = exchange
cloudwatch_exporter.metrics.5 = queue
cloudwatch_exporter.metrics.6 = connection
cloudwatch_exporter.metrics.7 = channel
The following error messages are triggering our log file alarms:
2019-10-19 03:00:58.888 [error] <0.24899.20> ** Generic server rabbitmq_cloudwatch_exporter terminating
** Last message in was export_metrics
** When Server state == [[{period,60},{collectors,[overview,vhost,node,exchange,queue,connection,channel]},{namespace,<<"RabbitMQ">>}],[{region,<<"us-east-1">>}]]
** Reason for termination ==
** {#{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.17.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.18.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>49c3c87b-87d0-4247-9507-b11b6d063b4a</RequestId>\\n</ErrorResponse>\\n\", headers: [{\"x-amzn-RequestId\", \"49c3c87b-87d0-4247-9507-b11b6d063b4a\"}, {\"Content-Type\", \"text/xml\"}, {\"Content-Length\", \"401\"}, {\"Date\", \"Sat, 19 Oct 2019 03:00:58 GMT\"}, {\"Connection\", \"close\"}], status_code: 400}}}\n">>},[{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,66}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter',handle_info,2,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,60}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
2019-10-19 03:00:58.889 [error] <0.24899.20> CRASH REPORT Process <0.24899.20> with 1 neighbours crashed with reason: #{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.17.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.18.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>49c3c87b-87d0-424...">>} in 'Elixir.ExAws':'request!'/2 line 66
2019-10-19 03:00:58.891 [error] <0.189.0> Supervisor 'Elixir.Singleton.Supervisor' had child 'Elixir.Singleton.Manager' started with 'Elixir.Singleton.Manager':start_link('Elixir.RabbitMQCloudWatchExporter.Exporter', [], rabbitmq_cloudwatch_exporter, 'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ') at <0.23548.20> exit with reason #{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.17.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.18.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>49c3c87b-87d0-424...">>} in 'Elixir.ExAws':'request!'/2 line 66 in context child_terminated
dependency issue during v1.0.0 installation
Hello!
I am trying to install a plugin, downloaded *.ez files from v1.0.0 release, copied them to rabbitmq plugins directory but got next error:
rabbitmq-plugins list
Error: dependent plugins [elixir,jsx,lager,logger] not found; used by [erlcloud,
ex_aws,
ex_aws_cloudwatch,
lager_cloudwatch,
poison,
singleton,
ssl_verify_fun].
rabbitmq-plugins enable rabbitmq_cloudwatch_exporter
Error: dependent plugins [elixir,jsx,lager,logger] not found; used by [erlcloud,
ex_aws,
ex_aws_cloudwatch,
lager_cloudwatch,
poison,
singleton,
ssl_verify_fun].
Fail to build on Elixir 1.9
Hi,
I tried to build from sources cloudWatch exporter and unfortunately it fails with Elixir 1.9 with the following error:
** (UndefinedFunctionError) function Mix.Dep.loaded/1 is undefined or private
(mix) Mix.Dep.loaded([env: :dev])
lib/mix/tasks/archive/build.deps.ex:72: Mix.Tasks.Archive.Build.Deps.list/1
lib/mix/tasks/archive/build.deps.ex:54: Mix.Tasks.Archive.Build.Deps.build_archives/1
lib/mix/tasks/archive/build.all.ex:62: Mix.Tasks.Archive.Build.All.run/1
(mix) lib/mix/task.ex:331: Mix.Task.run_task/3
(elixir) lib/enum.ex:783: Enum."-each/2-lists^foreach/1-0-"/2
(elixir) lib/enum.ex:783: Enum.each/2
(mix) lib/mix/task.ex:331: Mix.Task.run_task/3
make[1]: *** [/tmp/rabbitmq-cloudwatch-exporter/deps/rabbit_common/mk/rabbitmq-dist.mk:185: plugins/rabbitmq_cloudwatch_exporter-0.2.1.ez] Error 1
make[1]: Leaving directory '/tmp/rabbitmq-cloudwatch-exporter'
make: *** [/tmp/rabbitmq-cloudwatch-exporter/deps/rabbit_common/mk/rabbitmq-dist.mk:201: dist] Error 2
It seems that Mix.Dep.loaded has been removed from Elixir 1.9.
How to reproduce:
Start an Elixir 1.9.1 container with docker run -it elixir:1.9.1 /bin/bash
Then build the plugin as usual:
apt-get update && apt-get install -y zip rsync
git clone https://github.com/noxdafox/rabbitmq-cloudwatch-exporter.git
cd rabbitmq-cloudwatch-exporter
make dist
Best,
Adrien
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.