Application doesn't work because of AMQP issue

[zulip]

Application doesn't work because of AMQP issue

[SystemZ]

Thx for link @timabbott ! This solves problem for me :) Zulip works even after vps reboot, finally 😄

service supervisor stop
dpkg -P rabbitmq-server
apt-get install rabbitmq-server
/root/zulip/scripts/setup/configure-rabbitmq
service supervisor start

I'm stopping supervisor because bad AMQP restarts workers every second, causing 100% CPU.

It looks like installation of rabbit-mq goes bad in zulip setup script or maybe it's my env (OpenVZ container).

[timabbott]

Looks like a clear case of the RabbitMQ server it depends on either isn't running or isn't configured properly.

You should have a process with "beam.smp" in its name in your full ps output if rabbitmq is running; if it's not, you should debug that (check rabbitmq-server is installed, service restart it, etc.).

Otherwise try running scripts/setup/configure-rabbitmq

[myshov]

@timabbott yes after restarting rabbitmq everything is working now
For some reasons rabbitmq was crashed or killed or I don't know. In the log there are nothing:

=INFO REPORT==== 26-Sep-2015::22:50:18 ===
accepting AMQP connection <0.1905.0> (127.0.0.1:42031 -> 127.0.0.1:5672)

=INFO REPORT==== 27-Sep-2015::15:53:41 ===
Starting RabbitMQ 3.2.4 on Erlang R16B03
Copyright (C) 2007-2013 GoPivotal, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

=INFO REPORT==== 27-Sep-2015::15:53:41 ===
node           : rabbit@ubuntu
home dir       : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/rabbitmq.config
cookie hash    : 4jnxrW5/B/MI3gsF51Oesw==
log            : /var/log/rabbitmq/[email protected]
sasl log       : /var/log/rabbitmq/[email protected]
database dir   : /var/lib/rabbitmq/mnesia/rabbit@ubuntu

=INFO REPORT==== 27-Sep-2015::15:53:42 ===
Limiting to approx 924 file handles (829 sockets)

As you can see above in the log there was the last connection yesterday and after that rabbitmq crashed or something. And after I restarted it and now it's working. By the way there are no errors in the other logs of rabbitmq.

[timabbott]

Yeah rabbitmq is not great about providing verbose logging information. If it crashed due to an OOM kill or something, there could be something in dmesg/syslog/etc., so maybe worth checking those to see if there's any evidence for what happened.

[SystemZ]

In my case rabbitmq-server never starts

service rabbitmq-server restart
 * Restarting message broker rabbitmq-server   [fail] 

I tried more fresh build of erlang with

echo "deb http://packages.erlang-solutions.com/ubuntu trusty contrib"  > /etc/apt/sources.list.d/erlang.list
apt-get update
apt-get install erlang

But it didn't help.
I'm getting

ERROR: epmd error for host "vps124": address (cannot connect to host/port)

in /var/log/rabbitmq/startup_err

[timabbott]

Try purging and reinstalling the rabbitmq-server package, following these instructions:

http://www.techsfo.com/blog/2013/06/rabbitmq-breaks-when-you-rename-hostname/

Please report back whether that works!

[0ptimus]

I believe that I am having similar issues to this ticket. Getting "500: server error" when clicking link from email to activate new account I checked my log (/var/log/rabbitmq/[email protected]) and I see a lot of these errors...

=ERROR REPORT==== 29-Sep-2015::13:23:35 ===
closing AMQP connection <0.520.0> ([::1]:43088 -> [::1]:5672):
{handshake_error,starting,0,
                 {amqp_error,access_refused,
                             "PLAIN login refused: user 'zulip' - invalid credentials",
                             'connection.start_ok'}}

=ERROR REPORT==== 29-Sep-2015::13:23:37 ===
closing AMQP connection <0.524.0> ([::1]:43089 -> [::1]:5672):
{handshake_error,starting,0,
                 {amqp_error,access_refused,
                             "PLAIN login refused: user 'zulip' - invalid credentials",
                             'connection.start_ok'}}

I am not familiar with troubleshooting rabbitmq, can anyone provide some tips to fix this? Want to try out Zulip for my IT dept.

[timabbott]

This issue is most likely that somehow the password Zulip is using doesn't
match the password rabbitmq has configured --
scripts/setup/configure-rabbitmq should fix this.

I should add you'll want to restart the Zulip server via supervisorctl restart all afterwards

[timabbott]

rabbitmq-server installation seems to have problems in environments with weird networking configurations; so yeah this probably is an OpenVZ container problem...

[jmandel]

@SystemZ thanks -- I needed to follow your purge-then-reinstall suggestion on a fresh Ubuntu 14.04 install (following the default instructions from https://zulip.org/server.html)

[3ruce]

Yep, this also worked for me after a reboot...

service supervisor stop
dpkg -P rabbitmq-server
apt-get install rabbitmq-server
/root/zulip/scripts/setup/configure-rabbitmq
service supervisor start