There are two ways to cluster RabbitMQ. The first is to use the commands provided by rabbitmqctl and the second is to use configuration. This tutorial will only demonstrate the first way, in part, because it can be done in a consistent fashion. We encountered a couple of issues using "Auto-configuration" which caused contention between nodes. This may have been an error on our part, but given the fact that the rabbitmqctl method is actually easier, we didn't invest any time into figuring out the error in our ways.
Ensure you can reach all the nodes in your cluster.
If you have been following this tutorial, you will note that we have not setup DNS or edited the hosts file. For this tutorial we will add some entries into the host file as a simple solution. If you already have networking setup in your cluster and can reach nodes by hostname, please ignore.
a. Find out the IP's of each node in your cluster.
On each server, execute:
Write down the IP address of that server. It will probably be the inet addr entry under either eth0 or eth1 (left column).
b. Edit the /etc/hosts file of each server.
sudo vi /etc/hosts
Add an entry in the hosts file for every server in the cluster but the one you are physically on:
We've also added the rabbit3 entry the localhost so the machine knows it should loopback to itself.
Save the file and you are done. Make sure all the nodes can reach each other:
Press control-z to stop the ping command from pinging the server.
Ensure all nodes share the same Erlang cookie.
The Erlang Distribution protocol will have Erlang processes authenticate with each other to ensure those processes are allowed to interact. The algorithm used to perform the authentication is hash-based, that hash being stored in the "cookie" file.
a. Chose a server in your cluster whose cookie will serve as the "canonical" cookie of the cluster. Which server doesn't particularly matter.
b. Copy the cookie from that server to all other servers participating in the cluster. The cookie is stored in the/var/lib/rabbitmq directory and is called .erlang.cookie. Please note the period (.) in the name of the cookie (a hidden file in UNIX), so it won't come up in a ls command unless you use the -a switch (ls -a).
We recommend using scp to perform the copy, but you may choose any mechanism you want. We will demonstrate thescp method.
Please note that there is no period "." in the name of erlang.cookie file we are transferring to the rabbit3 server. This is because Linux will tell you Permission denied because it does not allow the creation of hidden files from a remote machine. This is also the reason why we are copying the file to the home directory and not directly to /var/lib/rabbitmq.
If you do not have scp installed on your machine (which is likely if you are using CentOS minimal profile), you can install it using sudo yum install openssl-client.
Move the cookie to the /var/lib/rabbitmq directory:
And we've added the period "." back to the file's name.
This next step is precautionary. Moving the cookie from one server to another may cause the file to be unaccessible to therabbitmq user account (which runs the server). With this in mind, we will transfer ownership of the cookie file to the rabbitmq user:
To cluster a RabbitMQ node, you need to stop the RabbitMQ application on the Erlang VM, reset the node's configuration, tell it to going the cluster, and then start back up.
Stop the broker:
sudo rabbitmqctl stop_app
Reset the configuration on the node. This is only necessary if your node was previously joined to another cluster.
sudo rabbitmqctl reset
Join the cluster:
sudo rabbitmqctl join_cluster rabbit@rabbit1
Start the broker:
sudo rabbitmqctl start_app
rabbit@rabbit1 probably doesn't mean what you think. It actually means the Erlang Distribution identifier for a process called "rabbit" on the host "rabbit1". When joining a cluster, there isn't any kind of "master" or "leader" node. You could actually join the cluster by specifying rabbit@rabbit2 and it would work identically.
By default, your node will be a "disk node". Disk nodes store their state both on disk and in memory. You can choose to have "ram nodes" which store their state in memory, unless a queue declared on that broker is configured to be persistent.
rabbitmqctl offer the capability to determine the cluster's status from the shell:
sudo rabbitmqctl cluster status
The RabbitMQ Management Console is probably the best way to see the cluster's status, providing rich statistics about each node in the cluster. Simply navigate to one of your Console instances in the browser.