\r\nrabbitmq-plugins.bat enable rabbitmq_management\r\n<\/pre>\nThis command will prepare RabbitMQ<\/i> management website, so it will be easier for us to see what is going on in given queue. In the next step, we have to prepare the queue, the logs will be sent to. You can do it via the website we’ve just enabled (http:\/\/localhost:15672\/<\/i>) or via RabbitMQ<\/i> admin console. As I prefer to automate things as much as possible I will do it via command line. What is quite unusual when it comes RabbitMQ CLI<\/i> is the fact that it is a python script you have to download and run locally (this is not an executable). The script can be found on management site under this address<\/a>. Once the script is downloaded (in my case it is saved as rabbitmqadmin.py<\/i>) you can start preparing necessary elements: exchange, queue and the binding.<\/p>\n\r\npython rabbitmqadmin.py declare exchange name=logger type=topic -u username -p password\r\n<\/pre>\n\r\npython rabbitmqadmin.py declare queue name=MyAppLogginQueue auto_delete=false durable=true -u username -p password\r\n<\/pre>\n\r\npython rabbitmqadmin.py declare binding source=logger destination=MyAppLogginQueue routing_key=MyApp -u username -p password\r\n<\/pre>\nAs you can see I’ve created exchange called logger<\/i> which is bound to MyAppLogginQueue <\/i> queue using MyApp<\/i> route. This means that every message with topic MyApp<\/i> sent to logger<\/i> exchange will be pushed to MyAppLogginQueue <\/i>.<\/p>\n3. Preparing Logstash<\/h3>\n
Logstash<\/i> configuration will be modified version of my previous<\/a> config. I will just add another input source. Here is a basic usage<\/p>\n\r\ninput {\t\r\n\trabbitmq {\r\n host => \"localhost\"\r\n queue => \"MyAppLogginQueue\"\r\n heartbeat => 30\r\n durable => true\r\n password => \"password\"\r\n user => \"username\"\r\n }\r\n}\r\n<\/pre>\nAs you can see we will be consuming messages from MyAppLogginQueue<\/i> which is deployed on localhost<\/i>. For password<\/i> and user<\/i> properties use your own credentials. That is basically it, so now it is time to see if everything is working.<\/p>\n4. Testing coniguration <\/h3>\n
In order to test the configuration you have to run the Elasticsearch<\/i>, Kibana<\/i> and use new config for Logstash<\/i>. I’ve shown how to do it in one of my recent post <\/a>. For sending messages to the queue I will just use RabbitMQ<\/i> management website API<\/i>. The API<\/i> exposes <\/p>\n\r\napi\/exchanges\/%2F\/{exchange_name}\/publish\r\n<\/pre>\nendpoint accepting POST <\/i> verbs which can be used for publishing messages to given exchange. In my case POST<\/i> body will look as follows<\/p>\n\r\n{\r\n \"vhost\": \"\/\",\r\n \"name\": \"logger\",\r\n \"properties\": {\r\n \"delivery_mode\": 2,\r\n \"headers\": {}\r\n },\r\n \"routing_key\": \"MyApp\",\r\n \"delivery_mode\": \"2\",\r\n \"payload\": \"TimeStamp=2016-11-01 00:13:01.1669 CorrelationId=77530786-8e6b-45c2-bbc1-31837d911c14 Level=INFO Message=Request completed with status code: 200\",\r\n \"headers\": {},\r\n \"props\": {},\r\n \"payload_encoding\": \"string\"\r\n}\r\n<\/pre>\nand I will be sending it to<\/p>\n
\r\nhttp:\/\/localhost:15672\/api\/exchanges\/%2F\/logger\/publish\r\n<\/pre>\nNote that I will be sending messages to the exchange, not to the queue itself. The exchange’s responsibility is to route the message to all bound queues. Here is how it looks in practice
\n![\"ezgif-com-gif-maker\"](\"http:\/\/tpodolak.com\/blog\/wp-content\/uploads\/2016\/11\/elastic-stack-reading-logs-queue\/ezgif.com-gif-maker.gif\")
<\/a>
\nAs you can see our configuration is valid and messages are shown on Kibana’s<\/i> dashboard almost in real time. <\/p>\n