Tracking down issues by searching though log files can be difficult, especially if your application is deployed across multiple instances. Thankfully this can be made easier by offloading all of your Rails logs to an external ELK stack which consists of Elastic Search, LogStash and Kibanna.

To get our ELK stack up and running we will use Docker and Docker Compose pulling the hugely popular sebp/elk docker image. As it is though, sebp/elk isn’t pre-configured with support for UDP so we need to add a few modifications for that.

Updated For Rails 6 - 10th of October 2019

Following some requests I have now updated this guide to work with Rails 6 and provided an example application, put together using this guide, that is available over on GitHub.

Dockerfile configuration

Firstly create a new Dockerfile pulling from the sebp/elk image and set our working directory to ${LOGSTASH_HOME}. Then for LogStash to support receiving events over UDP we need to add a couple of plugins, these are logstash-input-udp for enabling the UDP protocol, and logstash-codec-json_lines as we are going to send or logs across in this format.

# ELK Stack Dockerfile with UDP
from sebp/elk

WORKDIR ${LOGSTASH_HOME}

# Plugins to install
RUN gosu logstash bin/logstash-plugin install logstash-input-udp
RUN gosu logstash bin/logstash-plugin install logstash-codec-json_lines

Next we need to configure a LogStash listener so it knows hoe to receive the logs from our Rails Application. Create a new file called rails-udp-input.conf alongside your Dockerfile and place the following content in it. This simply tells LogStash what protocol and port it should listen on along with the codec that it should use to deserialize any logs received.

input {
  udp {
    port => 5228
    codec => json_lines
  }
}

Now we need to tell LogStash to use this config file by adding the following to our Dockerfile.

# Add LogStash Listeners
ADD ./rails-udp-input.conf /etc/logstash/conf.d/rails-udp-input.conf

Docker Compose configuration

To run your ELK stack you can put the following into a docker-compose.yml file alongside your Dockerfile, you can then run your stack with the docker-compose up -d command. The important thing to note here, is that for any LogStash listeners you have added you will need to add a port mapping for them in the compose configuration. In this instance I have added - "5228:5228/udp", the internal port mapping also needs to be prepended with /udp as our listener is using the UDP protocol.

version: "3"

services:
  elk:
    build: .
    ports:
      - "5228:5228/udp"
      - "5601:5601"
      - "9200:9200"
      - "5044:5044"
    volumes:
      - elk-data:/var/lib/elasticsearch

volumes:
  elk-data:

Rails configuration

Now that our ELK stack is up and running we need to tell our Rails Application to start sending its logs to it. For this we will use the logstash-logger gem which you will need to add to your Gemfile as below.

# Add LogstashLogger for sending logs to Elk
gem 'logstash-logger'

Then we need to create a logstash initilaizer under config/initializers/logstash.rb with the contents below. You will also need to make LOGSTASH_HOST and LOGSTASH_PORT available as environment variables, note that the value of LOGSTASH_HOST should not include any protocol such as https:// or http:// as we are using UDP.

# config/initializers/logstash.rb
# Configure LogStashLogger to send logs to our remote Elk stack in production
if ENV["LOGSTASH_ENABLED"] == "true"
  Rails.logger = LogStashLogger.new(
    type: :multi_delegator,
    outputs: [
      { type: :file, path: "log/#{Rails.env}.log" },
      { type: :udp, host: ENV["LOGSTASH_HOST"], port: ENV["LOGSTASH_PORT"] }
    ]
  )

  # Add metadata for Logstash
  LogStashLogger.configure do |config|
    config.customize_event do |event|
      event["application"] = Rails.application.config.session_options[:key].sub(/^_/,"").sub(/_session/,"")
      event["environment"] = Rails.env

      event["@metadata"] = {
        beat: event["application"]
      }
    end
  end
end

Thats it! You should now be able to run your Rails application and see your logs appearing in the Losgstash instance.

For further reference you can find a working example of a Rails application using Logstash Logging as detailed in this guide over on GitHub.