Setup Logstash on DigitalOcean with a Rails App hosted on Heroku : Full SSL

Logstash is a part of the ELK (Elasticsearch — Kibana — Logstash) stack, from Elastic. Its a centralized log drain for all your logs from various sources.

I couldn’t find any nice tutorials on setting up Logstash on a DigitalOcean droplet, and sending logs to it from a Heroku Rails App, over TCP with SSL. Certificate creation also seems poorly documented online.

So let’s get to it.

DigitalOcean Droplet Setup

Use this Github Repo to setup basic packages.

Postfix-Mailgun-Setup

The tutorial is pretty good, provided you already have mailgun configured.

Certificates/Logstash Setup & Installation:

A word of caution!

I spent nearly 6–8 hours on all this and eventually realized that there is no simple tool to automate the process. (6–8 hours is a long time for me)

I created a Github Repository, that you can find below, that does all the heavy lifting for you. I would strongly suggest you use it, rather than tinker with OpenSSL on your own.

More importantly, I’ve included a simple sh script to check whether Logstash responds to your client over SSL. It took me a long time to figure out how to even test the SSL connection.

Full instructions on my Github Repo.

Rails Heroku Logger Setup

Gems

gem “lograge”
gem ‘logstash-event’
gem ‘logstash-logger’

Configure Rails App For Heroku

I’m outlining my configuration, but it can get messy, primarily because I used Rails.logger.(debug/error) all throughout my application. So it became necessary to extend Rails.logger to output its stuff to Logstash in a json friendly format.

Create a module in your /lib directory:

This simplifies setting up the SSL Connection.

module SslModule    class Context

def self.certificates_path
Rails.root.join("config","certificates","certificates")
end
def self.get_context
ssl_context = OpenSSL::SSL::SSLContext.new()
ssl_context.cert = OpenSSL::X509::Certificate.new(File.open("# {certificates_path}/ssl_certificates/host/host.crt"))
ssl_context.ca_file = "#{certificates_path}/ssl_certificates/ca/ca.crt"
ssl_context.key = OpenSSL::PKey::RSA.new(File.open("#{certificates_path}/ssl_certificates/host/host.key"))
ssl_context.verify_mode = OpenSSL::SSL::VERIFY_PEER
ssl_context.ssl_version = :SSLv23
ssl_context
end
def self.get_machine_ip
ip = Socket.ip_address_list.detect{|intf| intf.ipv4_private?}
ip.ip_address
end
def self.get_logstash_ip
machine_ip = get_machine_ip
#puts "machine ip is :#{machine_ip}"
#puts "ip address is #{ENV['IP_ADDRESS']}"
return ENV["LOGSTASH_SERVER_IP_ADDRESS"] unless ENV["LOGSTASH_SERVER_IP_ADDRESS"].blank?
unless ENV["LOGSTASH_SERVER_IP_ADDRESSES"].blank?
ENV["LOGSTASH_SERVER_IP_ADDRESSES"].split(",").each do |ip|
if machine_ip == ip
return ip
end
end
end
return "0.0.0.0"
end
##should check eth for the ip address or hit 0.0.0.0
## if more than one ip address is specified.
## if no IP_ADDRESS is specified, defaults to eth / or 0.0.0.0
def self.logstash_logger
ls_ip = get_logstash_ip
#puts "ls ip is #{ls_ip}"
LogStashLogger.new(type: :tcp, host: get_logstash_ip, port: ENV["LOGSTASH_PORT"], ssl_context: get_context)
end
end
end

Create an initializer:

#logger.rb
Rails.logger = ::SslModule::Context.logstash_logger
Delayed::Worker.logger = Rails.logger

Add the following to your environment files:

#development.rb
Rails.application.configure do
config.use_logstash = false
config.lograge.enabled = false

config.lograge.logger = Rails.logger
config.lograge.formatter = Lograge::Formatters::Logstash.new
config.log_tags = [:uuid]
config.lograge.custom_payload do |controller|
{
host: controller.request.host,
user_id: controller.current_user.try(:id),
request_id: controller.request.request_id || RequestStore.store[:request_id],
params: controller.request.params.to_s
}
end
end

This should get everything working, and you should see the logs popping up in your logstash server.

Comment if you face any problems!

All the best, this is not easy.

Post-Graduate in Clinical Pathology, Lab Director - Pathofast, Computer Vision Enthusiast, Founder algorini.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store