Daemonizing Ruby Processes Responsibly

Introduction #

this week, During the recent Jakarta.rb meet-up held at Midtrans HQ, I had the chance to share my insights on the topic of daemonizing Ruby processes. This post will recap that discussion, giving you a deeper understanding of what daemon processes are, why it’s necessary to manage them responsibly, and how to achieve this in Ruby.

What is a Daemon Process? #

To begin with, let’s define what a daemon process is. In computing, a daemon is a long-running background process that answers requests for services. The term originated with Unix, but most operating systems use daemons in some form or another. For more details, you can visit this link.

Why is Responsibly Necessary? #

One may wonder why we need to put an emphasis on daemonizing processes responsibly. The answer is simple: to take control of your process as much as possible. By managing your daemon processes responsibly, you maintain a stable and efficient system environment. below is the classic way to run a long process. where you don’t have control to manage the process to run you application gracefully.

# file : app-classic.rb
module MyApp
  extend self
  def start
    # do long process
    loop do
      sleep 1
      puts Time.now.to_i
    end
  end
end
MyApp.start

How to Create Daemon Processes in Ruby Responsibly #

Creating a daemon process in Ruby involves several key steps:

1. Separate your application logic from the main process: This ensures that your application logic is isolated, making it easier to manage and less likely to disrupt the main process.

# file : cli
require_relative 'daemon'
require_relative 'app-responsibly'

Daemon.new(watch_process: true, logfile: './cli.log', pidfile: './cli.pid')
  .process{ MyApp.start }
  .on_interrupt{ MyApp.on_interrupt }
  .start

the code above experess, daemon & app-responsibly are separated, the way Daemon class initiate are clearly show how the MyApp is integrated.

1. Trap the signal from the main process, not from your application logic: Signals sent to your process should be handled at the process level, not within your application logic. This separation helps to prevent unexpected behavior in your application.

# file : daemon.rb
require 'logger'
Class Daemon
  def start_main_process
    begin
      signal_pipe = []

      %w{INT TERM USR1 USR2 TTIN HUP}.each do |signal|
        begin
          trap signal do
            signal_pipe << signal
          end
        rescue ArgumentError
          puts "Signal #{signal} not supported"
        end
      end

      @process_thread = safe_thread { @process.call }
      while true
        if readable_signal = signal_pipe.shift
          signal = readable_signal.strip
          handle_signal(signal)
        end

        watch_process_thread
      end
    rescue Interrupt
      stop
      exit(0)
    end
  end

  def safe_thread
    Thread.new do
      begin
        yield
        true
      rescue Interrupt
        stop
        false
      rescue => e
        tag = "[PROCESS ERROR] ".freeze
        logger.error tag + e.class.to_s
        logger.error tag + e.message
        logger.error tag + e.backtrace.join("\n")
        false
      end
    end
  end

  def handle_signal signal_code
    print "\n"
    logger.debug "Receiving #{signal_code} signal"
    case signal_code
    when 'INT', 'TERM'
      raise Interrupt
    when 'USR1'
      logger.info "Receives USR1, stop consuming event"
    when 'USR2'
      logger.info "Receives USR2, reopening log file"
      # NOTE: send to logger to reopen the log file
    when 'TTIN'
      logger.info "Receives TTIN, inspecting consumer thread"
      # NOTE trigger poll manager to inspect current state of each consumer thread
    when 'HUP'
      logger.info "Receives HUP, reloading configuration"
      # NOTE: reload configuration
    end
  end

end

the main thread on this process is responsible to watch the signal from the process, while the application is run in separate thread through safe_thread method.

1. Monitor application status at all times: Regular monitoring allows you to catch any issues as soon as they arise, ensuring the stability and reliability of your application.

  def watch_process_thread
    return unless @watch_process
    thread_alive = @process_thread.alive?
    return if thread_alive

    thread_value = @process_thread.value

    if thread_value == true
      exit(0)
    else
      exit(1)
    end
  end

1. Write the process ID (PID) to a file: This allows you to easily track and manage your process.

  def run
    daemonize
    write_pid

    start_main_process
  end

  def write_pid
    if path = pidfile
      pidfile = File.expand_path(path)
      File.open(pidfile, 'w') do |f|
        f.puts ::Process.pid
      end
    end
  end

1. Redirect logs: Logs should be redirected to a location where they can be monitored and reviewed, enabling you to analyze the behavior of your daemon process.

  def daemonize
    ::Process.daemon(true, true)

    [$stdout, $stderr].each do |io|
      File.open(logfile, 'ab') do |f|
        io.reopen(f)
      end
      io.sync = true
    end

    $stdin.reopen('/dev/null')
    initialize_loggger
  end

logging and claiming the PID is the first thing the process should do before anything else.

Demo #

In the meet-up, I demonstrated how to practically apply these steps in Ruby, you can find the source code of the demonstration on my GitHub. and the Deck from here

Conclusion #

Daemonizing processes is a powerful technique that can enhance the performance and reliability of your Ruby applications. However, it is crucial to manage these processes responsibly to avoid potential issues. By following the steps outlined in this post, will help you mastering the art of daemonizing Ruby processes responsibly.