SOA - Introducing RackRabbit

Thu, Sep 4, 2014

In this series we have discussed:

an introduction to SOA
implementing SOA using HTTP
implementing SOA using AMQP

We talked about why we might implement a Service Oriented Architecture in order to scale to meet…

the demands of our users
the demands of our dev team

We also talked a little bit of theory on what an SOA looks like, how to define the boundaries for our services, the practical implications of building a distributed system, and the 3 main communication patterns of a distributed system:

Synchronous Request/Response
Asynchronous Worker Queue
Asynchronous Publish/Subscribe

When implementing SOA using HTTP, we discovered that we have access to a robust set of technical options such as Rack, Unicorn, HTTParty, Faraday, Redis, Resque, or Beanstalkd, but no single technology can provide for all 3 communication patterns.

When implementing SOA using AMQP, we discovered that we can implement all 3 patterns using a single technology, but the implementation details are a little more tricky, programming directly to the AMQP interface, and the infrastructure and community support for hosting production-ready consumers are lacking (e.g. there is no easy-to-program-to Rack interface, or Unicorn style server for easily load balancing RabbitMQ consumer processes).

Just like everything in our profession, there are trade-offs.

Introducing RackRabbit

I really like the idea of having a single technology for our distributed communication, so in order to make the concept of SOA using AMQP easier to manage I have created the RackRabbit project to provide:

A Unicorn-style forking server for hosting AMQP subscribers as load balanced applications implemented using any Rack-compliant framework (e.g. Sinatra)
A simple client library for communicating with those services using the 3 primary communication patterns of a distributed system, synchronous request/response, asynchronous worker queue, and asynchronous publish/subscribe.

You can find the full documentation, code, tests, and examples on Github:

What Unicorn does for HTTP services, RackRabbit can do for hosting AMQP services, and more:

	HTTP	AMQP
Make a synchronous request/response	Unicorn	rabbitMQ + RackRabbit
Asynchronous worker queue	Redis + Resque	rabbitMQ + RackRabbit
Asynchronous publish/subscribe	Redis	rabbitMQ + RackRabbit

Example of Synchronous Request/Response (using Sinatra)

Consider this simple sinatra service in config.ru:

require 'sinatra/base'

class Service < Sinatra::Base

  get "/hello" do
    "Hello World"
  end

  post "/submit" do
    "Submitted #{request.body.read}"
  end

  put "/update" do
    "Updated #{request.body.read}"
  end

  delete "/resource" do
    "Deleted resource"
  end

end

run Service

Host and load balance this service using rack-rabbit:

$ rack-rabbit --queue myqueue --workers 4 config.ru

Connect to the worker from the command line using the rr command:

$ rr request -q myqueue GET /hello
Hello World

$ rr request -q myqueue POST /submit "data"
Submitted data

$ rr request -q myqueue PUT /update "data"
Updated data

$ rr request -q myqueue DELETE /resource
Deleted resource

Connect to the worker from our application using the RR class:

require 'rack-rabbit/client'

RR.get    :myqueue, "/hello"              # returns "Hello World"
RR.post   :myqueue, "/submit",   "data"   # returns "Submitted data"
RR.put    :myqueue, "/update",   "data"   # returns "Updated data"
RR.delete :myqueue, "/resource"           # returns "Deleted resource"

You can find more examples on GitHub, including:

Summary

The goals of the RackRabbit project are to make it…

easier to write RabbitMQ subscriber services
easier to communicate with RabbitMQ subscriber services
easier to load balance RabbitMQ subscriber services
easier to daemonize and manage RabbitMQ subscriber services

… and basically to extract and generalize all of the boilerplate code needed when implementing an SOA over AMQP.

You can find the full documentation, code, tests, and examples on Github:

If you are currently using RabbitMQ to build an SOA or distributed system and think the RackRabbit project might be useful please try it out and let me know how it fits. I’d love to see it used in the real world beyond my own projects and find out if it has any value. You can always reach me at jake@codeincomplete.com

Enjoy!

UPDATE: Yes, since starting work on RackRabbit, I am aware of a very similar project called Jackalope that has some of the same goals and ideas. I plan to reach out to the project maintainers shortly.