Please note: this blog has been migrated to a new location at https://jakesgordon.com. All new writing will be published over there, existing content has been left here for reference, but will no longer be updated (as of Nov 2023)

SOA - Introducing RackRabbit

Thu, Sep 4, 2014

In this series we have discussed:

We talked about why we might implement a Service Oriented Architecture in order to scale to meet…

We also talked a little bit of theory on what an SOA looks like, how to define the boundaries for our services, the practical implications of building a distributed system, and the 3 main communication patterns of a distributed system:

When implementing SOA using HTTP, we discovered that we have access to a robust set of technical options such as Rack, Unicorn, HTTParty, Faraday, Redis, Resque, or Beanstalkd, but no single technology can provide for all 3 communication patterns.

When implementing SOA using AMQP, we discovered that we can implement all 3 patterns using a single technology, but the implementation details are a little more tricky, programming directly to the AMQP interface, and the infrastructure and community support for hosting production-ready consumers are lacking (e.g. there is no easy-to-program-to Rack interface, or Unicorn style server for easily load balancing RabbitMQ consumer processes).

Just like everything in our profession, there are trade-offs.

Introducing RackRabbit

I really like the idea of having a single technology for our distributed communication, so in order to make the concept of SOA using AMQP easier to manage I have created the RackRabbit project to provide:

You can find the full documentation, code, tests, and examples on Github:

What Unicorn does for HTTP services, RackRabbit can do for hosting AMQP services, and more:

HTTP AMQP
Make a synchronous request/response Unicorn rabbitMQ + RackRabbit
Asynchronous worker queue Redis + Resque rabbitMQ + RackRabbit
Asynchronous publish/subscribe Redis rabbitMQ + RackRabbit

Example of Synchronous Request/Response (using Sinatra)

Consider this simple sinatra service in config.ru:

require 'sinatra/base'

class Service < Sinatra::Base

  get "/hello" do
    "Hello World"
  end

  post "/submit" do
    "Submitted #{request.body.read}"
  end

  put "/update" do
    "Updated #{request.body.read}"
  end

  delete "/resource" do
    "Deleted resource"
  end

end

run Service

Host and load balance this service using rack-rabbit:

$ rack-rabbit --queue myqueue --workers 4 config.ru

Connect to the worker from the command line using the rr command:

$ rr request -q myqueue GET /hello
Hello World

$ rr request -q myqueue POST /submit "data"
Submitted data

$ rr request -q myqueue PUT /update "data"
Updated data

$ rr request -q myqueue DELETE /resource
Deleted resource

Connect to the worker from our application using the RR class:

require 'rack-rabbit/client'

RR.get    :myqueue, "/hello"              # returns "Hello World"
RR.post   :myqueue, "/submit",   "data"   # returns "Submitted data"
RR.put    :myqueue, "/update",   "data"   # returns "Updated data"
RR.delete :myqueue, "/resource"           # returns "Deleted resource"

You can find more examples on GitHub, including:

Summary

The goals of the RackRabbit project are to make it…

… and basically to extract and generalize all of the boilerplate code needed when implementing an SOA over AMQP.

You can find the full documentation, code, tests, and examples on Github:

If you are currently using RabbitMQ to build an SOA or distributed system and think the RackRabbit project might be useful please try it out and let me know how it fits. I’d love to see it used in the real world beyond my own projects and find out if it has any value. You can always reach me at jake@codeincomplete.com

Enjoy!

UPDATE: Yes, since starting work on RackRabbit, I am aware of a very similar project called Jackalope that has some of the same goals and ideas. I plan to reach out to the project maintainers shortly.