Using Faye to implement a publish-subscribe design pattern

Screen shot 2013-02-20 at 11.09.52 AM

My last blog post, Example of a Real-Time Web App, was an overview of the architecture of WhitePages’s Mailer.  We used Ruby on Rails, Spine.js, Faye, and Sidekiq to build an asynchronous web app.  Rails is a server-side MVC framework, Spine.js is a client-side MVC framework, Sidekiq processes jobs, and Faye ties those three pieces together with HTTP server push.  This blog dives deeper into Faye, which can be the basis for implementing a publish-subscribe design pattern.

What is Faye?

In their own words:

“Faye is a publish-subscribe messaging system based on the Bayeux protocol. It provides message servers for Node.js and Ruby, and clients for use on the server and in all major web browsers.”

In my words:

“Faye is an abstraction layer that simplifies HTTP server push.”

Faye is one of many options for abstracting the details of server push.  Popular alternatives include hosted solutions like Pusher.com and PubNub, and self-hosted solutions like Socket.io and Firehose.  Choosing the best fit for your project is an exercise left to the reader.

How does it work?

Faye consists of two pieces: a subscriber and a publisher.  Both the subscribing and publishing components are Faye clients.  For basic server-to-browser pushing, the browser client (in JavaScript) subscribes to the server client (in Ruby) – the server then broadcasts to a channel that the client is listening to.

Get Started with Faye on Rails

To get started, run a Faye server mounted at ‘/faye’.  Getting your server up is well-documented on Faye’s site.  With a Faye server up and running, include the Faye javascript code in your project and create a client listening to the default port (9292) with the following code:

$(function() {
var faye_subscriber = new Faye.Client(‘http://localhost:9292/faye’);
});

To create a Faye client on the Ruby server, the syntax is similar:

faye_publisher = Faye::Client.new(‘http://localhost:9292/faye’)

Now the client can subscribe, here we’ll define a channel called “updates” and alert whatever data is broadcast to the channel:

faye_subscriber.subscribe(‘/updates’, function (data) {
alert(data);
});

And the server can publish data – we’ll publish the message “hello push!”.

faye_publisher.publish(‘/updates’, “hello push!”)

That’s all there is to it.  After publishing “hello push!”, the client-side subscriber hears it and alerts the data.

How’d we use it?

When a user imports their friends from Facebook, we process each friend in a background job and publish the results as friends are processed.  The published message includes everything the UI needs to display it, so we can update users’ lists without waiting for the whole job to finish.  As soon as a friend is processed, the user gets it.

Conclusion

Pub/sub, in symphony with other real-time web technologies, allows you to close the performance gap between native and web apps, and Faye makes it dead simple to get started.

Example of a Real-Time, Asynchronous Web Application

steps1 2

There’s been a lot of buzz about the “real-time” web among developers lately. As browsers and frameworks mature, it’s becoming increasingly advantageous to build fully asynchronous websites. But what is the real-time web? The answer may vary depending on who you ask, but the one common thread is that real-time websites enable the client (web browsers) to react instantly to changes on the server. Real-time web apps allow performance that rivals native apps, avoiding most of the delay intrinsic to traditional client-server systems (in which browsers only get new content by requesting it from servers). This post is intended to give an overview of the architectural decisions for an asynchronous web application — our newest product, WhitePages Mailer.

Screen shot 2012 12 12 at 10.12.00 AM

 

The core ingredients:

  • Ruby on Rails — our server-side framework.
  • Spine.js — our client-side framework.
  • Sidekiq — for asynchronous processing.
  • Faye — pub/sub messaging, connecting client to server via websockets.

Ruby on Rails

Ruby on Rails is a server-side MVC framework that provides database, templating, session, and asset management.  We primarily use Ruby on Rails as a JSON API, and for serving assets.  Rails has an “Asset Pipeline” that’s responsible for compressing and serving JavaScript.  We have a lot of JavaScript, so much that we have another MVC framework to offer some abstraction: Spine.js.

Spine.js

Spine.js is a client-side MVC framework designed to provide abstraction for asynchronous, responsive user interfaces.  To ensure asynchronicity and responsiveness, all view rendering is done client-side, which means we don’t really use the Ruby on Rails template management: we use Spine’s client-side JavaScript templates.  (It’s actually CoffeeScript but let’s ignore implementation details for now).  Spine gets some of its data from AJAX calls to the JSON API on the Rails server, and some of its data from background processes that run on Sidekiq servers.

steps1 2

Sidekiq

Sidekiq is a server-side message processor.  Ruby on Rails (like Django) assumes synchronous programming.  Without Sidekiq to process background jobs asynchronously, any long-running processes (such as importing all of a user’s friends from Facebook) would block Rails. With Sidekiq, when any client requests a long-running job, it’s handed off to Sidekiq (via Redis), which processes the job and then then publishes the result to the client via a websocket.

steps3 4

animation

Faye

Faye is a publish-subscribe messaging system responsible for transporting the messages between servers and clients.  There’s a server-side, Ruby component of Faye that publishes messages (by broadcasting to an established channel), and a client-side, JavaScript component of Faye that subscribes to that channel and receives the messages.  By using websockets we’re able to push data from the server to the client whenever it’s ready, instead of waiting for the client to request it (e.g. polling).

With this architecture, page transitions can be instantaneous, and the UI can react in real-time to server activity (such as updating the progress bars in import and export jobs, and rendering each contact as they’re imported).  Check it out at WhitePages Mailer and let us know what you think.

 

 

Find Jared on Google +

Explore the World of Names

Names_screenshot

For the past six months, I have been working with a small team at WhitePages to create a new web app designed to provide users with everything they could want to know about a Name.  This site is fast, easy to navigate, and surfaces public data from both the Social Security Administration & Whitepages.  Whether you’re an expecting mother looking for baby name inspiration, want to know how to pronounce Siobhan in preparation for a business meeting or are just curious about the age distribution of people named “Mildred”, our new Names app has you covered.

When you land on Names, there are a few options to explore the site.  The home page allows you to see the most popular names and a hand-picked “Name of the Day”.  You can also use the no-fuss search feature at the top of every page to look for a first name, last name or both.

Once you arrive on a details page for a name you are presented with data on pronunciation, meaning, age distribution, gender distribution, geographic distribution, popularity, contact information, and social network activity — it’s a lot of data, but we’ve designed it to be as compact and easily readable as possible.

If you are curious about names but aren’t sure where to start, click “Discover” from the top of any page and check out our page for Names exploration.  A personal favorite is the Search By Time module - a draggable slider that lets you view the most popular and least popular names of the past century.  Did you know in the 90′s there are 119 people named Andromeda and they are all girls?

We’ve also made it easy to share with friends whenever you find an unusual name or an interesting fact — through Google+, Facebook, or Twitter.  If you Tweet about WP Names, you can be listed on our Twitter leader board, gaining publicity for your Twitter account.  I’m currently dominating the Jared pages with 23 followers.

We’ve laid out some of the features in this infographic, but the best way to get to know the new Names site is by playing around with it yourself.  Enjoy!