I've discussed in a previous blog post about the Twitter API. Specifically in that post I talked about making a specific call, and then getting a response back in a traditional request/response manner. However, Twitter also has a streaming service that you can connect to that will push to messages real time in a persistent, open HTTP connection.
In general, this type of service can be very powerful, because it makes it very easy to stream information to clients with out the client having request the updates. In the past, streaming data to a browser was not been easy. In actuality, websites were really just making the browser appear to be streaming data but they were really just polling for data with multiple requests. However, today's browsers now support the WebSockets protocol which makes this type of streaming rather easy to implement. It essentially gives the browser the ability to open a connection to a host service and retrieve data through the open connection. Here is what is said about this functionality on Wikipedia.
"The WebSocket protocol makes possible more interaction between a browser and a web site, facilitating live content and the creation of real-time games. This is made possible by providing a standardized way for the server to send content to the browser without being solicited by the client, and allowing for messages to be passed back and forth while keeping the connection open. In this way a two-way (bi-direction) ongoing conversation can "take place between a browser and the server."
Enter Node.js
Since this type of service is asynchronous, its the type of functionality that really fits into the Node.js sweet spot, because JavaScript is by its very nature asynchronous.
To set this Node.js application up, I am going to need a few packages. Like Ruby on Rails' Gems package manager and Microsoft.Net's Nuget package manager; Node.js has a package manager called NPM. I'm not looking to reinvent the wheel here. I just want an easy way to get access to the Twitter stream. So to accomplish, this I am using the package nTwitter. There are tons of Twitter packages for Node.js, but I found this one the easiest to get up and running.
I am also going to set up the Express NPM package which gives Node.js the ability to have a more MVC approach to building a website.
So the first thing to do is to reference all the needed packages and also reference my custom nTwitter module.
var express = require('express') , routes = require('./routes') , socketIo = require('socket.io') , twitter = require('ntwitter') , util = require('util') , twitterModule = require('./modules/twitterModule');
My twitter module reference looks like this. Here I am initializing my nTwitter reference and passing all the needed Twitter credentials to make the streaming call. As aside, you need to go the Twitter Development Site and set up an application to get all your needed credentials.
var twitter = require('ntwitter'), util = require('util'); // Twitter Setup module.exports.twit = new twitter({ consumer_key: 'consumer key', consumer_secret: 'consumer secret', access_token_key: 'access token', access_token_secret: 'access token secret' });
Enter Socket.IO
In my example, I am going to stream tweets that are in my general location. To do this, the "statuses/filter" streaming API needs to be referenced. This is the streaming API provided by Twitter.
To stream the API from my server to the client I am going to use Socket.IO. Socket.IO was built mainly for Node.js, although it can be used by other languages, to be the broker that sits on both the server and the client and handles all the heavy lifting of passing the data back in forth. The nice thing is, Socket.IO has the ability to check the browser for compatibility, so if the current client browser does not support WebSockets, then Socket.IO will revert to something like polling instead.
Thus the only server code that is needed to stream the tweets to the client is this:
io.sockets.on('connection', function(socket) { var twit = twitterModule.twit; twit.stream('statuses/filter', {'locations':'-80.10,26.10,-80.05,26.15'}, function(stream) { stream.on('data',function(data){ socket.emit('twitter',data); }); }); });
Socket.IO has an "on" function, where a callback function is passed in. Inside this callback function the Twitter API stream is called, and as the stream "emits" new tweets, those entries get sent to the client. The locations parameters specifies the general longitude and latitude I want the tweets to originate from.
Socket.IO and JSRender on the Client
On the client there is the corresponding Socket.IO code that is configured to listen to the URL that was set up on the server for emitting the tweets. When each tweet is emitted to the client, an event is fired to render a new item to the list.
To render the tweets on the browser, I am using jsRender. If you have ever used jQuery Templates, jsRender is very similar and is probably going to end up replacing jQuery Templates going forward; although, at the time of this writing jsRender is still in Beta.
(function($) { $(document).ready(function() { var $container = $('ul.tweets'), socket = io.connect('http://localhost:3000'), template = $('#tweetTemplate'); socket.on('twitter', function(data) { $container.append(template.render(data)); }); }); })(jQuery);
For my Node.js views, I am using Jade. Very similar to HAML in Ruby on Rails, it is very terse and clean way creating markup. No angle brackets to deal with which is nice. Jade works by determining the indentation of the each line to base how things are nested. By the way, you can get this syntactical goodness on ASP.Net MVC as well by using The Spark View Engine.
This view also contains the jsRender template that is appended for each tweet.
h1= title
p #{title}
#local-tweet-container
ul.tweets
script(id="tweetTemplate", type="text/x-jsrender")
{{:#index+1}}: <li><img src='{{:user.profile_image_url}}' /> {{:text}}</li>
Running the App
When start the application by running node app.js and then browse to http://localhost:3000, the Socket.IO code makes the following request.
References
