Steven Moseley

I've discussed in a previous blog post about the Twitter API. Specifically in that post I talked about making a specific call, and then getting a response back in a traditional request/response manner. However, Twitter also has a streaming service that you can connect to that will push to messages real time in a persistent, open HTTP connection.

In general, this type of service can be very powerful, because it makes it very easy to stream information to clients with out the client having request the updates. In the past, streaming data to a browser was not been easy. In actuality, websites were really just making the browser appear to be streaming data but they were really just polling for data with multiple requests. However, today's browsers now support the WebSockets protocol which makes this type of streaming rather easy to implement. It essentially gives the browser the ability to open a connection to a host service and retrieve data through the open connection. Here is what is said about this functionality on Wikipedia.

"The WebSocket protocol makes possible more interaction between a browser and a web site, facilitating live content and the creation of real-time games. This is made possible by providing a standardized way for the server to send content to the browser without being solicited by the client, and allowing for messages to be passed back and forth while keeping the connection open. In this way a two-way (bi-direction) ongoing conversation can "take place between a browser and the server."

Enter Node.js

Since this type of service is asynchronous, its the type of functionality that really fits into the Node.js sweet spot, because JavaScript is by its very nature asynchronous.

To set this Node.js application up, I am going to need a few packages. Like Ruby on Rails' Gems package manager and Microsoft.Net's Nuget package manager; Node.js has a package manager called NPM. I'm not looking to reinvent the wheel here. I just want an easy way to get access to the Twitter stream. So to accomplish, this I am using the package nTwitter. There are tons of Twitter packages for Node.js, but I found this one the easiest to get up and running.

I am also going to set up the Express NPM package which gives Node.js the ability to have a more MVC approach to building a website.

So the first thing to do is to reference all the needed packages and also reference my custom nTwitter module.

var express = require('express')
  , routes = require('./routes')
  , socketIo = require('socket.io')
  , twitter = require('ntwitter')
  , util = require('util')
  , twitterModule = require('./modules/twitterModule');

My twitter module reference looks like this. Here I am initializing my nTwitter reference and passing all the needed Twitter credentials to make the streaming call. As aside, you need to go the Twitter Development Site and set up an application to get all your needed credentials.

var twitter = require('ntwitter'),
	util = require('util');
 
// Twitter Setup
module.exports.twit = new twitter({
  consumer_key: 'consumer key',
  consumer_secret: 'consumer secret',
  access_token_key: 'access token',
  access_token_secret: 'access token secret'
});

Enter Socket.IO

In my example, I am going to stream tweets that are in my general location. To do this, the "statuses/filter" streaming API needs to be referenced. This is the streaming API provided by Twitter.

To stream the API from my server to the client I am going to use Socket.IO. Socket.IO was built mainly for Node.js, although it can be used by other languages, to be the broker that sits on both the server and the client and handles all the heavy lifting of passing the data back in forth. The nice thing is, Socket.IO has the ability to check the browser for compatibility, so if the current client browser does not support WebSockets, then Socket.IO will revert to something like polling instead.

Thus the only server code that is needed to stream the tweets to the client is this:

io.sockets.on('connection', function(socket) {
  var twit = twitterModule.twit;
  twit.stream('statuses/filter', {'locations':'-80.10,26.10,-80.05,26.15'},
    function(stream) {
      stream.on('data',function(data){
        socket.emit('twitter',data);
      });
    });
});

Socket.IO has an "on" function, where a callback function is passed in. Inside this callback function the Twitter API stream is called, and as the stream "emits" new tweets, those entries get sent to the client. The locations parameters specifies the general longitude and latitude I want the tweets to originate from.

Socket.IO and JSRender on the Client

On the client there is the corresponding Socket.IO code that is configured to listen to the URL that was set up on the server for emitting the tweets. When each tweet is emitted to the client, an event is fired to render a new item to the list.

To render the tweets on the browser, I am using jsRender. If you have ever used jQuery Templates, jsRender is very similar and is probably going to end up replacing jQuery Templates going forward; although, at the time of this writing jsRender is still in Beta.

(function($) {
	$(document).ready(function() {
		var $container = $('ul.tweets'),
			socket = io.connect('http://localhost:3000'),
			template = $('#tweetTemplate');
			
 
	    socket.on('twitter', function(data) {
	        $container.append(template.render(data));
	    });
	});
})(jQuery);

For my Node.js views, I am using Jade. Very similar to HAML in Ruby on Rails, it is very terse and clean way creating markup. No angle brackets to deal with which is nice. Jade works by determining the indentation of the each line to base how things are nested. By the way, you can get this syntactical goodness on ASP.Net MVC as well by using The Spark View Engine.

This view also contains the jsRender template that is appended for each tweet.

h1= title
p #{title}
#local-tweet-container
	ul.tweets
 
script(id="tweetTemplate", type="text/x-jsrender")
	{{:#index+1}}: <li><img src='{{:user.profile_image_url}}' /> {{:text}}</li>

Running the App

When start the application by running node app.js and then browse to http://localhost:3000, the Socket.IO code makes the following request.

HTTP/1.1 101 Switching Protocols

Upgrade: websocket

Connection: Upgrade

Sec-WebSocket-Accept: vCWC+o5mWP5mnD2enVI3/PJOhMk=

 

The 101 status means that the requestor has asked the host to change its protocal and in this case the change was to WebSockets.

 

If I look at my browser I see the tweets getting added realtime.

It's kind of hard to demonstrate what is happenning in screenshot but the tweets are being added to the page and the actual page is not doing anything. Looking on Fiddler, no post or get requests are being made from my localhost domain.

 

Pretty cool stuff!

 

References

• Node.js
• Socket.IO
• Twitter Development Site
• jsRender

 

This is the fifth post in a series of posts about how I went about building my blogging application.

1. Building a Blog Redux - Why Torture Myself (Part 1)
2. Building a Blog Redux - The tools for the trade (Part 2)
3. Building a Blog Redux - Entity Framework Code First (Part 3)
4. Building a Blog Redux - Web fonts with @font-face and CSS3 (Part 4)

So I put a right rail column on my blog site thinking I probably would put some content over there to engage you users. I had my tag cloud, and eventually I am going to put a search box there as a future feature, but I wasn't really excited about what else to put there. I thought about a dated archive list but I have a dedicated page for that and besides, who really ever clicks on those links.

I could put some ads, but admittedly, I don't have enough traffic to consider that, and even if I did, I still probably would not do that right away. Although, I did put placeholder on the right side in case I changed my mind.

I could put my twitter feed, but I don't know; I don't think that is very interesting to user reading my blog, plus it's sort of been done by everybody already.

Goodreads

Enter Goodreads. If you read books and are social, Goodreads is the perfect site for you. You can keep track of all the books you read, categorizing them by  "Currently Reading", "Read", and "To Read". You can also categorize books by genres or whatever classification you want. The great thing about categorizing the books is Goodreads will give you recommendations based on how you have your books categorized.

Goodreads is social site, so just like Twitter, Facebook, etc. You make friends and then you can see what books they are reading and they have rated and reviewed.

And just like most other social sites, Goodreads has a developer section with an API and other tools for you to use. You just go to their website, request an application key, agree to their terms of use and then you are pretty much good to go.

The API

The API call I am using is the one called "user.show". It pretty much contains the basic information about, me, the user, such as what my bookshelves are, and (what I am most interested in) my book status updates.

The particular node I want is the action_text. Every time I update my status with a book I read or started reading, I get a new one of these nodes. The node value is HTML encoded so that task is already done by Goodreads.

<action_text>
  <![CDATA[
  gave 4 stars to: <a href="http://www.goodreads.com/book/show/12371896-the-node-beginner-book">
  The Node Beginner Book (ebook)</a> by <a href="http://www.goodreads.com/author/show/5132009.Manuel_Kiessling">
  Manuel Kiessling
  </a>
]]>
</action_text>

To get the XML for this particular message, I just need to make a GET reques to their server. Unlike Twitter, I didn't have to add a whole bunch of entries in the request header (I am going to write a post about how to do that later), I only needed the Goodreads API URL which looks like this.

http://www.goodreads.com/user/show/{user_id_number}.xml?key={your_key}

The call is a basic HttpWebRequest and HttpWebResponse.

public class HttpRequestHelper : IHttpRequestHelper
    {
        public string GetResponse(HttpWebRequest request)
        {
            ServicePointManager.Expect100Continue = false;
 
            if (request != null)
                using (var response = request.GetResponse() as HttpWebResponse)
                {
 
                    try
                    {
                        if (response != null && response.StatusCode != HttpStatusCode.OK)
                        {
                            throw new ApplicationException(
                                string.Format("The request did not compplete successfully and returned status code:{0}",
                                              response.StatusCode));
                        }
                        if (response != null)
                            using (var reader = new StreamReader(response.GetResponseStream()))
                            {
                                return reader.ReadToEnd();
                            }
                    }
                    catch (WebException exception)
                    {
                        return exception.Message;
                    }
                }
            return "The request is null";
        }
 
 
        public HttpWebRequest GetRequest(string fullUrl, string authorizationHeaderParams, string method)
        {
            var hwr = (HttpWebRequest)WebRequest.Create(fullUrl);
            if (! string.IsNullOrEmpty(authorizationHeaderParams)) hwr.Headers.Add("Authorization", authorizationHeaderParams);
            hwr.Method = method;
            hwr.Timeout = 3 * 60 * 1000;
            return hwr;
        }
    }

I have a function to build the HttpWebRequest and then I pass that request to function which makes the request and gets the HttpWebResponse.

Mapping the XML Response

I could have serialized the XML to an object using the XmlSerializer, but since I was only needing a few of the fields from a rather large XML response, I decided to just map the XML to a CLR object manually using LINQ to XML.

var view = new GoodReadsUserShowViewModel {Updates = new List<GoodReadsUpdateViewModel>()};
XDocument doc = XDocument.Parse(xml);

Creating the MVC Partial View

Once I have my object built, I then pass it back to the MVC controller which will then intern pass it back to a partial view.

        [OutputCache(Duration = 60000, VaryByParam = "*")]
        public ActionResult Index(string id)
        {
            try
            {
                GoodReadsUserShowViewModel result = GetGoodReadsUserShowViewModel(id);
                return PartialView("_Goodreads", result);
            }
            catch(Exception ex)
            {
                ErrorSignal.FromCurrentContext().Raise(ex);
                return PartialView("_Goodreads", new GoodReadsUserShowViewModel {ErrorMessage = ex.Message});
            }
        }

Things to note up until this point. Since I know that this feed is not going to change much (I mean I can only read books so fast), I am caching the response for a fairly long time. In this case, performance is more important than timeliness because I don't care that you see the latest status the second after I update it.

Also, I could have passed back a JsonResult, but in this case, Backbone.js actually recommends that you have your data already bootstrapped into the request. Therefore, on my Goodreads partial view, I am taking the view model and passing it to a helper class that will serialize my model JSON and put it in a JavaScript variable.

<script src="@Url.Content("~/js/goodreads.js")" type="text/javascript"> </script>
<script type="text/javascript">
    var grApp = new GoodreadsApp(@Model.ToJson());
    grApp.start();
</script>

The helper class ToJson looks like this.

using System.Web.Mvc;
using Newtonsoft.Json;
 
namespace AviBlog.Core.Helpers
{
    public static class JavaScriptSerializerHelper
    {
 
        public static MvcHtmlString ToJson(this object model)
        {
            string json = JsonConvert.SerializeObject(model);
            return new MvcHtmlString(json);
        }
         
    }
}

So when the page is actually rendered the JavaScript model that is rendered will look something like this.

<script type="text/javascript">

    var grApp = new GoodreadsApp({
        "UserId":"3425042",
        "UserName":"avington",
        "Name":"Steve Moseley",
        . . . });

    grApp.start();

</script>

Backbone.js Implimentation

Here is what the Backbone.js website has to say about what backbone.js is.

"Backbone.js gives structure to web applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing API over a RESTful JSON interface."

It essentially provides a clean way to bind your front-end code to different events to provide a rich user interaction. In this case, I am using it in a very simple fashion and just binding the initial response of the Goodreads data to template and binding it using jQuery Templates. Later on I am probably going to switch out jQuery Templates for jsRender, but for now jQuery Templates it is.

Derick Baily who probably has the best blog articles on Backbone, put a response in StackOverflow as what the best way to render data on a page on the initial load and as you can see from his response and my code, I pretty much did the same thing. Here is my backjone.js code.

GoodreadsApp = (function (Backbone, _, $) {
    var GoodreadsModel = Backbone.Model.extend({});
 
    var GoodreadsCollection = Backbone.Collection.extend({
        model: GoodreadsModel
    });
 
    var GoodreadsView = Backbone.View.extend({
        el: $('.goodreads-widget'),
 
        initialize: function () {
            this.collection.bind("reset", this.render, this);
        },
 
        render: function () {
            var data = this.collection.models[0].attributes;
            if (data) {
                var $template = $('#goodreads-template');
                var goodreadsHtml = $template.tmpl(data);
                $(this.el).html(goodreadsHtml);
            }
        }
    });
 
    var goodreadsApp = function (initialModels) {
        this.start = function () {
            this.models = new GoodreadsCollection();
            this.myView = new GoodreadsView({ collection: this.models });
            this.models.reset(initialModels);
        };
    };
 
    return goodreadsApp;
})(Backbone, _, jQuery);

In this example taken from StackOverflow, when the start function is called, an empty model is initialized. The reset backbone even is called which clear out the collection of models and fills the collection with a new set (in this case taken from the serialized JSON data). The render event is hooked up to the reset function so that when it is called the template is rendered.

The jQuery template looks like this.

<script id="goodreads-template" type="text/x-jquery-tmpl">
    <h4>Goodreads Status</h4>
    <ul>
        {{each Updates}}
        <li class="clear">
            <img class="goodreads-book-img" src="${BookImageUrl}"/>
            <div><a href="${UserLink}">${Name}</a> {{html ActionText}}</div>
        </li>
        {{/each}}
        <li class="last-goodreads-item">Follow me on <a href="${UserLink}">Goodreads</a></li>
    </ul>
    
 
</script>

Conclusion

I would have put a picture of the result here, but if you take a look over here on the right. Yeah the right hand panel you can see it there for yourself. I think its a pretty cool section for only a couple hours of work. As always, you can see all the code for the application at Github.

Resources

• Goodreads API
• Backbone.js
• Backbone.js Screencast
• Derick Baily's StackOverflow response on how to bootstrap initial data into Backbone.js

Introduction

I have been playing around with Node.js for a bit. You know--the hot new kid on the programming block that everyone is talking about. I'm getting old and maybe this is just a mid-life crisis, but I like learning the cool, new stuff. JavaScript on the server! Who knew?

Why Node

Node is popular for a couple reasons. For one thing, it is JavaScript, so pretty much every web developer already knows it. The engine that executes the scripts server side is called V8. This is the very same engine that executes JavaScript scripts in your Chrome browser and it is fast. The thing that makes Node.js standout from other languages is that it is by it's JavaScript nature, non-IO blocking. There is some good information about it on Orielly, but essentially, what this means is Node.js handles it's I/O asynchronously and this makes it quite preforment.   Node.js does not tie up threads waiting on long running call-outs, instead it uses what is called an event loop to handle all of the call-outs at the same time. That's not to say that Node.js doesn't have it's issues, it does. It's still a bit immature (especially on a Windows machine) and you have to be careful with the whole asynchonous thing as it can cause some unusual results. Still, I like it. I am having fun learning it.

​Setting It Up:

First off, a quick shoutout to the guys at http://nodecasts.org/ for getting me started. The screencast was very helful. Because the age of the screencast, the code was a little bit depricated (Express was the culprit), but it didn't take much to find the new code syntax and correct the problems. I am going a little bit further in my example and making the message call via AJAX instead of in the command line via CURL.

To get Node.js up on your machine, go to http://nodejs.org and follow their installation instructions.

Once node is up and running, got http://npmjs.org/ and install the Node.js Package Manager. If your a .Net guy it is the Nuget equivalent, or the Ruby's Gem equivalent.

On my Windows machine, I've setup a directory call Node (i.e. c:\Node\ ). I then created a project folder ( C:\node\faye_sample ).

From the command line, navigate to to your project folder and run the following commands.

• npm install express
• npm install faye

These commands will install the Express and Faye packages on your machine that you can then reference in your code.

Express is basically a Node framework that gives you MVC styled architecture inspired by Sinatra in the Ruby world. For this example, I am only going to create a one file application and use Express for some shortcuts. There are some really good Express screencasts at NodeTuts, so if you want to go further with Node, I recomment that you give those a look at.

Faye is subscriber/publisher framework that allows any HTTP client to talk to a server real time as well many clients talk to each other real time via the server.

​The Server Code

Okay, lets take a look at the server side code. First thing I need to do is get a reference to Express and Faye.

var express = require('express'),
    faye = require('faye');

The next thing I need to do is initialize Faye on the server.

var bayeux = new faye.NodeAdapter({
    mount: '/faye',
    timeout: 45
});

In case you are wondering about that funny variable name. Faye uses the Bayeux protocol as its mechanism to transport the messages back and forth.

Next step, initialize the Node server using the Express wrappers.

var app = express.createServer();
app.configure(function() {
    app.use(express.bodyParser());
    app.use(express.static(__dirname + '/public'));
});

The express.bodyParser will parse the request coming in and place the data coming in into the req.body object. As you will later see, it will contain my incoming message that I can then get to rather easily.

The variable __dirname is the current directory my application is running in. the function express.static is basically telling Node that all my static content is located in the static folder inside the current application folder.

To capture the posts from the different browsers, I need to set up a route handler that will capture a post request and then broadcast that request to all of the client listeners.

app.post('/message', function(req, res) {
    bayeux.getClient().publish('/channel', { text: req.body.message });
    console.log('broadcast message:' + req.body.message);
    res.send(200);
});

The first line of code is capturing the post request from "/message" and passing in the request and response object on a callback function. The callback function then gets the message from the request and publishes it to the clients. I am also logging the message so I know that the server got the message. Because I called bodyParser when I configured my server, I now have access to the req.body.message which contains my message from the client.

The last thing I need to do is start up my node server. In this case, I am listening on port 8123.

bayeux.attach(app);
app.listen(8123);

The Client

Okay, for simplicity sake here, I am going to create two HTML files with exact same code in it and then hard code Client #1 and Client #2 in the respective files. So lets look at the client code.

In my HTML markup, I have the following elements.

                <h1>Chat Client #1</h1>
		<div id="messages"></div>
		<textarea rows="2" cols="35" id="chat"></textarea>
		<input type='button' value='Chat' id='fire' />

I need to grab a script reference to the latest jQuery, and also the Faye script on the client side.

<script src='/faye/browser/faye-browser.js'></script>

With the Faye reference, I can then setup a listener that will update my HTML when ever some sends out a message.

                        var client = new Faye.Client('/faye',{
				timeout: 20
			});
			
			client.subscribe('/channel', function(message) {
				$('#messages').append('<p>' + message.text + '</p>');
			});

Next: I am going to use jQuery to post a message when ever a user clicks on a chat button.

    		    var $chat = $('#chat');
    		    $('#fire').on('click',null, function() {
    		        var url = 'http://localhost:8123/message';
				
    		        var message = {message: 'Client 1: ' + $chat.val()};
    		        var dataType = 'json';
    		        $.ajax({
    		            type: 'POST',
    		            url: url,
    		            data: message,
    		            dataType: dataType,
    		        });
    		        $chat.val('');
    		    });
    		    

That's it!

I can now start up my server from the command line.

• ​node server.js

Next, I open two browsers and start chatting.

Now, if I look at my command line server log I should see the messages because I put a line in the code to log them.

​Conclusion

I set up a GitHub repository with this code in it that you can take a look at. As I explore Node some more, I'll probably post some more posts on Node.js so stay tuned.

References

• Node JS
• NPM Node Packager
• Express
• Faye
• Faye Screencast via NodeCast
• Orielly article on the success of Node

Working with a map on a website, or any other application platform for that matter, may seem like a daunting task, but after working with it for a little bit, it is actually not all that difficult. The documentation for the API is a bit sparse, in my opinion, so I thought I would document what I did to get simple map functionality up and running so that others who might wanting to add a map to a webpage might find this article useful.

I am using the Bing API for this demonstration but you could use Google or Yahoo and not have to do too much different.

Get Your Key

First things first. You will need to go to the Map Account Center and create an account and obtain an application key. You can get development keys and production keys or use one key for both environments. The portal also has your usage stats, SDKs, a blog and other useful information.

The HTML

Once you have your key, your in business and can start configuring your web page to use the Bing Map API.

The only markup you need for your web page is a div tag.

 1: <div id="map_div" class="map">

 2: </div>

My style for this div looks like this.

 1: #search_main_conent .map 

 2: {

 3:     position: absolute; 

 4:     top: 20; left: 10; 

 5:     width: 400px; 

 6:     height: 400px; 

 7:     border:#555555 2px solid;

 8: }

The JavaScript

What I want to do for this particular map, as far as functionality, is load the map, and when the user clicks on the map and changes the view by moving it around or zooming in or out, is when the views finishes changing, fire an event that will populate two text boxes with the longitude and latitude.

Here is the JavaScript code to make that all work.

 1: $(document).ready(function () {

 2:     twitterMap.LoadMap();

 3: });

 4:  

 5: function twitterMap() { }

 6: twitterMap._map = null;

 7:  

 8: twitterMap.LoadMap = function () {

 9:     var $maps = $('#search_main_conent .map');

 10:     $maps.each(function (index) {

 11:         if (index === 0) {

 12:             twitterMap._map = new Microsoft.Maps.Map(

 13:                 this, {

 14:                     credentials: "Your Key",

 15:                     center: new Microsoft.Maps.Location(

 16:                         26.225948, 

 17:                         -80.188093),

 18:                     mapTypeId: Microsoft.Maps.MapTypeId.road,

 19:                     zoom: 7

 20:                 });

 21:  

 22:             Microsoft.Maps.Events.addHandler(

 23:                 twitterMap._map, 

 24:                 "viewchangeend", 

 25:                 twitterMap.onViewChangeEnd);

 26:         }

 27:     });

 28: };

 29:  

 30: twitterMap.onViewChangeEnd = function () {

 31:     var location = twitterMap._map.getCenter();

 32:     var longitude = location.longitude;

 33:     var latitude = location.latitude;

 34:     if (longitude) {

 35:         $('#search_main_conent #Longitude').val(longitude);

 36:     }

 37:     if (latitude) {

 38:         $('#search_main_conent #Latitude').val(latitude);

 39:     }

 40:     

 41:  

 42: };