RESTful architecture simply explained

I can't believe I missed this gem that Roy T. Fielding put out in October! If you're not up for reading the monster dissertation at least read this post and the comments.

A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC's functional coupling].

Spot on Roy! "Form follows function"

I understand how hard it is to implement an architecture and want to get it right according to the specification. Every time I look at his dissertation I come across something I missed last time I visited it. His post is a quick and dirty checklist that I can now referr to.

Users of the new Fellowship One will be seeing the above as well as other principles lined out in the post implemented in the new RESTful API (the API demoed in the video was a prototype. URIs, hypertext, and the like in the finished version conform to RESTful architectural principles).

Making ASP .NET MVC URIs RESTful

Out of the box the ASP .NET MVC implementation is nice, easy, and elegant. From what I understand they used a few patterns from, the RoR world. One thing that is not part of the default implementation is RESTful routing.

Thanks to the guys who are infinitely smarter than me that did MVCContrib, adding RESTful behavior (using SimplyRestfulRouteHandler) to a .NET MVC app is pretty simple.

1. Download the MVCContrib libraries (I downloaded the source because I like to break stuff)
2. Open up your ASP .NET MVC application and add a reference to the MVCContrib.dll
3. Open up the Global.asax.cs and add the following line inside the ResgisterRoutes method: MvcContrib.SimplyRestful.SimplyRestfulRouteHandler.BuildRoutes(routes);

This is what you get:

Action	URL	Http	Form
Show	[controller]/[id]	GET
Create	[controller]	POST
Update	[controller]/[id]	PUT
Update	[controller]/[id]	POST	PUT
Destroy	[controller]/[id]	DELETE
Destroy	[controller]/[id]	POST	DELETE
Index	[controller]	GET
New	[controller]/new	GET
Edit	[controller]/[id]/edit	GET
Delete	[controller]/[id]/delete	GET

So if we we're dealing with people, for instance, the URI and verb maps might look like:

HTTP verb: [GET] (for show)
URI: https://api.mydomain.com/v1/People/1234

HTTP verb: [PUT] (for update)
URI: https://api.mydomain.com/v1/People/1234

HTTP verb: [POST] (for create)
URI: https://api.mydomain.com/v1/People/

so on...

So what if you wanted to address hierarchy with the object using the same URI scheme? Let's say you wanted to get all of the addresses for a particular person and addresses were a child of person:

HTTP verb: [GET]
URI: https://api.mydomain.com/v1/People/1234/addresses

Thanks to the slick implementation of the RouteHandler all you have to do is add another call to BuildRoutes using an overload that accepts an "areaPrefix"

SimplyRestfulRouteHandler.BuildRoutes(routes, "People/{individualID}");

So now you can use:
https://api.mydomain.com/v1/People/1/addresses <-- collection
https://api.mydomain.com/v1/People/1/addresses/3 <-- single
https://api.mydomain.com/v1/addresses/3 <--single no context

RESTful thoughts - OAUTH and statelessness

I keep a copy of Roy Fielding's 162 page dissertation in my laptop bag, yeah I know - ich bin ein nerd, but hey it only adds another 2~3 lbs to my bag and I like rules and architecture - "form follows function" as Fielding puts it.

I am facing an architectural paradox right now where form seems to be tripping over function - client-stateless-server (CSS).

One component of the REST definition is that it must be stateless as defined by the principle of client-stateless-server - check out section 3.4.3 of his document.

Here's the rub: the OAUTH protocol defines patterns to pass tokens back and forth (also recommending that the tokens have an expiration set on them) as well as store nonce for signature and URI uniqueness. If these things are stored / retrieved / used by the Server application (i.e. the RESTful API), doesn't it seem that the client becomes more bound to the server; especially when the authentication / authorization is part of the payload. The separation of concerns is no longer afforded to the client - it removes the benefit of CSS; just as Fielding describes stateless as being paramount to visibility, reliability, and scalability.

Now you're going to ask about the key and secret for OAUTH. Those things are different - while they are stored on both the client and server they are evaluated per request within the context of that request. Whereas a token (Access or Request) gets evaluated and attached to a user or that user's hypermedia and the like. Feels a bit like "session" - something that can expire, holds info about or can be tied to a user.

I maybe splitting hairs but I feel that in order for an application to be stateless each request from the client to the server must contain all of the information to properly understand the request. The tokens have meaning on the server more than just a being token.

I absolutely love the RESTful architectural style and the OAUTH protocol. But can the two work together without breaking the rules?

Generally you'd want to abstract authentication from your application. OAUTH addresses both authentication (am I who I say I am) and authorization (can I do what I am trying to do) - so meaning is given to the token passed between the applications.

Honestly this incongruity between the two patterns will not stop me from using both together, I am not sure it's worth loosing the functionality of form - as fielding puts it:

Like most real-world systems, not all components of the deployed Web architecture obey every constraint present in its architectural design. REST has been used both as a means to define architectural improvements and to identify architectural mismatches. Mismatches occur when, due to ignorance or oversight, a software implementation is deployed that violates the architectural constraints. While mismatches cannot be avoided in general, it is possible to identify them before they become standardized.

Anyone have any thoughts on this?

RESTful thoughts - Why ASP .NET MVC?

Let me begin by building the frame of how we approached the idea for a new API before I go into the details of the technologies we chose. We needed an API that would address the following:

1. Secure
2. Follows common community standards / practices
   
3. Public interface
4. Extremely easy to consume
   
5. Supported formats (at least) XML, JSON --> optionally to be extended to XHTML, ATOM, etc...
6. Needs to scale and understand complex relationships
7. Needs to scale fast and with agility
   

The architecture / protocols we chose were - REST(delivery / interaction) + OAUTH(Authentication). I personally feel that this approach is instrumental to the success of your software when you're developing an externally facing API.

Technologies face sweeping changes , patterns / standards change either incrementally or not at all. Fielding did not create REST, he just documented the architectural principles (extremely well). Consider how much simpler it is to develop for a browser when it follows css standards (Firefox, Safari, Opera).

Why REST? The pattern addresses 6(#2-7) of the 7 needs above. Plus, adoption since Fielding's dissertation in 2000, has sky rocketed. Most Web Devs have at least heard of REST and I am pretty sure the majority of the community has knowingly / unknowingly used the pattern.

That guy is old! But he has one job and he does it well... (courtesy: Disney)

Patterns are tried and true - that's why they are patterns. Communities of people come together and agree that pattern x or specification y should be a standard. This blog is my opinion, though I would like to think it is "fact."

The real fact is that there are people out there in the tech pot that are far more intelligent than I am, that are writing specifications that my opinions try to enforce - accurately or inaccurately. Remember the I/O tower in Tron? It had 1 purpose -> communication and apparently it did it well, just look at how old that guy was.

That's how I want this API to turn out, not old and crusty, but constantly predictable. Let me and the patterns worry about the consistency of the API while you enjoy the serendipity of consuming it.

MCP: Ironic that the same acronym is used for a Microsoft Certification?(courtesy: Disney)

Why OAUTH? Look at the majority of web facing, public APIs available to us. Talk about a seriously cool time to be a developer - the days of making an RPC to some arcane server buried at some university are gone. Remember the MCP in Tron? Yes he gobbled up all of the resources and shut down systems, but that's not the point I am trying to make, so we'll ignore that. He acted as the gatekeeper - he was built to be a way to protect and secure stuff, to check and authorize credentials. OAUTH is a lot like that and it is a proven pattern with implementation libraries to boot!

Consumers of my API will not have to wait for me to produce documentation and sample code libraries for some custom authentication implementation I tossed together - they can just head over to oauth.net. OAUTH is an open protocol that has been tested, is proven, and is simple to implement - #1 answered, as well as supporting #2 and #4.

For an API, you don't want to have to introduce anything synonymous to the holly hack or your typical web app "browser test" --> that road will take you to an API that collects dust and not consumers. So, first thing first - standards over technology

Once we selected the standards we started targeting some web frameworks for considerations. Here are some of the technologies that we considered:

Ruby + Framework (Rails, Merb, etc...)
While Rails or Merb we're solid contenders. Rails specifically, I enjoy the opinionated approach, the scaffolding that comes right out of the box, and the DRY that it embraces. I built the 2nd version of the API using Rails but later bailed because implementation would be more difficult considering our existing domain and established environments. If we ever did want to come back around and use Rails or something like it we could do so without impacting the consumers - remember pattern / standards over technology.

The good:

• Opinionated
• Testability
• Elegant, simplistic
• RESTful patterns built in

The bad:

• No established local or prod environments to support the code,
• Existing domain is .net, existing domain / data model is not opinionated - so we would either modify Rails or our model which would exceed our time line to implementation.

ADO.NET Data Services / WCF + RESTful Toolkit
I'll be honest, when I saw Astoria come up on the Microsoft pipe I immediately thought SOAP web services. Was that the right thing to do? No. Did it skew my perception of ADO.NET data services? Yes. So far, regarding some patterns, Microsoft has taken x and added .NET to it. This is OK, in fact it's inevitable - you embrace patterns and are passionate about your technology some of your principles and opinions will become integrated into the pattern.

Sark: Guy got too big for his shoes and got his wig split(courtesy: Disney)

Unfortunately sometimes those opinions cross lines which make it difficult for developers to implement the technology. XML web services as Microsoft SOAP web services, REST as WCF / ADO .NET services.

Last Tron analogy, I promise. Let's talk about Sark? The guy was an overbearing heavy hitter. He wasn't like Tron. Tron played games, that's all he did and he did it extremely well.

End result: he beat the guy who tried to do everything.

The fact is that no piece of software or framework will be everything to everyone - so don't try to make it that. Why does Microsoft have 2 technologies that attempt to do the same thing 2 different ways?

So why not use WCF or ADO .NET WS for a RESTful API? I answered that question with a question: What is it that I am trying to develop?

I simply need a RESTful web application, bottom line.

WCF + RESTful Toolkit
The good:

• It does everything I need and lots more
• It can be web based
• Has a super-cool feature for documentation

The bad:

• It does lots more - too much in fact
• It is highly complex
• It was originally based on SOAP and WS-*
• URIs are difficult to control / manipulate
• HTTP modules are necessary to have extensionless URLs (I am definitely not a fan of .svc)
• According to Flanders' new book RESTful .NET WCF "...actually did have support for building services using REST, but the WCF programming model lacked explicit support for doing so."

WCF, to me, feels like communication between systems was a priority and communication via Hypermedia based systems "RESTfully" is kinda like an add on. I need more focus and less technology getting in my way. I wrote the 1st prototype of the API using WCF, but trashed it because of complexity and overhead.

ADO.NET Data Services

The good:

• Web based
• Microsoft apparently started from scratch on this one - not a Web Services carry over - according to Pablo Castro's Mix08 presentation
• Supports ATOM, json, XML content by default
• Some cool mechanisms for querying
• Wicked easy to extend and develop for / against

The bad:

• Architecture (Data provider and DataSets) - DataSets have historically been bloated. I cannot help but think that there was some sort of pattern carry over from the old Web Services
• I personally do not like the URIs - http://api/service.svc/Users(1)/Bookmarks
• The service as a whole feels "REST-like" and not RESTful
• Things like "$parm", "Expand", etc... tell me that they are trying to be everything to everyone - They cannot use normal request parms because they are tied to LINQ
• URIs are difficult to control / manipulate
• HTTP modules are necessary to have extensionless URLs (I am definitely not a fan of .svc)

I used ADO .NET web services back in the Astoria days, I liked it, I thought it had some really cool features. The technology, however, does not meet my expectations or my RESTful needs

ASP .NET MVC (Beta)

The good:

• Simply, a web application
• I love the MVC pattern
• Easy to develop for
• Easy to consume
• Routes can easily be configured to impose "pure" REST architecture principles - no "REST-like", it is all HTTP - no need for some services framework to lay on top of it and "interpret" requests, low overhead
• No complexities of a service framework
• You can extent request and response formats easily
  
• Most of all - implementing RESTful architecture is simple because ASP .NET MVC at its root is a web based, hypermedia application framework

The bad:

• You have to enforce RESTful principles (unlike Rails) - MVCContrib helps,
• No default documentation or documenting feature (WCF RESTful toolkit has a very nice documentation feature built in)

Tron: Yeah, all I do is play games, but I do it well!(courtesy: Disney)

As I stated above, simple and focused can get you really far down the road. We talked to a lot of people, and did a lot of research, however, most of the above is my opinion. After using APIs that claimed to be RESTful across the web, I realized that the most important thing for this effort would be to stick to simplicity and go pure REST -> ASP .NET MVC fit our needs and wants to get that done.

Automating - 7 Controller actions with Visual Studio templates

I am not big on repeating code or typing more than I have to. So, as I started adding entities to a RESTful API (which is built on .NET MVC) I quickly saw a need for automation. It seemed that I was almost always building out my controllers using the 7 standard actions (8 if you separate Delete and Destroy) that, say, a Rails application might automatically spin out - Index, Show, New, Edit, Create, Update, Delete and Destroy.

I wanted the elegance of the Rails controller code generation in a .NET application with out having to do much work - so I used Visual Studio templates to help me automate.

I wanted to produce this:


I began by building out a default controller exactly how I always do, with all of the actions, comments, etc...

Then I browsed to File Menu >> Export Template...


Select "Item template" and the project where the files is that you want to base your template on...


Choose the file (mine is the "PersonController")....


Choose the references (I rolled with System, System.Web, and System.Web.Mvc)...



Then set the name and description of the template...

Once you click finish, you're good to go...

Right Click on the folder where you want to create your new controller >> Add >> New Item...


You should be able to see your new template in the C# root (I moved mine from the root to C# >> Web >> MVC - by going to the templates directory and moving the new template zip into the proper folder).

That's it... now I am whipping out controllers for the API.

WS-Deathstar how I hate thee

Long ago, in a web far, far away Microsoft brought us .NET SOAP (JAVA the Hut was also part of the SOAP movement) propagating WS-* ("Deathstar") principles and practices.

The WSDL entered into SOA giving some developers strongly typed web service implementations rapidly. The WSDL was not a requirement to have a SOAP endpoint, but it made it really easy for .NET apps to consume .NET web services.

Anyone who has ever implemented an API and opened it up for public consumption knows that the interface contract typically rules the roost, so-to-speak. It impacts everything from when you version, what is open to the consumers, content returned by your API, and so on...

When the interface changes on your API, with out a solid versioning architecture, your consumers will, and do hurl curses at you and your API (and would probably seek alternatives).

So, back to the "Deathstar" - Years ago I dropped a .NET web service out into the smelting pot to be consumed by all. I made many mistakes but refined / fixed it as much as I could without killing my consumers - WS-* made it pretty easy to make "big mistakes." One such mistake that I made was that I never set the namespace on the service.

[WebService()] <--bad
[WebService(Namespace="http://sub.domain.suffix")] <--good

I basically forgot to do it, and once I realized it - my consumers we're eating tempuri soup and I was feeling a bit like a clown. There is no silver bullet to fix what is currently out there without forcing consumers to change code - I will live with that mistake there to remind me look before I leap.

lesson learned: Set my namespaces on my SOAP web services before they make it to production, or don't use the "Deathstar."