Grids and Minis

There are two opposing forces separating the hardware aspect of the computer industry - on one hand we have the Mac Mini showing that we do not need to have a (multiple) large ugly beige box(es) around the house. At the other end we have the ubiquitous computing platform from which all we ask is for storage and compute cycles.

In the household the ease of use, no wires, and seamless connectivity with all other appliances is going to be the driver for consumer purchases. Macintoshes are just so much easier to use and integrate into a digital lifestyle and for techies we can get to the internals of Unix with no problems - think remote management of Unix versus Windows. How long before we have remote management services for the home network?

On the corporate front I have gone from managing large internal data centers, to leasing rackspace, power and bandwidth to totally managed systems, where I request boxes on a demand basis and then install application software. Here we are still dealing with unique boxes and leasing boxes as a representation of computer resources. It is cheaper in most cases to lease boxes on demand rather than own them, but the real advantage will come when we break free of the computer box and have a true grid.

It is a question of degrees of freedom - with a box (even in a 1U form factor) the degrees of freedom are limited and the amount of management required for load balancing etc is too much. Installing software on multiple boxes and then configuring load balancing etc. should not be required. This is true whether we are in a home setting or using corporate resources - our thinking is constrained by physical boxes. The ideal situation is being able to inject software into a domain and have it managed on demand.

Today we have a one to one correspondence between boxes and resources. This dependency needs to be broken to create the next wave of innovation and ease of use. The question is whether the innovation will come from the hardware vendors or whether some smart software will abstract the hardware into a virtual grid is the question. Realistically both need to happen, however the software will probably come first as the hardware makers are too entrenched in shipping boxes to change.

Once this happens and it will the question will be how to access resources in the local and global grid - this is when things will get fun as there are no longer compute or storage boundaries - we are no longer navigating static information but information that is changing as we interrogate it and follow meaning provided by the compute resources. A bit far fetched? not really, I believe we are only a 5-7 years away from this in terms of software. The hardware will follow the software as the need to own hardware is going to go the way of owning power generators and telephone switches. It is perhaps time to start thinking about the software foundations of this new platform.

Is it the message or the transport

To make integration work the discussion seems to focus on the transport rather than the message. We are trying to convey meaning the transport is largely irrelevant providing both sides speak it and HTTP is the lowest common denominator.

Phil Windley as usual makes good points in his post ? On the death of SOAP | Between the Lines | ZDNet.com but I think the issue is in the white space between the transport - HTTP and the message - XML

If I may make a bold statement - between disparate systems (even similar systems ) the transport protocol is HTTP and the message format is XML, end of story. There are other use cases and approaches but they are edge cases and very domain specific and have propriety end points such as AS/2. For the foreseeable future the common transport is HTTP(S), protocol independence is a futile exercise and (IMHO) should be abandoned.

The real issue for most applications is which end is going to do the transformation into the format necessary to process the message. Whether the message is formatted as a SOAP or REST message is largely irrelevant to the end point that needs to do the transformation, the issues that need to be addressed are:

• Is the message understandable?
• Is there sufficient fidelity for understanding?
• Can the message be converted into the receiving format?
• Can the response be returned in an understandable format to the sender?

I am not sure REST or SOAP address these issues. A well documented schema and a well written HTML page probably contribute more to message integration than any WSDL or REST URI

This is because any standard needs to both exist for at least five years and be actively deployed for five years before it makes any impact on the industry as a whole. In my current environment we are integrating systems from Windows 3.1 to the present day. This is real life, and it will always be this way, legacy systems will always be in the majority. Interesting however the information flows are pretty constant, how they are described and formatted has not changed much - they have become formal and faster but the core information is the same.

The point is therefore the transport of the information needs to be the lowest common denominator while the ability to convey meaning must the focus not the protocol.

Impedance Reduction

One of the usual starting points for me in a design is the flow of information. That usually leads me to creating a set of XML Schemas for the information flow. This works well until I need to start developing code. In the past this has been a little tiresome to create (and recreate as the design evolves) the bridge between xml and the application.

This problem seems to be rapidly going away both in the Java and .Net environments, in Java I am using XMLBeans which once you get it installed and working provides a slick and easy way to map between schema and Java. In the .Net world I have been using a tool called xsd that is part of Microsoft .Net toolkit.

On the two approaches XMLBeans seems to be the most polished and functional but it is great to have similar functionality in both environments to improve productivity and reduce the drudgery.

REST and SOAP: Fitting them into your toolbox

Two appropriately cautionary posts about SOAP and REST raise some good issues,
Jon Udell: Don't throw out the SOAP with the bathwater and
Steve Vinoski: SOAP vs REST: stop the madness.

I have always had the metaphor of a toolbox where I keep the set of tools I need to do my job. Taking it a little further I have seen the standards process (both informal and formal) as the series of events that shape and separate the tools in the toolbox. This shaping and separation ensures that the tools can work together and overlap enough to solve most problems. The struggle I am having is where do these tools fit in my toolbox?. Jon Udell asks the question:

Why must we make everything into a rivalry? Why can't we just enjoy the best of all worlds?

Today I use both for different purposes but do I want to support both for all customers - this becomes an economic issue that significantly reduces the enjoyment. If I support one over the other, what do I say to a CIO about why one versus the other should be in our mutual toolbox. Is it clear when to use SOAP and when to use REST - if it is clear then we have the best of both worlds - until then we need to keep working on shaping our tools.

RESTful coding standards

In AJAX Considered Harmful Sam Ruby starts to lay out a set of useful coding standards for REST. He addresses Encoding and Idempotency, however going a little further has helped me. I have tried to apply the KISS principal and be a minimalist i.e. only add layers of complexity when you need them

Most use cases I have come across can be reduced to a CRUD interface (and yes there are exceptions, and exceptions prove the rule). The starting point is some standard naming conventions to create some degree of consistency for users for example:

• Create POST to http://xml.domain.com/resource/Create


• Read GET from http://xml.domain.com/resource?QueryString


• Update POST to http://xml.domain.com/resource/Update


• Delete POST to http://xml.domain.com/resource/Delete

The other issue is how to pass arguments and results back and forth. Some decisions are obvious, GET always passes a query string and the return data is always XML both using UTF8. I always have issues with POST though for invocations I recommend using XML always as it is hard on a user to have to switch between POST form parameters and XML for different methods. If XML is always used it keeps everything consistent. However when a delete method only requires a single key it is very tempting to just pass form parameter or even just do a GET with a query string. So far I have stayed with consistency - I would be interested in other opinions.

The other of standardization that helps is error returns, SOAP Faults are a good idea that REST can easily adopt. With a CRUD interface there are a simple set of standard errors that can be defined that come back with a HTTP 500 status code:

• Create: Item Already Exists User has no permissions ...


• Read: Item Does not Exist User has no permissions ....


• Update: Item Does not Exist Item Locked User has no permissions ...


• Delete: Item Does not Exist Item Locked User has no permissions ....

A simple REST Fault schema can be used to always return the same format such that common utility code can be created to handle errors in a uniform way. For errors to return permissions errors there needs to be the idea of identity which requires authentication

For authentication the lowest common denominator seems to work most of the time - interested to hear about real cases where it does not. If the request is over HTTPS and the authentication is HTTP-Auth then we have pretty good security. One step further would be to PGP the message but that would only be needed where the message needed to be highly secure and it traveled over unsecured pipes before and/or beyond the HTTPS stream.

I would be interested in feedback and extensions to this as it would be nice to have a consistent pattern for programming REST interfaces. The consistent pattern would enable us all to build interfaces faster and learn to use the work of others faster and easier.